Overview

Datadog Experiments helps teams run and analyze randomized experiments, such as A/B tests. These experiments help you understand how new features affect business outcomes, user behavior, and application performance, so you can make confident, data-backed decisions about what to implement.

Datadog Experiments consists of two components:

• An integration with Datadog Feature Flags for deploying and managing randomized experiments.
• A statistical analysis of Real User Monitoring (RUM), Product Analytics, and warehouse data to evaluate experiment results.

Getting started

To start using Datadog Experiments, configure at least one of the following data sources:

• Real User Monitoring (RUM) for client-side and performance signals.
• Product Analytics for user behavior and journey metrics.
• Data warehouse for running experiment analysis directly in your warehouse using Snowflake, BigQuery, Redshift, or Databricks.

After configuring a data source, follow these steps to launch your experiment:

1. Create a feature flag and implement it using the SDK to assign users to the control and variant groups. A feature flag is required to launch your experiment.
2. Create a metric to evaluate your experiment.
3. Create an experiment to define your hypothesis, choose an analysis method, and optionally calculate a sample size.
4. Launch your experiment to see the impact of your change on business outcomes, user journey, and application performance.

After your team has run several experiments against the same metric, use Cumulative Impact to see a noise-adjusted estimate of how much your experiments have moved that metric over time.

Further reading

Documentation, liens et articles supplémentaires utiles:

Measure the business impact of every product change with Datadog ExperimentsBLOG Feature FlagsDOCUMENTATION Product AnalyticsDOCUMENTATION