Reporting on experiments

AI-generated Key Takeaways

outlined_flag

• Treatment campaigns retain their own performance metrics throughout the experiment lifecycle and these metrics are never merged with the control campaign's metrics.

• Upon promotion or graduation, the control campaign preserves its original metrics while the treatment campaign also keeps its independent metrics.

• Both control and treatment campaigns can be individually reported on using standard reporting methods, and can be distinguished using the campaign.experiment_type field in queries.

• Draft campaigns do not accumulate any performance data and are excluded from standard search queries unless specifically included using the include_drafts=true parameter.

A treatment campaign does not copy previous metrics from the control campaign. It is instead treated as an entirely new campaign. During the experiment, metrics for the control campaign and the treatment campaign accrue separately; each one has its own impressions, clicks, etc. This does not change when either promoting or graduating the experiment. Metrics stay where they are and are never copied over to another campaign.

After promotion, the control campaign keeps all of its past metrics and goes forward with the new changes copied into it. The metrics from the treatment campaign are still associated with the treatment campaign after promotion.

After graduation, the control campaign and treatment campaign continue to exist as separate entities and each one keeps its own metrics for reporting.

You can report on all manifested campaigns in both control and treatment arms the same ways you could report on regular campaigns. You can differentiate a experiment campaign and a base campaign in your search query by selecting campaign.experiment_type, which will be BASE (for the control) or EXPERIMENT (for the treatment) to differentiate the type of campaign.