Meridian GeoX is now available. Run geo experiments to calibrate your MMM. Get started here

Set custom ROI priors using past experiments

Meridian provides automated prior calibration using past experiments with CalibrationBuilder to automate the translation of incrementality experiment results into ROI priors for paid media and reach & frequency channels. The builder accepts results from any experiment source, featuring direct integration for Meridian GeoX to streamline the workflow.

Calibration requires defining a Meridian prior, and incrementality experiment results are a strong way to drive that intuition. However, MMMs and experiments differ in important ways: specifically in their scope, timing and duration. To help you bridge this gap the CalibrationBuilder is designed to automatically account for these differences, translating your experiment results into robust priors.

CalibrationBuilder resolves these differences by automatically applying adjustments to the experiment results before merging them into a single calibrated prior distribution per channel.

The workflow consists of:

Registering experiments: User adds generic or Meridian GeoX experiment results to a channel.
Applying adjustments: CalibrationBuilder computes and applies spend, recency, and duration adjustments.
Merging distributions: CalibrationBuilder combines the adjusted distributions from all registered experiments using Bayesian updating.
Fitting parametric distributions: CalibrationBuilder fits a parametric distribution family (LogNormal, Truncated Normal, Gamma, or Normal) to the posterior to produce the final roi_m and roi_rf prior distributions.

Step 1: Registering experiments

You can register experiments using the CalibrationBuilder Python API. The builder supports two types of experiment sources:

Meridian GeoX Experiments: Registered using with_meridian_geox_experiment_result. Directly accepts the AnalysisResult object from Google's GeoX tool. CalibrationBuilder automatically extracts the point estimate, standard error, spend, and experiment dates from the AnalysisResult to compute adjustments.
Generic Experiments: Registered using with_incrementality_experiment_result. Useful for non-GeoX experiments where you manually specify the point estimate, standard error, spend, and dates.

For complete interactive code walkthroughs, see the TensorFlow Colab or JAX Colab.

Step 2: Applying calibration adjustments

An experiment and an MMM have different measurement goals and conditions. To bridge this gap, Meridian adjusts the experiment's point estimate (scaling it by a factor $\gamma$) and increases the standard error (adjusting variance by a factor $\tau$) to reflect the additional uncertainty of translating the experiment to the MMM.

The adjusted mean ($\mu_{adj}$) and standard deviation ($\sigma_{adj}$) for a single experiment are calculated as:

$$ \mu_{adj} = (\gamma_{duration} + \gamma_{user}) \cdot \mu_{exp} $$

$$ \sigma_{adj} = \sigma_{exp} \cdot \sqrt{1.0 + \tau_{spend} + \tau_{recency} + \tau_{duration} + \tau_{user} } $$

Where:

$\mu_{exp}$ is the experiment point estimate.
$\sigma_{exp}$ is the experiment standard error.
$\gamma_{duration}$ is the duration scaling factor (automatically computed; baseline is $1.0$ when the experiment captures 100% of the adstock effect).
$\gamma_{user}$ is an optional user-specified point estimate adjustment (default is $0$).
$\tau_{spend}$, $\tau_{recency}$, and $\tau_{duration}$ are the variance adjustments for spend, recency, and duration respectively (automatically computed; baseline is $0.0$ when no adjustment is needed).
$\tau_{user}$ is an optional user-specified standard error adjustment (default is $0$).

Automation and defaults

While $\gamma_{user}$ and $\tau_{user}$ allow manual fine-tuning and default to $0$, the other adjustments ($\tau_{spend}$, $\tau_{recency}$, $\tau_{duration}$, and $\gamma_{duration}$) are calculated automatically by CalibrationBuilder:

Meridian GeoX Experiments: The adjustment calculation is fully automated. CalibrationBuilder directly extracts $\mu_{exp}$ and $\sigma_{exp}$ from the GeoX icpd metrics (metrics.icpd.point_estimate and metrics.icpd.standard_deviation), the experiment spend from estimated_bau_spend (metrics.descriptive_metrics.estimated_bau_spend), and the start and end dates from analysis_config (analysis_config.analysis_start_date and analysis_config.analysis_end_date). Non-revenue KPI experiments are automatically converted to the revenue scale using the model's revenue_per_kpi. You don't need to manually supply spend, duration, or date parameters.
Generic Experiments: CalibrationBuilder calculates the adjustments using the identical formulas from the spend, start date, and end date arguments you provide in with_incrementality_experiment_result.
Model Defaults: Duration calculations use the CalibrationBuilder decay parameters, which default to adstock_decay_spec='geometric', alpha=0.5, and max_lag=8 (matching the ModelSpec defaults). Recency calculations use a fixed half-life of 52 weeks (1 year).

Spend adjustment

The spend adjustment ($\tau_{spend}$) accounts for the differences in spend scale and intensity. An experiment typically covers only a fraction of the total spend observed in the MMM window. To compare spend scale independently of experiment duration, Meridian compares the average daily spend of the experiment with the average daily spend of the channel across the modeled period.

For Meridian GeoX experiments, the experiment spend is extracted automatically from estimated_bau_spend in the GeoX AnalysisResult (under metrics.descriptive_metrics.estimated_bau_spend).

The spend adjustment is calculated as:

$$ \tau_{spend} = \frac{1.0 - r}{r} $$

Where $r$ is the spend ratio comparing average daily spend rates:

$$ r = \frac{ \min(\text{experiment_avg_daily_spend}, \text{channel_avg_daily_spend}) }{ \max(\text{experiment_avg_daily_spend}, \text{channel_avg_daily_spend}) } $$

Where:

\[\text{experiment_avg_daily_spend} = \frac{\text{experiment_total_spend}}{\text{experiment_duration_days}}\]
\[\text{channel_avg_daily_spend} = \frac{\text{total_channel_spend}}{\text{model_duration_days}}\]
If the experiment daily spend rate matches the channel daily spend rate ($r = 1.0$), then $\tau_{spend} = 0.0$ (no added uncertainty).
As the experiment spend rate becomes a smaller (or larger) fraction of the channel spend rate ($r \rightarrow 0$), $\tau_{spend}$ increases, which widens the prior distribution.
The spend ratio $r$ also accounts for differences in media granularity. For example, if you run a campaign-level experiment but model at the channel level in the MMM, the experiment spend rate will be compared to the total channel spend rate. If the campaign represents a large portion of the channel's activity, the campaign will be more representative of the channel and there is less uncertainty when translating the results. Conversely, if it is a small campaign, the adjustment will be larger to reflect the uncertainty of scaling that campaign's lift to the entire channel.

Recency adjustment

The recency adjustment gives more weight to more recent experiments as these may be more accurate to model up-to-date incremental outcomes. The further in the past the experiment was conducted relative to the end of the MMM modeling period, the less likely its results reflect the current business reality.

For Meridian GeoX experiments, the experiment end date is extracted automatically from analysis_config.analysis_end_date in the GeoX AnalysisResult.

The recency adjustment is calculated using an exponential decay with a half-life of 52 weeks (1 year):

$$ \lambda = 0.5^{w / 52.0} $$

$$ \tau_{recency} = \frac{1.0 - \lambda}{\lambda} $$

Where $w$ is the number of weeks passed between the experiment end date and the last modeled date in the MMM.

If the experiment ended at the last modeled date ($w = 0$), then $\lambda = 1.0$ and $\tau_{recency} = 0.0$.
For older experiments, $\lambda$ decreases, causing $\tau_{recency}$ to increase and add more uncertainty.
If the last modeled date falls before the experiment, no adjustment is applied ($\tau_{recency} = 0.0$).

Duration adjustment

The duration adjustment ($\gamma_{duration}$ and $\tau_{duration}$) accounts for the difference between the experiment's duration ($D$, in weeks) and the model's maximum lag window ($L$, in weeks).

For Meridian GeoX experiments, the experiment duration ($D$) is computed automatically from analysis_config.analysis_start_date and analysis_config.analysis_end_date in the GeoX AnalysisResult.

Because marketing effects carry over time (modeled by adstock decay), a short experiment might not capture the full, long-term effect of the media spend. Conversely, if the experiment is very long, it might capture effects beyond the model's truncated lag window.

Meridian calculates the capture proportion ($p$), which is the proportion of total adstock decay weights captured during the experiment's active weeks.

Let $w_s$ be the decay weights calculated using the adstock decay function (geometric or binomial) with decay rate $\alpha$ over the model window $L + 1$ (configured on CalibrationBuilder, defaulting to adstock_decay_spec='geometric', alpha=0.5, and max_lag=8):

If the experiment duration is shorter than or equal to the model window ($D \le L + 1$):
- Binomial and Geometric Decay: The capture proportion $p$ is the sum of the normalized decay weights up to $D$:

$$ p = \frac{\sum_{s=0}^{D-1} w_s}{\sum_{s=0}^{L} w_s} $$

If the experiment duration is longer than the model window ($D > L + 1$):
- Binomial Decay: Binomial weights decay to zero within the lookback window, so the experiment captures all possible effects: $p = 1.0$
- Geometric Decay: Geometric weights decay infinitely, so the numerator includes the tail of the geometric series up to $D - 1$:

$$ p = \frac{\sum_{s=0}^{L} w_s + \sum_{s=L+1}^{D-1} \alpha^s}{\sum_{s=0}^{L} w_s} $$

The capture proportion $p$ is bounded to a minimum value of $10^{-6}$ and a maximum value of $1.0$ (for binomial decay).

Using $p$, the duration adjustments are:

Point estimate scaling factor:

$$ \gamma_{duration} = \frac{1.0}{p} $$

If the experiment is shorter than the model window ($p < 1.0$), the point estimate is scaled up ($\gamma_{duration} > 1.0$) to estimate the full long-term effect. If the experiment is longer ($p > 1.0$ for geometric decay), the point estimate is scaled down ($\gamma_{duration} < 1.0$).

Standard error adjustment:

$$ \tau_{duration} = \max\left(\frac{1.0 - p}{p}, 0.0\right) $$

If the experiment is shorter ($p < 1.0$), add uncertainty ($\tau_{duration} > 0.0$) because the unobserved tail of the effect is extrapolating. If the experiment is longer or equal ($p \ge 1.0$), no additional uncertainty is added ($\tau_{duration} = 0.0$).

Step 3: Merging experiments

If you register multiple experiments for a single channel, Meridian combines them (along with an optional baseline_prior) using Bayesian updating.

By Bayes' theorem, the posterior density of the channel ROI is proportional to the product of the baseline prior and the likelihoods of all independent experiments:

$$ p(\text{ROI} \mid \{\mu_{adj,i}, \sigma_{adj,i}\}) \propto \pi(\text{ROI}) \prod_{i=1}^N L_i(\text{ROI}; \mu_{adj,i}, \sigma_{adj,i}) $$

Where the likelihood function for each experiment $i$ is:

$$ L_i(\text{ROI}; \mu_{adj,i}, \sigma_{adj,i}) = \frac{1}{\sqrt{2\pi}\sigma_{adj,i}} \exp\left(-\frac{(\mu_{adj,i} - \text{ROI})^2}{2\sigma_{adj,i}^2} \right) $$

Note that we can think of this product definition as its own Bayesian model. Mathematically, we treat each adjusted experiment as a noisy observation of the true channel ROI. The model is formulated as:

$$ \begin{align*} \mu_{adj,i} &\sim \text{Normal}(\text{ROI}, \sigma_{adj,i}^2) \quad \text{for each experiment } i \\ \text{ROI} &\sim \pi(\cdot) \end{align*} $$

Where:

$\mu_{adj,i}$ is the adjusted point estimate for experiment $i$.
$\sigma_{adj,i}$ is the adjusted standard error for experiment $i$.
$\text{ROI}$ is the true, unknown ROI parameter to be estimated for the channel.
$\pi(\cdot)$ is the baseline_prior distribution. If no baseline prior is specified, an improper flat uniform prior over positive real numbers is used ($\pi(\text{ROI}) \propto 1$ for $\text{ROI} > 0$).
$N$ is the number of registered experiments.

The builder evaluates this posterior density numerically on an adaptive parameter grid to find the combined distribution. See the Technical Appendix for more details.

Step 4: Fitting and building the calibrated prior

Finally, Meridian fits a standard parametric distribution family (Truncated Normal, LogNormal, Gamma, or Normal) to the numerical posterior distribution computed in Step 3. Candidate distribution families are chosen based on the baseline_prior: Truncated Normal is evaluated for baseline priors with non-negative support (the default), whereas Normal is evaluated when the baseline prior permits negative support. In either case, LogNormal and Gamma are also evaluated if the posterior mass on negative values does not exceed 10%. The best-fitting distribution shape is chosen by minimizing the cross-entropy loss (equivalent to minimizing the KL divergence) against the numerical posterior.

Once the optimal prior distribution is found for each calibrated channel, CalibrationBuilder wraps them. For any channels that did not undergo calibration, CalibrationBuilder assigns the custom prior provided in custom_prior. The default ROI prior of $\text{LogNormal}(0.2, 0.9)$ is only used if a custom prior is not provided for that channel.

The final output of build() is a CalibratedPriors dataclass containing:

priors.roi_m: A CalibratedDistribution (subclass of IndependentMultivariateDistribution) representing the calibrated prior for paid media channels.
priors.roi_rf: A CalibratedDistribution (subclass of IndependentMultivariateDistribution) representing the calibrated prior for reach & frequency channels.

These calibrated distributions are passed to PriorDistribution and then to ModelSpec to initialize the Meridian model.

Technical Appendix

This section provides the mathematical formulations and algorithmic details underlying the CalibrationBuilder numerical posterior computation and distribution fitting.

Posterior Distribution Formulation

For a given channel, let $\theta$ represent the true channel ROI. Observe $N$ independent adjusted experiment results. For each experiment $i \in \{1, \ldots, N\}$, there is an adjusted point estimate $\mu_{adj,i}$ and an adjusted standard error $\sigma_{adj,i}$.

Assume each adjusted point estimate is a normally distributed observation around the true ROI:

$$ \mu_{adj,i} \mid \theta \sim \text{Normal}(\theta, \sigma_{adj,i}^2) $$

The likelihood function for the $i$-th experiment is:

$$ L_i(\theta; \mu_{adj,i}, \sigma_{adj,i}) = \frac{1}{\sqrt{2\pi}\sigma_{adj,i}} \exp\left(-\frac{(\mu_{adj,i} - \theta)^2}{2\sigma_{adj,i}^2} \right) $$

If a baseline_prior is provided, denote its probability density function as $\pi(\theta)$. If no baseline prior is provided, use an improper flat uniform prior over positive values, $\pi(\theta) \propto I_{\{\theta > 0\}}$, where $I$ is the indicator function.

By Bayes' theorem, the posterior density of the channel ROI, $p(\theta \mid \{\mu_{adj,i}, \sigma_{adj,i}\})$, is proportional to the product of the prior and the likelihoods of all independent experiments:

$$ p(\theta \mid \{\mu_{adj,i}, \sigma_{adj,i}\}) \propto \pi(\theta) \prod_{i=1}^N L_i(\theta; \mu_{adj,i}, \sigma_{adj,i}) $$

Grid-Based Numerical Integration

Since the posterior distribution $p(\theta \mid \{\mu_{adj,i}, \sigma_{adj,i}\})$ may not have a standard analytical form (such as when using non-Normal baseline priors like LogNormal or Gamma), Meridian evaluates it numerically on a grid.

Scouting Pass: Meridian first performs a "scouting" pass to locate the region of high probability mass. It defines a broad grid centered around the empirical mean of the distributions and finds the range containing $99.8\%$ of the cumulative probability (from the $0.1$-th percentile to the $99.9$-th percentile).
Fine Grid Evaluation: It then constructs a fine grid of 10,000 equally spaced points over this resolved range $[\theta_{\text{min}}, \theta_{\text{max}}]$ and evaluates the unnormalized posterior density at each point.
Normalization: The density is normalized using Riemann sum integration:

$$ p(\theta_j \mid \{\mu_{adj,i}, \sigma_{adj,i}\}) = \frac{u(\theta_j)}{\sum_k u(\theta_k)\Delta\theta} $$

where $u(\theta)$ is the unnormalized posterior density, $\theta_j$ is the $j$-th grid point, and $\Delta\theta$ is the step size.

Distribution Fitting

Once the numerical posterior PDF is computed on the grid, Meridian fits a parametric distribution $q(\theta; \phi)$ (where $\phi$ represents the distribution parameters) to approximate the posterior.

Meridian determines candidate distribution families based on the support of the baseline_prior:

Non-negative support (Default): If no baseline_prior is provided (which defaults to non-negative support) or if the baseline_prior restricts support to non-negative values (such as LogNormal or Gamma), Meridian considers Truncated Normal (left-truncated at zero: $\text{low}=0$, $\text{high}=\infty$, restricting support to $[0, \infty)$) and does not consider Normal.
Negative support: If the baseline_prior assigns finite probability density to negative values (such as Normal), Meridian considers Normal (defined on $(-\infty, \infty)$) and does not consider Truncated Normal.
LogNormal and Gamma: For both cases, LogNormal ($(0, \infty)$) and Gamma ($(0, \infty)$) are also evaluated as candidate families provided that the fraction of posterior probability mass in the negative region ($\theta < 0$) does not exceed $10\%$.

For each candidate family, Meridian finds the parameters $\phi$ that minimize the cross-entropy loss between the numerical posterior $p(\theta)$ and the candidate distribution $q(\theta; \phi)$:

$$ \text{Loss}(\phi) = -\sum_j p(\theta_j) \log q(\theta_j; \phi) \cdot \Delta\theta $$

Minimizing this loss is mathematically equivalent to minimizing the Kullback-Leibler divergence $D_{KL}(p \parallel q)$ from the approximation $q$ to the true numeric posterior $p$.

Meridian uses the L-BFGS-B optimization algorithm to find the optimal parameters. The distribution family that achieves the lowest loss is selected as the final calibrated prior for the channel.

Calibrate treatment priors

Set custom priors when outcome is not revenue

Set custom ROI priors using past experiments Stay organized with collections Save and categorize content based on your preferences.

Step 1: Registering experiments

Step 2: Applying calibration adjustments

Automation and defaults

Spend adjustment

Recency adjustment

Duration adjustment

Step 3: Merging experiments

Step 4: Fitting and building the calibrated prior

Technical Appendix

Posterior Distribution Formulation

Grid-Based Numerical Integration

Distribution Fitting

Set custom ROI priors using past experiments