FAQs

What's the difference between GeoX and user-based lift measurement solutions?

Balancing measurement precision and user privacy, GeoX and user-based measurement solution methods rely on fundamentally different approaches:

• GeoX groups audiences by geographic regions (geos) and uses aggregate data, completely ignoring user-level attribution.
• User-based lift tracks individual cookies or ad impressions to attribute conversions.

Why should I use GeoX?

Using Meridian GeoX provides the following benefits:

• Cross-publisher incrementality measurement: By setting up multi-cell designs, GeoX allows the measurement of cross-publisher incrementality, such as Google versus non-Google media, which is usually not supported by publisher-specific user-based conversion lift.
• Offline and omni-channel measurement: GeoX is the primary methodology for measuring offline and omni-channel impact such as in-store sales and CRM-tracked conversions. Because it uses aggregate regional sales data, it avoids the difficulty of linking online user-level ad impressions to offline store transactions.
• Privacy-durable approach: GeoX relies on aggregate, unattributed conversions in a specific region rather than tracking individuals. Because of this, GeoX is immune to measurement gaps caused by modern privacy challenges such as cookie deprecation and iOS restrictions.

What's the difference between Meridian GeoX open source library and other Google GeoX open source software, such as time-based regression matched markets (TBRMM)?

The Meridian GeoX library is a comprehensive, unified upgrade that offers the following capabilities and features:

• One-stop shop: Acts as a single library for both study design (pre-test planning) and analysis (post-test inference), covering multiple methodologies. GeoX supports the time-based regression (TBR) methodology.
• Direct design comparison: Includes a compare_designs feature to directly compare different study designs side-by-side to see which best fits budget and minimum detectable effect (MDE) requirements.
• Modern statistical approach: Uses JAX-optimized computations for fast candidate generation and design-aware placebo inference, improving power and precision in confidence interval estimation.
• Multi-cell and flexibility: Natively supports multi-cell designs and allows flexible constraints like excluding specific geos or forcing geos to control.

Why does Meridian GeoX recommend a significantly higher budget and larger MDE compared to TBRMM?

The differences in recommendations stem from different design philosophies and validation techniques of TBRMM and Meridian GeoX:

• TBRMM estimates design power based on the pre-test in-sample fit. The same historical data is used to fit the model and estimate its variance. In ideal situations, the estimated variance using this procedure should converge to its true variance. However, in reality, the pre-test variance could often be underestimated compared to what is experienced during the actual test. When the actual post-test variance turns out to be higher, the study can end up underpowered. This historical optimization can therefore lead to optimistic or lower budget and MDE recommendations.
• Meridian GeoX addresses this by incorporating out-of-sample validation or honest data-splitting, and placebo testing during the design phase. By evaluating the design's stability on validation periods that weren't used for model fitting, it captures a more realistic estimate of the post-test variance, leading to more conservative budget and MDE recommendations.

Each method is valid under its own technical assumptions, but Meridian GeoX is designed to build in a buffer against more realistic post-design volatility, ensuring that experiments have high probability of delivering conclusive results.

Can I use net revenue which can include negative values due to refunds as the conversion metric?

No. While the schema validation doesn't explicitly block negative conversions and only checks that the total conversions is positive, negative conversion values can disrupt the candidate generation (greedy assignment) and statistical modeling. It's recommended to use non-negative, absolute values like gross revenue or gross conversion counts. If refunds are significant, run the analysis on gross sales and apply a historical net-to-gross ratio post-test.

Is campaign spend data always mandatory during the design phase, and how do I source it?

The campaign spend data requirement depends on your experiment type:

• Go-dark and heavy-up experiments: Daily geo-level spend data is mandatory to calculate baseline spending levels and project test budgets. The validation will fail if the spend column is missing for these experiment types.
• Holdback experiments: Optional for net-new campaign launches during the design phase. However, you should provide a cost per incremental conversion (CpIC) reference to estimate budget requirements.

The spend data you extract must be daily and aggregated at the same geographic level as your conversions, such as DMA, state, or postal code.

• Google Ads: Extract daily geo-spend using the Google Ads API through reports like campaign_location_target_report.
• Other publishers: Use publisher-specific APIs or reporting UI exports to retrieve daily spend by geographic target.

Ensure that geographic identifiers match exactly between your conversions and spend datasets. If you're using postal or ZIP codes, make sure preceding zeros are preserved as strings during extraction and export to prevent validation matching failures. For example, your postal or ZIP codes should be "02138" instead of "2138".

For more information, see Prepare your pretest data.

Our daily conversion data is highly volatile. Can we upload weekly data instead to smooth out the noise?

No. GeoX strictly requires daily time-series data. Weekly data isn't supported and will trigger validation errors. To handle daily volatility, consider:

• Extending the test duration, such as changing from four weeks to six or eight weeks to accumulate more data points potentially lowering MDE.
• Using a shallower conversion KPI, such as add-to-carts or sign-ups, that has higher volume and fewer zero-count days.
• Increasing the target treatment group size or maximum conversion percent.

Does the Meridian Geox library allow the ability to include more than one KPI?

No. The Meridian GeoX library only supports a single conversion KPI per run, which is the conversions column in the data schema. This means that you can't simultaneously optimize a design for multiple distinct metrics, such as:

• Segmented conversions, like tracking new and returning buyers as separate target KPIs in the same run.
• Different conversion actions, like tracking store visits and sales simultaneously, or sign-ups and purchases.

A separate design is required for each KPI, as each has its own historical volume, trend, and volatility. If you run the design engine separately for each KPI, they'll likely recommend different geo-splits and budgets.

Can we force a specific market to treatment or control group, and how does this impact study design?

• Force to control: You can specify geos to be forced into the control group using included_control_geos in Constraints.
• Exclude geos: You can exclude geos entirely from the experiment in scenarios such as protecting key revenue markets or avoiding media disruptions using excluded_geos in Constraints.
• Force to treatment: Not supported in the core library.

If there are no business specific reasons or outlier geos, forcing geos to control or excluding them restricts the randomization pool and may degrade model fit. This might result in a larger MDE to achieve statistical significance.

Are my geo units too few for GeoX to be effective?

You should include a minimum of 10 geos for a single-cell design.

• Use random sampling for 10-20 geos: In designs with fewer than 20 geo units, stratified sampling is less effective because clustering into strata is constrained. If stratified sampling doesn't yield a satisfactory design for you, you should design with a simpler random assignment.
• Use stratified sampling for >20 geos: When your design has few geos, statistical power is likely to be low, leading to high MDEs or requiring very large budgets. For optimal results, a larger number of geos—for example, ranging from 50 to more than 100—is recommended to allow effective stratification and robust matching.

What if I have a flexible budget and want to input a target iCPA or iROAS and get a required budget?

This depends on your experiment type:

• Holdback experiments: You input the target CpIC using cost_per_incremental_conversion in DesignConfig. If conversions represent revenue, CpIC is equivalent to 1/iROAS. The library then calculates the required budget to detect the MDE with the preferred power.
• Go-dark and heavy-up experiments: You don't input the target CpIC. However, the budget is determined by the historical spend in the selected treatment geos. You can apply a budget_pct constraint—for example, -100% for go-dark or a percentage increase for heavy-up experiments—and the library projects the design implied CpIC that you can detect with that budget.

Clarification on budget and MDE

The statistical MDE or the minimum percentage change in conversions that the experiment can detect is determined solely by the historical volatility and matching of the geos. Changing the budget doesn't alter this statistical limit.

While the MDE remains constant, the efficiency targets you can successfully measure do change with the budget:

• Higher budget: Lets you detect a higher, less efficient CpIC, or a lower, more conservative iROAS. This increases the likelihood of a conclusive test for campaigns with moderate performance.
• Lower budget: Restricts you to measuring campaigns with a very low CpIC or very high iROAS. If the actual campaign performance is moderate, the test may be inconclusive.

How should we decide if a design is viable other than required budget and MDE?

Evaluate the viability using the following:

• R-squared (\(R^2\)): \(R^2\) ≥ 0.8 is recommended. A low \(R^2\) such as less than 0.5 indicates the control geos don't predict the treatment geos well, making the post-test analysis unreliable.
• AA test p-value: The library filters designs using placebo simulations. Ensure the selected design passes the A/A test (p-value ≥\(\alpha\)) to minimize false positive risk.
• Feasibility of MDE: Ensure the projected MDE is business-viable, meaning you believe the campaign can actually achieve a lift larger than the MDE.

R-squared adjustment

The minimum required \(R^2\) is controlled by the min_r2 parameter in geox.DesignConfig. By default, the minimum required \(R^2\) value is 0.8. If the design search returns no viable candidates at 0.8, you have the flexibility to relax the threshold, such as adjusting it to 0.75 or 0.70:

      design_config = geox.DesignConfig(
        experiment_duration=datetime.timedelta(days=28),
        experiment_types=geox.ExperimentType.HEAVY_UP,
        methodology=geox.Methodology.TBR,
        min_r2=0.7,  # Adjust threshold if 0.8 yields no candidate designs
        )
    

If I'm running GeoX for the first time, what are the general guidelines to implement my experiment?

For first-time GeoX users, refer to the following general guidance:

• Select your setup pathway:
  • Platform UI (manual setup): Best for standard studies.
  • Platform editor (bulk setup): Best for complex or multi-campaign studies. This requires generating and importing an editor backup file.
  • Platform API (automated): Best for enterprise-level programmatic management. Note that ZIP code targeting isn't supported through the API.
• Determine campaign setup strategy:
• Modify directly (keep same campaign): Use when budgets are uncapped. Restricting geos directly keeps historical machine learning intact.
• Duplicate (copy-and-paste campaign): Mandatory when campaigns have capped daily budgets. Simply restricting geos on a capped campaign will cause the algorithm to spend the saved budget in the remaining control geos, contaminating the control group. Duplication isolates the budgets.
• Targeting configuration: Apply the geo splits strictly using presence targeting to ensure ads are only served in treatment geos and excluded from control geos.
• Plan for a cooldown period: After the test ends, maintain a 1-2 week cooldown period before returning to business-as-usual (BAU) to capture lagged conversions.

For detailed instructions on how to execute your experiment in Google, refer to the step-by-step campaign implementation guide in the Google Ads Help Center.

Can we keep user-level tests active on the same campaigns during a GeoX experiment?

No. All user-level measurement studies such as conversion lift and brand lift should be deactivated on the campaigns under evaluation during the experiment. User-level studies create hidden randomized holdouts within your geos, which contaminates the treatment and control split, dilutes statistical power, and biases the post-test analysis.

Does a multi-cell design allow me to test the statistical significance of the difference between two active treatment arms?

No. Results for each treatment cell are evaluated independently against the shared control. The library doesn't provide a direct statistical metric to test if the difference between two active treatment arms is significant.