Data validation and quality checks

To ensure that your design and analysis inputs are sound and reduce risks of data errors, Meridian GeoX has built-in features for double-checking your process. The library evaluates your data using two distinct modules: data validation checks and data quality checks.

Types of data checks

It's important to understand how the library handles different types of data issues:

  • Validation checks: These checks ensure your data adheres to certain requirements. If a validation check fails, the library raises a ValueError and halts execution.

  • Quality checks: These checks identify anomalies that might compromise result quality, but they don't stop the code from running. Instead, they log warning messages to the console and return details in a structured QualityCheckResult object.

Baseline validation checks

Before proceeding to specific algorithms, the library executes foundational validations on any input data:

  • Schema compliance: Data must contain date (no nulls), location (non-empty strings and no nulls), and conversions (no nulls). If spend is provided, values must be non-negative and non-null.
  • Positive conversions: The total sum of the conversions column across the dataset must be strictly greater than 0.
  • Data granularity: The data must follow a daily pattern. Weekly patterns are not supported and are blocked.

Design phase checks

When you run geox.run_design(), the library evaluates your pretest data to ensure it can generate well-powered study candidates.

Validation checks

The following validation checks apply during the design phase:

  • Pretest data length: The data must contain unique dates equal to at least three times the experiment_duration.
  • Geos availability: After removing any user-excluded geos, the remaining pool must contain at least 2 * (cell_count + 1) geos, where cell_count is the number of treatment cells.
  • Constraint overlaps: Excluded geographies must not overlap with control geographies that are forced to be included.
  • Alpha and power bounds: Both alpha and power values must be strictly between 0 and 1.
  • Experiment-specific requirements: Depending on your designated experiment type, the library enforces strict data and parameter requirements:
    • GO_DARK and HEAVY_UP: Your dataset must contain a daily spend column. For a single-cell study or the first cell in a multi-cell study, it expects a column named spend or spend_cell_1. For subsequent cells, it expects a corresponding column (for example, spend_cell_2).
    • HOLDBACK: Spend time series data is not required. However, you must ensure the cost per incremental conversion (cost_per_incremental_conversion) parameter is greater than 0. Note: The default cost_per_incremental_conversion is 1.0, so this validation only fails if you explicitly set it to 0 or negative, or if you omit a specific holdback cell when passing a multi-cell dictionary.
  • Max conversions: The max_conversions_percent must be strictly less than 0.5.

Quality checks

The following quality checks apply during the design phase:

  • Budget configuration warnings: The library warns you if your constraints don't align with your experiment type. For GO_DARK or HEAVY_UP, it expects a percentage change (budget_pct) and warns if an absolute budget is provided. For HOLDBACK, it expects an absolute budget and warns if a percentage is provided.
  • Isolating outliers: The library checks for geos that have spend greater than 0 but no recorded conversions. By default (exclude_geos_no_response=True), the library automatically excludes them from candidate splits.
  • Data sparsity: Warnings are logged if the fraction of missing conversion days exceeds 30%, or if missing spend days exceed 30% for active GO_DARK or HEAVY_UP treatment cells. A warning is also logged if overall zero-conversions exceed 50%.
  • High cardinality: A warning is logged if the unique geo count exceeds 500, as excessive granularity can usually contaminate lift estimates due to population flow across geos.
  • Duplicate entries: If multiple entries exist for the same date and location, a warning is logged and the entries are automatically aggregated.

Analysis phase checks

When executing post-test analysis using geox.analyze(), the library enforces strict synchronization with your original pretest study design.

Validation checks

The following validation checks apply during the analysis phase:

  • Geo set consistency: The locations in your uploaded analysis dataset must match the exact union of control and treatment geos established (after removing excluded geos) during the design phase.
  • Pretest duration: The pretest period in your analysis data must have a duration of at least three times the experiment_duration.
  • No overlap: The pretest end date (if set) must be strictly before the analysis start date.

Quality checks

During analysis, the library runs the following quality checks, confining its evaluation specifically to the pretest period to avoid post-treatment bias:

  • Isolating outliers: The library checks for geos that have spend greater than 0 but no recorded conversions.
  • Data sparsity: Warnings are logged if the fraction of missing conversion days exceeds 30%, or if missing spend days exceed 30% for active GO_DARK or HEAVY_UP treatment cells. A warning is also logged if overall zero-conversions exceed 50%.
  • High cardinality: A warning is logged if the unique geo count exceeds 500.
  • Duplicate entries: If multiple entries exist for the same date and location, a warning is logged and the entries are automatically aggregated.

Configure and view quality checks

You can control automated filtering behaviors using the QualityCheckConfig object and view the results of the evaluation using the QualityCheckResult object.

For a complete list of parameters, default thresholds, and returned data types, see the Data quality module in the API reference.

The following example demonstrates how to configure automatic filtering behaviors:

# Configure automatic filtering behavior
import meridian_geox as geox

quality_config = geox.QualityCheckConfig(
    # Automatically excludes geos with spend but no conversions (design phase
    # only)
    exclude_geos_no_response=True,
    exclude_outlier_dates=True,  # Automatically excludes outlier dates
)

# QualityCheckResult is returned as part of your design or analysis outputs.
# It contains:
# - quality_metrics (pd.DataFrame with metric, value, message, and threshold)
# - outlier_geos (Set of identified outlier locations)
# - outlier_dates (Set of identified outlier dates)