Definition
A geo holdout is an incrementality test that turns ads off in selected geographies and compares outcomes versus similar geographies where ads remain on.
Example
You pause ads in two matched regions and compare sales to two regions where ads continue.
How to use it
- Match geos by baseline demand and seasonality; otherwise results are noisy.
- Use longer windows for longer sales cycles or higher conversion lag.
- Control for distribution changes, pricing changes, or competitor shocks.
- Use multiple matched pairs to reduce volatility from local events.
- Confirm media weight is large enough to create a measurable delta.
- Run a pre-period to confirm the geos track closely before the test.
Common mistakes
- Using too few regions and over-interpreting noise.
- Choosing geos with different baseline demand patterns.
- Letting organic or PR spikes in a single region distort outcomes.
- Changing budgets outside the test and masking the true lift.
Measured as
Measure Geo Holdout with a fixed attribution window, conversion event, and spend basis before comparing campaigns or creative tests.
Misused when
- Using too few regions and over-interpreting noise.
- Choosing geos with different baseline demand patterns.
- Letting organic or PR spikes in a single region distort outcomes.
- Changing budgets outside the test and masking the true lift.
Operator takeaway
- Match geos by baseline demand and seasonality; otherwise results are noisy.
- Use longer windows for longer sales cycles or higher conversion lag.
- Control for distribution changes, pricing changes, or competitor shocks.
- Use Geo Holdout only inside a stable attribution rule, conversion definition, and time window so campaign comparisons stay honest.
- If performance changes, check whether the metric moved for a real business reason or because the measurement setup changed underneath you.
Next decision
- Quantify the impact with Incrementality Lift Calculator if you need to turn the definition into an operating assumption.
- Read Attribution vs incrementality: what to trust, when, and how to test if the decision depends on interpretation, policy, or trade-offs beyond the raw formula.
Where to use this on MetricKit
Calculators
- Incrementality Lift Calculator: Estimate incremental conversions, incremental ROAS, and incremental profit from a holdout test.
Guides
- Attribution vs incrementality: what to trust, when, and how to test: A practical guide to attribution vs incrementality: common attribution models, window pitfalls, how MER/marginal ROAS fit in, and how to run holdout/geo tests.