Definition
Incrementality asks: what would have happened without ads- The incremental lift is the difference between outcomes with ads and outcomes in a comparable no-ads baseline.
Why ROAS can mislead
- Attribution often claims credit for conversions that would have happened anyway.
- Retargeting can inflate ROAS by capturing already-intent customers.
- Cross-channel effects (brand, email, organic) make last-click comparisons noisy.
Common incrementality tests
- Geo holdout: pause ads in matched regions and compare outcomes.
- Audience split: randomize a subset of eligible users into a holdout group.
- Budget experiments: step changes in spend and measure response curves.
Practical checklist
- Pick a primary outcome (profit, conversions, or contribution margin).
- Run the test long enough to smooth weekly seasonality.
- Validate that holdout and exposed groups are comparable.
- Use the result to set ROAS/CPA targets and scale rules.
Test design QA
- Confirm tracking parity: events, values, and attribution windows match.
- Lock creative and targeting during the test window.
- Check for external shocks (promo, pricing, PR) before concluding.
- Validate conversion lag so results are not premature.