Conversion Lift Test

Definition

A conversion lift test compares a control group (not exposed) with a test group (exposed) to estimate true incremental impact beyond attribution.

Example

Exposed users convert at 2.4% vs control at 2.0%, implying a 0.4-point lift.

How to use it

• Use holdouts when attribution is biased (retargeting, branded search).
• Define success metrics before the test and use enough duration for conversion lag.
• Balance sample sizes and check baseline conversion rates for parity.
• Lock creative and audience definitions to avoid leakage between test and control.
• Track both incremental conversions and incremental revenue if AOV differs by group.
• Confirm randomization and avoid overlap across experiments running at the same time.

Common mistakes

• Running tests without enough sample size to detect the target lift.
• Changing creative or offers mid-test and contaminating results.
• Comparing results without checking that control and test groups stayed balanced.
• Stopping early after a few good days and overestimating lift.
• Using mismatched attribution windows between control and test groups.

Why this matters

This term matters because it affects how you interpret performance and make budget decisions. If you use inconsistent definitions or windows, ROAS/CPA can look "better" while profit gets worse.

Practical checklist

• Write a 1-line definition for "Conversion Lift Test" that your team will use consistently.
• Keep the time window consistent (weekly/monthly/quarterly) when comparing trends.
• Segment results (channel/plan/cohort) before drawing big conclusions from blended averages.
• Use a calculator that references this term (e.g., Incrementality Lift Calculator) to sanity-check assumptions.
• Read the related guide (e.g., Attribution vs incrementality: what to trust, when, and how to test) for context and common pitfalls.

Where to use this on MetricKit