Incrementality Testing for AI Ad Campaigns

The short version: AI-automated campaigns like Advantage+ and Performance Max report the highest ROAS in your account and often deliver the least incremental revenue. They optimize toward the cheapest conversions, which are frequently buyers who would have purchased without the ad. Incrementality testing, withholding ads from a matched group and measuring the gap, is the only way to know how much revenue your paid media caused. Run it quarterly and before you scale any automated campaign.

The best-performing campaign in many ad accounts can mislead you. It reports a 9x return, so you feed it more budget, and blended ROAS barely moves. The campaign may be taking credit for sales that were already coming.

This is the core problem with AI-automated buying. Advantage+ and Performance Max are built to find the cheapest conversions. Those conversions often come from buyers closest to purchase. The algorithm serves them an ad, they buy, and the platform claims the sale. This post explains how we separate that reported number from real lift across our Paid Media work.

What Is Incrementality Testing?

Incrementality testing measures the revenue your ads actually caused, not the revenue the platform reports. You withhold ads from one group of users or markets, run them normally for a matched group, and compare the two. The revenue difference is your incremental lift.

The gap between the two numbers is where budgets go to die. Platform-reported ROAS is measured inside a closed system that has every incentive to keep your spend flowing. Research from Common Thread Collective found Google branded search carries a median incremental ROAS of just 0.27x. For every dollar spent, only 27 cents was incremental. The rest was buyers who were already going to convert.

Incrementality corrects for that. At the P&L level, it answers a direct question: if I turned this campaign off, how much revenue would I lose?

Why Do AI-Automated Campaigns Inflate Reported ROAS?

AI campaigns inflate reported ROAS because their objective and your objective are not the same. You want incremental sales. The algorithm wants cheap conversions. The cheapest conversions are people already near the buy button, so automation crowds toward existing demand and reports it as new.

Advantage+ can widen this gap compared with manual campaigns. It optimizes across a broad pool with no obligation to prove the sale was net-new. In Haus’s incrementality dataset, 58% of brands saw higher incremental ROI from manual campaigns than from Advantage+. Advantage+ over-reported by roughly 12 percentage points against its actual incremental delivery.

That does not mean you should turn off Advantage+. It often still wins. But you cannot rank campaigns by reported ROAS when some are AI-automated and others are not. You are comparing two different measurements.

Metric	Reported ROAS	Incremental ROAS
What it counts	Every conversion the platform attributes	Only sales the ad actually caused
Who reports it	The ad platform	A holdout test you control
Incentive to inflate	High, it keeps your budget flowing	None, it is a controlled experiment
Effect of AI automation	Rises, algorithm harvests existing demand	Unchanged, measures net-new revenue
Use it to	Optimize inside a campaign	Decide budget between campaigns

What Are the Main Incrementality Test Methods?

Three methods matter for most advertisers: geo holdouts, platform conversion lift, and public service announcement (PSA) tests. Each withholds ads from a control group in a different way. Geo holdouts are the most trusted for automated campaigns because they sit outside the platform’s own measurement.

A geo holdout splits your markets into a test group that sees ads and a control group that does not, then compares revenue. Conversion lift runs the same logic inside Meta or Google using a user-level holdout the platform manages. A PSA test shows the control group a charity ad instead of nothing, isolating ad exposure from audience quality.

Method	Best for	Main limitation
Geo holdout	Automated campaigns, cross-channel truth	Needs national or multi-region spend
Conversion lift	Single-platform reads, faster setup	Graded by the platform being tested
PSA test	Isolating exposure from audience quality	Costs budget on the control ad

For AI-automated campaigns, we lean on geo holdouts. When the platform runs the automation and also grades the homework, an external test gives you a cleaner read.

How Do You Run a Geo Holdout Test on an AI Campaign?

Split comparable markets into test and control, hold the automated campaign dark in the control markets, and run it normally in the test markets. Keep everything else equal, wait for the test to reach significance, then compare per-market revenue. The lift over control is your incremental result.

Careful setup makes the number trustworthy. Match your test and control markets on size, seasonality, and baseline demand before you start. A common failure is comparing your best market against your weakest and calling the difference lift.

Run these steps in order:

1. Pick 10 or more markets per group so the average is stable, not driven by one outlier.
2. Match the groups on historical revenue and trend, not just population.
3. Establish a clean pre-period baseline before any change.
4. Hold the control dark for the full window, usually 4 to 6 weeks.
5. Compare incremental revenue against incremental spend to get true incremental ROAS.

Then act on the gap. If a campaign reports 8x but tests at 2.8x, it is still profitable, but nowhere near where you were scaling it. That single correction reshapes the budget. It is the same discipline we bring to a full paid media audit, where reported numbers get pressure-tested against reality.

How Often Should You Re-Test Incrementality?

Re-test every quarter, and immediately after any major change: launching a new automated campaign type, shifting a large budget, or scaling a channel you suspect is coasting on brand demand. Incrementality is not a fixed property. It moves as auction competition, creative, and your demand mix change.

Treat the lift number like a financial control, not a one-time audit. A campaign that tested 3.5x incremental in Q1 can drift as it scales into thinner demand and its marginal buyers get less incremental. This is why we pair incrementality with the unit-economics view in our CAC benchmarks work: the two together tell you where the next dollar should go.

The test gives you a better budget map. You stop scaling campaigns that harvest demand and start funding the ones that create it. In an account full of AI automation, that distinction determines which campaigns deserve more budget.

If you want a clear read on how much of your paid media is actually incremental, Book a Free Strategy Call and we will map the test to your account.