Measurement Methodology
Incrementality Testing
Attribution tells you who gets credit. Incrementality tells you what actually worked. Most agencies only do the first. We do both - because the difference between 'attributed' and 'incremental' is where most wasted spend hides.
The uncomfortable question every brand should ask:
"How much of what Google claims to have driven would have happened anyway?"
The Attribution Problem
Google Ads claims credit for conversions using attribution models - last click, data-driven, or position-based. These models answer: "Which ad touchpoint should get credit for this sale?"
But they never answer the more important question: "Would this sale have happened without the ad?"
Brand searches are the clearest example. A customer who types your brand name into Google was already going to buy. The brand ad intercepts them - and the platform claims the conversion. Your agency reports a 12x ROAS on brand. Everyone celebrates. But nothing was actually created by the ad.
This isn't fraud. It's how attribution works. But if you're making budget decisions based on attributed performance alone, you're almost certainly overspending in some areas and underspending in others.
A Typical Discovery
What the platform reported
4.2x
Blended ROAS
£340k
Attributed Revenue
After incrementality testing
1.8x
Incremental ROAS
£145k
Incremental Revenue
57% of 'attributed' revenue would have happened without ads. The brand was overspending by £32k/month on non-incremental activity.
How We Test Incrementality
Four methodologies, each suited to different account structures and commercial questions.
Geographic Holdout
Pause all paid activity in a matched region. Compare sales trends against an active region with similar demographics and seasonality.
Best For
Brands with national coverage and consistent regional demand
Duration
4-6 weeks minimum
Measures
What percentage of 'attributed' sales would have happened anyway
Worked Example
A pet food brand paused Google Ads in the North East for 5 weeks. Platform attributed 1,200 orders/month to ads. During the holdout, 840 orders still came through organically. True incrementality: 30%, not 100%.
Campaign-Level Holdout
Pause a specific campaign type (e.g., Brand, PMax, Generic) while keeping everything else running. Isolate the true lift of that campaign.
Best For
Measuring the real value of brand campaigns or Performance Max
Duration
3-4 weeks
Measures
Whether pausing a campaign reduces total sales or just shifts attribution
Worked Example
A home furnishings brand paused Brand campaigns for 3 weeks. CPC costs dropped £8k/month. Revenue fell by only £2k. The brand campaign was cannibalising organic traffic at a cost of £6k/month.
Budget Scaling Test
Increase or decrease spend by a fixed percentage in a controlled period. Measure whether the marginal spend produces marginal profit.
Best For
Testing whether 'scaling' actually improves outcomes
Duration
2-4 weeks per increment
Measures
Marginal POAS - the profit generated by the last £1,000 of spend
Worked Example
A supplements brand increased daily budget by 25%. Revenue rose 12%. But COGS-adjusted profit fell 8%. The marginal spend was buying revenue at a loss - invisible in platform reporting.
Channel Isolation
Run one channel in isolation while pausing others. Understand true channel contribution without cross-attribution noise.
Best For
Multi-channel brands unsure which platform drives genuine demand
Duration
4-6 weeks
Measures
True channel contribution vs. platform-claimed attribution
Worked Example
A fashion brand running Google, Meta, and TikTok paused Meta for 4 weeks. Google's reported ROAS improved by 15% - not because Google got better, but because Meta was no longer claiming credit for the same conversions.
The 5-Phase Process
From baseline to reallocation in 10 weeks. Every test produces a CFO-ready output.
Week 1-2
Commercial Baseline
- Establish true P&L baseline (not platform P&L)
- Map current attribution claims vs. bank-reconciled revenue
- Identify the gap between reported and actual
- Define the commercial question the test will answer
Week 2-3
Test Design
- Select test methodology based on account structure
- Define control and test groups with statistical rigour
- Set minimum detectable effect thresholds
- Align test duration with business cycles (avoid peak/promo periods)
Week 3-8
Execution & Monitoring
- Implement test with clean controls
- Monitor for contamination (e.g., organic changes, competitor activity)
- Track both platform metrics AND commercial metrics simultaneously
- Weekly check-ins to ensure test integrity
Week 8-9
Commercial Analysis
- Reconcile platform data against actual revenue and margin
- Calculate true incremental contribution (not platform-attributed)
- Build diminishing returns curve for marginal spend
- Produce CFO-ready summary with P&L impact
Week 9-10
Reallocation
- Redirect budget from non-incremental to genuinely incremental activity
- Set new POAS targets based on proven incrementality
- Establish ongoing measurement cadence (quarterly re-testing)
- Document findings for board/finance review
Why Most Agencies Don't Do This
Incrementality testing is uncomfortable. It almost always reveals that a portion of 'performance' is actually platform-claimed organic activity. Brand campaigns with 12x ROAS often show 60-80% of conversions are non-incremental. Performance Max often absorbs credit from Shopping campaigns running alongside it.
For an agency whose fee is justified by attributed ROAS, this is a structural threat. Proving that 40% of spend isn't incremental means recommending a 40% budget cut - and a corresponding fee reduction.
We charge fixed fees specifically so this incentive conflict doesn't exist. When we find waste, we recommend cutting it. When we find incrementality, we recommend scaling it. The commercial alignment is clean.