Tests
Experiments and budget tests that measure what ads actually caused.
Experiments
3
1 concluded · 2 running
Running
2
No verdict until the window closes
Significant results
1
Of 1 concluded · p<0.10 (90%)
Tests at a glance
| Test | Dates | Lift (90% range) | Decision |
|---|---|---|---|
| RunningMeta Android incrementality (DACH geo holdout)Meta Ads | Oct 8 → Nov 5 | — | Measuring until Nov 5 |
| RunningTikTok Spark Ads: creator hook vs product demoTikTok Ads | Oct 18 → Nov 1 | — | Measuring until Nov 1 |
| Read outApple Ads brand keywords: incremental or cannibalising?Apple Ads | Aug 25 → Sep 22 | +14% (+6% to +22%) | Real increment |
What we can conclude today
Read from spend that actually stopped. No estimate is shown unless the data supports one.
No incrementality conclusion yet
Spend is still running on the last observed day (Oct 26, 2026). No switch-off to read.
Budget tests
Stop rules agreed before the money is spent.
View only — owners and admins set up budget tests.
No budget test yet.
Design an experiment
Incrementality and geo holdout are what give you a causal answer. Design one and start it running.
geo holdoutRunningMeta Ads
Oct 8 → ongoingMeta Android incrementality (DACH geo holdout)
Hypothesis: Meta reports more Android installs in Germany, Austria and Switzerland than it causes. Hold out Austria to measure the real increment.
Measuring18d elapsed / 28d expected
No conclusion until significance and the interval settle. Calling it early reads noise as a result.
A/BRunningTikTok Ads
Oct 18 → ongoingTikTok Spark Ads: creator hook vs product demo
Hypothesis: A creator ordering coffee on day 7 lifts install rate by 20% over a screen-recorded product demo.
Measuring8d elapsed / 14d expected
No conclusion until significance and the interval settle. Calling it early reads noise as a result.
incrementalityConcludedApple Ads
Aug 25 → ended Sep 22Apple Ads brand keywords: incremental or cannibalising?
Hypothesis: Most brand-keyword installs would have come through organic search anyway.
Incremental lift (point estimate) +14.0%
Significantp = 0.01 (threshold 0.10)Modeled
+6% low+22% high
Uncertainty interval
Conclusion
A real increment: lift 14% (range 6% to 22%).
Design templates — three kinds of test
Pick by what you need to know. Closest to a true causal answer: geo holdout and MMM, then A/B.
A/BCreative / copySplit one audience across two variants and compare CVR and trial starts. This is a relative comparison inside one platform — it cannot tell you the incremental effect.
geo holdoutGeo holdoutTurn ads off in matched regions and measure the install difference against the regions still served. Does not depend on per-user attribution.
incrementality / MMMMedia mix (installs)Compares installs while a channel was funded against the days it was off, with an exact interval. A full mix model needs at least two channels varying independently; until then we read one channel at a time and say so.