The boring magic behind every lift number.

Marketing platforms report “conversions.” Most of those conversions would have happened anyway. The thing a CFO cares about is incremental revenue — what would not have happened without the campaign. That number is impossible to read off Meta Ads Manager, GA4, or a Klaviyo dashboard because none of them observe the counterfactual.

LiftProof is built around a single discipline: every number we publish answers a counterfactual question. We construct a credible “what would have happened” and report the gap to what actually did, with honest uncertainty around it.

User-level holdout tests require the ad platform to cooperate — Meta’s Conversion Lift, Google’s Conversion Lift. They’re excellent and they’re expensive (high spend minimums, single-platform, opaque under the hood).

A geo experiment sidesteps that. You expose some geographic markets to the campaign (treatment), leave the rest as-is (control / donor pool), and reconstruct the treatment group’s counterfactual from a weighted combination of the controls. Done right, you get a clean causal estimate that works for any channel that can be geo -targeted: paid social, direct mail, TV, podcasts, OOH.

The catch is the math. Picking which markets, which weights, and how to call significance is exactly what synthetic-control econometrics is designed for.

The synthetic-control estimator finds non-negative weights w over your donor (control) markets such that a weighted combination of donors tracks your treatment market in the pre-period. Solved as a constrained least-squares problem on the probability simplex (weights sum to 1, all non-negative).

Once the pre-period fit is established, you project the same weights forward into the post-period. That projection is the counterfactual — what the treatment group should look like under business-as-usual. The gap between the real treatment series and the synthetic series, after launch, is the causal lift.

DiD assumes parallel trends: in the absence of treatment, treatment and control markets would move in lockstep. That assumption is sometimes wrong, but it’s a useful sanity check. If our synthetic-control estimate and our DiD estimate disagree sharply, the model ensemble widens its uncertainty — that’s a signal, not a bug.

We use level-form DiD with uniform donor weights. It’s deliberately the simplest possible variant: if even DiD says lift is positive, the result is robust to model choice. If only SCM says it, the headline becomes more cautious.

No single counterfactual estimator is universally best. Picking one and hoping is a methodological gamble. Instead we hold out a slice of the pre-period as a placebo window, fit every candidate model on the rest, score each on the placebo, and blend the survivors by inverse-RMSE weights.

1. Split pre-period into train and placebo windows.
2. Fit each model on train. Score by RMSE against placebo.
3. Refit each model on the full pre-period.
4. Counterfactual = inverse-RMSE-weighted average of refits.

The ensemble’s confidence interval is the union of its components, not a re-derived clean band — we’d rather be honest about model disagreement than pretend the blend has tighter uncertainty than its inputs.

Synthetic-control effects don’t have a clean closed-form sampling distribution. Asymptotic z-tests are a lie when you have ~30 geos. So we do what Fisher recommended in 1935: build the null distribution by repeatedly reassigning treatment at random across your markets and recomputing the effect each time.

• 300 permutations by default. Seeded Mulberry32 RNG + Fisher–Yates shuffle — same input, same output, reproducible.
• Two-sided p-value with add-one smoothing: p = (1 + #extreme) / (1 + N).
• 95% confidence interval via randomization inversion — the range of true effect sizes that would not be rejected at α = 0.05.
• Empirical power proxy — a sanity check that flags underpowered tests before you announce them at all-hands.

• Every reported lift comes with its 95% confidence interval and p-value. No exceptions.
• If p ≥ 0.05, the result is labeled inconclusive. Not “trending,” not “directional.”
• Pre-period fit quality (R², MAPE) is shown next to every result. Bad fit means trust the number less — and the UI says so plainly.
• The AI Copilot explains practical significance, never statistical significance in isolation.
• Concurrent campaigns, holidays, outages: flagged on the analysis screen so you can decide whether the counterfactual is contaminated.
• Spillover risk between adjacent markets is surfaced with commuting-zone granularity.

MMM is a great long-horizon planning tool but it’s the wrong tool for “did this $40K launch pay back.” Managed holdout providers run essentially the same academic methods we do; their moats are a benchmark database and a human strategist, both of which we replace with an explainable Claude-powered Copilot.

Every model, every permutation, every weight is pure TypeScript in lib/analysis/. No R, no Python bridge, no Docker sidecar, no proprietary binary. You can read the code before you trust the number.

Have a methodology concern? Open a GitHub issue or write to methodology@steenocki.com. We respond, in public, with the relevant paper and the relevant lines of code.

• Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic Control Methods for Comparative Case Studies.
  Journal of the American Statistical Association 105 (490).
• Ben-Michael, E., Feller, A., & Rothstein, J. (2021). The Augmented Synthetic Control Method.
  Journal of the American Statistical Association 116 (536).
• Fisher, R. A. (1935). The Design of Experiments.
  Oliver & Boyd, Edinburgh.
• Card, D., & Krueger, A. B. (1994). Minimum Wages and Employment: A Case Study of the Fast-Food Industry.
  American Economic Review 84 (4).
• Meta Open Source (2021). GeoLift: an end-to-end geo-experimentation library.
  facebookincubator/GeoLift.
• Duchi, J., Shalev-Shwartz, S., Singer, Y., & Chandra, T. (2008). Efficient projections onto the ℓ1-ball for learning in high dimensions.
  ICML.