This section applies theory and formulas to published statistics, and
runs a controlled computational experiment, to size the problem. It does
not report new field data. Every input is either a published statistic,
cited to its source, or a stated assumption, and the assumptions are
listed so that a reader can change them. The methods are described in
Section 3.1 and the results can be reproduced from the stated
parameters.
5.1 Formulas
Let R be the reported (attributed) return on ad spend, s the share of
attributed sales that are incremental, m the contribution margin, and L
the lift, meaning the percentage increase in conversions among exposed
users compared with a randomised control.
F1. Incremental return: iROAS = R multiplied by
s. The over-attribution ratio is R divided by iROAS, which equals 1
divided by s.
F2. Break-even incremental return: 1 divided by
m. The break-even share of attributed sales that must be incremental is
1 divided by (R multiplied by m).
F3. Profit per pound of spend: (iROAS multiplied
by m) minus 1.
F4. Incremental share implied by a lift: s = L
divided by (1 plus L). If every conversion among exposed users is
credited to the advertisement, the over-attribution ratio is (1 plus L)
divided by L.
F5. Sample size per group for a two-group test:
n = 2 multiplied by (z for significance plus z for power) squared
multiplied by CV squared, divided by L squared, where CV is the
coefficient of variation of the outcome. For a conversion outcome with
base rate p, CV squared is (1 minus p) divided by p. The values used are
5 per cent two-sided significance and 80 per cent power, so the sum of
the two z values is 2.80.
F6. Dorfman and Steiner condition: with
advertising elasticity e and contribution margin m, the
profit-maximising advertising-to-sales ratio is e multiplied by m, and
the marginal profit return per pound of advertising is (m multiplied by
e) divided by the advertising-to-sales ratio (Dorfman and Steiner,
1954).
5.2 What published
statistics imply
Gordon, Moakler and Zettelmeyer (2023) report, for 663 Facebook
experiments, median randomised lifts of 28, 19 and 6 per cent at the
upper, middle and lower funnel, and median absolute differences between
non-experimental and experimental lift estimates of 115, 107 and 62
percentage points. Applying F4 and simple division gives Table 1.
| Upper |
28% |
115 points |
4.1 |
21.9%, ratio 4.6 if all credited |
| Lower |
6% |
62 points |
10.3 |
5.7%, ratio 17.7 if all credited |
Table 1. Calculations from published medians (Gordon, Moakler and
Zettelmeyer, 2023)
Two results stand out. First, the error of non-experimental methods
grows relative to the true lift as the funnel narrows: the median error
is about 4 times the lift at the upper funnel and about 10 times at the
lower funnel, where the purchase is closest and the true lift is
smallest. Second, at a 6 per cent median lift only 5.7 per cent of the
conversions that follow exposure were caused by it. A rule that credits
every conversion following exposure would therefore over-attribute by a
factor of about 17.7 at the median lower-funnel outcome. Real
attribution windows and rules are narrower than this extreme case, so
17.7 should be read as an upper bound on the logic and not as a typical
dashboard error.
5.3 Break-even and
sensitivity
Table 2 applies F1 to F3 to a reported return of 4.0 and a 40 per
cent margin, so that break-even is an incremental return of 2.5 and a
break-even incremental share of 62.5 per cent.
| 10% |
0.4 |
10.0 |
minus 0.84 |
| 30% |
1.2 |
3.3 |
minus 0.52 |
| 62.5% (break-even) |
2.5 |
1.6 |
0.00 |
| 100% |
4.0 |
1.0 |
plus 0.60 |
Table 2. Reported return of 4.0 at a 40 per cent margin
Figure 3. Profit per pound of spend by incremental share, at
three margins
The break-even share depends on both the reported return and the
margin (Table 3). A reported return of 4.0 needs at least 62 per cent of
attributed sales to be incremental at a 40 per cent margin. A reported
return of 2.0 at a 40 per cent margin needs more than 100 per cent,
which is impossible, so that spend loses money even if every attributed
sale is incremental. Reported returns that look healthy are often below
break-even once margin is applied, before any question of incrementality
is asked.
| 2.0 |
167% (impossible) |
125% (impossible) |
100% (all sales) |
| 4.0 |
83% |
63% |
50% |
| 8.0 |
42% |
31% |
25% |
Table 3. Share of attributed sales that must be incremental for
spend to break even
5.4 Can the gap
be measured? Power analysis
Because incremental lifts at the lower funnel are small, tests need
very large samples. Table 4 applies F5 for two outcomes: a conversion
outcome with a 2 per cent base rate, and revenue per person with a
coefficient of variation of 10, the order of magnitude reported by Lewis
and Rao (2015) for individual-level sales.
| 1% |
7.7 million |
15.7 million |
| 5% |
308,000 |
628,000 |
| 10% |
77,000 |
157,000 |
| 28% |
9,800 |
20,000 |
Table 4. Users needed per group for 80 per cent power at 5 per
cent significance
Figure 4. Required sample size against the lift to be
detected
The minimum detectable lift for a revenue outcome with CV of 10 is
about 12.5 per cent with 100,000 users per group, 4.0 per cent with one
million and 1.25 per cent with ten million. Expressed as a return,
suppose mean revenue per person is 50 pounds, so the standard deviation
is 500 pounds, and the advertising costs 0.50 pounds per exposed person.
The 95 per cent confidence interval on incremental return is then about
17.5 return points wide with 100,000 users per group, 5.5 points wide
with one million and 1.75 points wide with ten million. This agrees in
kind with Lewis and Rao's finding that the median confidence interval on
return was over 100 percentage points wide. A single small test can
therefore fail to separate a profitable return from an unprofitable one,
which is a reason to plan sample size in advance, use geo designs and
pooled evidence, and treat a dashboard return with no interval as an
unsupported number.
5.5 A
computational experiment on selection bias
To show how a reported return can diverge from an incremental one
when exposure is targeted, a simulation was run with stated parameters.
Users are of two types. A high-intent group (10 per cent of users, such
as people searching for the brand) buys with probability 20 per cent
without advertising. The remaining 90 per cent buy with probability 1
per cent. Advertising is targeted: 60 per cent of high-intent users and
10 per cent of other users are eligible to see it. Ten per cent of all
users are held out at random and never exposed. The true effect of
exposure is a 6 per cent relative increase in purchase probability, the
median lower-funnel lift in Table 1. Three estimates were compared: the
true lift; a randomised holdout among eligible users; and a comparison
of exposed users with users who were never eligible, which is a common
observational method.
| True lift |
6.0% |
Assumed effect of exposure |
| Exposed versus never-eligible, expected estimate |
381% |
Overstates the true lift about 64 times, because exposed users are
mostly high-intent |
| Over-attribution ratio if all exposed conversions are credited |
17.7 |
(1 plus 0.06) divided by 0.06 |
Table 5. Expected results of the selection-bias
simulation
The observational comparison finds a lift of 381 per cent where the
true lift is 6 per cent. Nothing in the setup makes the advertising more
powerful. The entire gap comes from who is targeted, which is the
selection mechanism described by Lewis and Rao (2015) and Gordon et al.
(2019). The randomised holdout recovers the truth on average, but a
single test is noisy. Repeating the holdout 2,000 times at three
population sizes gave Table 6.
| 200,000 |
6.4% |
minus 6.2% to 20.8% |
18% |
| 20,000,000 |
6.0% |
4.8% to 7.4% |
100% |
Table 6. Results of 2,000 repeated holdout tests (10 per cent
holdout)
Figure 5. Selection-bias simulation: expected estimates (left)
and spread of repeated holdout tests (right)
Two lessons follow. An observational comparison can be wrong by an
order of magnitude even when advertising works exactly as assumed, and a
randomised test fixes the bias but needs enough users. At 200,000 users
the holdout had only an 18 per cent chance of detecting a real 6 per
cent lift, and the 95 per cent range of estimates ran from about minus 6
to plus 21 per cent. This is a stylised setting with assumed parameters,
so the figures illustrate mechanisms and are not estimates of any firm's
performance.
5.6 An economic theory
check
Dorfman and Steiner's (1954) condition gives a second, independent
check on whether a given level of advertising can pay. Table 7 applies
F6 at a 40 per cent margin. It shows the marginal profit return per
pound of advertising for different advertising elasticities and
advertising-to-sales ratios. A value below 1 means that the next pound
spent returns less than a pound.
| 0.01 |
0.4% |
0.20 |
0.08 |
0.04 |
| 0.05 |
2.0% |
1.00 |
0.40 |
0.20 |
Table 7. Marginal profit return per pound of advertising at a 40
per cent margin (Dorfman and Steiner, 1954)
Shapiro, Hitsch and Tuchman (2021) report a median television
advertising elasticity of about 0.01 across 288 brands. At that
elasticity the profit-maximising advertising-to-sales ratio is 0.4 per
cent, and a brand spending 5 per cent of sales on advertising earns only
about 8 pence of margin per extra pound. This is consistent with their
finding that marginal return on investment was negative for more than 80
per cent of brands. The theory also shows the way out. At an elasticity
of 0.10, spending up to 4 per cent of sales pays. Elasticities differ
greatly by brand, channel and context, so the conclusion is not that
advertising should be cut but that its elasticity has to be measured
before the budget is set.
5.7
Consistency check on the Multiplier Effect
WARC (2025) reports that moving from performance-only to a mixed
approach raised revenue return by 25 to 100 per cent, averaging about 90
per cent, and that the reverse move cut return by about 40 per cent on
average. These figures can be checked against each other. If a change
multiplies return by 1.9, reversing it divides by 1.9, a fall of 47 per
cent. An uplift range of 25 to 100 per cent implies a reverse fall of 20
to 50 per cent. A 40 per cent fall corresponds to an uplift of about 67
per cent, inside the reported range. The figures are therefore mutually
consistent, although this says nothing about whether the underlying
models are right, which only experiments can show.
5.8 Summary of the
quantitative findings
Reported returns must be compared with break-even, not with 1. A
reported return below 1 divided by the margin, which is 2.5 at a 40 per
cent margin, loses money even if every attributed sale is
incremental.
If lifts at the lower funnel are as small as the published median
of 6 per cent, then at most about 6 per cent of conversions among
exposed users were caused by the advertising, and credit rules that
award all such conversions overstate by up to 17.7 times.
Selection alone, with no advertising effect beyond the assumed 6
per cent, produced an observational lift estimate about 64 times too
high.
Measuring small lifts needs hundreds of thousands to millions of
users per group, so tests must be planned with a power calculation, and
single underpowered tests should not decide budgets.
Economic theory agrees with the empirical studies: at low
advertising elasticities, typical advertising-to-sales ratios cannot
pay, and the elasticity is the quantity to measure.