Sample Size for Two-Way ANOVA: How Many Replicates Per Cell?
Two-way experiments are usually planned around the number of cells: two temperatures times three materials is six combinations, so run each one twice or three times. That's a reasonable starting point, but it leaves a question unanswered: is that enough to detect the effects you care about, especially the interaction?
Often it isn't. Interactions are harder to detect than main effects of the same size, and the replication that comfortably detects a main effect can leave the interaction test with very little power. This article explains what determines sample size in a two-way design, gives reference tables, and works an example where the interaction drives the decision.
Three tests, three sample sizes
A two-way ANOVA tests three effects, and each has its own power. For a given number of replicates per cell:
Main effect of A is tested on a − 1 numerator df, using all observations at each level of A.
Main effect of B is tested on b − 1 numerator df.
Interaction is tested on (a − 1)(b − 1) numerator df, and depends on differences between individual cells.
The required replication is the largest of the three, for whichever effects you actually need to detect. If the interaction matters (and in most process experiments it does), plan for it.
Effect size: Cohen's f for each term
Effect size for each term is expressed as Cohen's f:
f = σ_effect / σ
where σ is the within-cell standard deviation and σ_effect is the standard deviation of that term's effects:
For factor A: the SD of the A-level effects (each A-level mean minus the grand mean), computed with a in the denominator.
For factor B: the same for B.
For the interaction: the SD of the interaction residuals across all ab cells, where each residual is cell mean − row mean − column mean + grand mean.
Cohen's general benchmarks are f = 0.10 (small), 0.25 (medium), and 0.40 (large). In engineering work, compute f from the differences you actually need to detect and a realistic σ.
Reference tables
Replicates per cell needed for 80% power at α = 0.05, for each term at a given f:
f = 0.25 (medium)
Design A B A × B
2 × 2 32 32 32
2 × 3 22 27 27
2 × 4 16 23 23
3 × 3 18 18 22
f = 0.40 (large)
Design A B A × B
2 × 2 13 13 13
2 × 3 9 11 11
2 × 4 7 10 10
3 × 3 8 8 9
These are per cell. A 2 × 3 design needing 11 per cell needs 66 runs in total.
Note that these tables compare terms at the same f. The catch is that a physically similar change gives the interaction a much smaller f than it gives a main effect, as the next section shows.
Why interactions need more data
Take a 2 × 2 design with within-cell SD σ.
A main effect of size d: factor A raises the response by d at both levels of B. Cell means: 0, 0, d, d. The A effects are ±d/2, so f_A = d / (2σ).
An interaction of size d: factor A raises the response by d at one level of B and does nothing at the other. Cell means: 0, 0, 0, d. The interaction residuals are ±d/4 in every cell, so f_AB = d / (4σ).
The interaction's f is half the main effect's f for the same physical change, d. Since required sample size scales roughly with 1/f², detecting an interaction of a given size takes about four times as many observations as detecting a main effect of the same size.
That's the most important planning fact for two-way experiments.
What typical designs actually achieve
Power for a 2 × 3 design at α = 0.05:
Replicates per cell A, f = 0.25 A×B, f = 0.25 A, f = 0.40 A×B, f = 0.40
2 11% 9% 22% 15%
3 16% 12% 35% 25%
5 26% 19% 56% 44%
With three replicates per cell, a large interaction (f = 0.40) is detected only a quarter of the time. A non-significant interaction from a design that size says very little about whether the factors are additive.
Worked example
An engineer plans a 2 × 3 experiment: two mold temperatures and three hold pressures. Historical data put the within-condition SD of part shrinkage at σ = 2 (in units of 0.01%). The engineer wants to detect this scenario:
P1 P2 P3
T1 50 52 54
T2 50 52 57
Pressure raises shrinkage in both rows, and at T2 the highest pressure adds an extra 3 units. That extra 3 is the interaction.
Compute the interaction residuals. Grand mean = 52.5. Row means: 52.0, 53.0. Column means: 50.0, 52.0, 55.5.
Residual for T1, P3: 54 − 52.0 − 55.5 + 52.5 = −1.0 Residuals across all six cells: +0.5, +0.5, −1.0 (row T1) and −0.5, −0.5, +1.0 (row T2).
Compute f for the interaction. SD of residuals = √[(4 × 0.25 + 2 × 1.0) / 6] = √0.5 = 0.707. f_AB = 0.707 / 2 = 0.354
Look up n. For a 2 × 3 interaction at f = 0.354, 80% power: 14 replicates per cell (84 runs). For 90% power: 18 per cell (108 runs).
For comparison, the pressure main effect in this scenario has f = 1.14 and needs only 3 per cell for 80% power. A design of three replicates per cell would reliably detect that pressure matters and would have little chance of detecting the interaction, which is the finding that actually determines the best operating point.
Getting σ
The calculation is only as good as the σ estimate. In order of preference:
A control chart or capability study on the same characteristic.
Residual MS from a previous experiment on the same process.
A small pilot run at one condition, replicated five to ten times.
Underestimating σ produces an underpowered experiment. Round up.
Ways to need fewer runs
Reduce σ. Better measurement, tighter control of uncontrolled variables, or blocking on a known nuisance factor (day, lot, shift). Halving σ cuts the required replication roughly fourfold.
Use two levels per factor. A 2 × 2 design concentrates the interaction into one contrast. Adding levels spreads it across more df and dilutes the test unless the interaction is concentrated where the extra levels are.
Choose levels further apart. Larger differences between levels generally produce larger effects, within the range where the process behaves predictably.
Decide whether the interaction really matters. If one factor is a nuisance block and you're confident it doesn't interact with the treatment, an unreplicated block design is far cheaper.
Accept a larger detectable interaction. If 14 per cell isn't feasible, compute what f you can detect with the replication you can afford, and say explicitly that smaller interactions may go undetected.
Checking after the fact
Once the experiment is run, partial eta squared for each term gives a rough check on how the observed effects compare with what was planned for: f = √[η²_p / (1 − η²_p)]. A two-way ANOVA calculator that reports the sums of squares for each term gives you the inputs. If the interaction comes out non-significant but the design had low power for the interaction you cared about, report the result as inconclusive rather than as evidence of additivity.
Summary
Sample size for two-way ANOVA has to be planned for each of the three effects, and the interaction usually governs. Compute Cohen's f for each term from the cell means you need to detect and a realistic within-cell σ. For the same physical change, an interaction has half the f of a main effect and needs roughly four times the data. Two or three replicates per cell detect large main effects reliably and large interactions rarely; in the worked example, the main effect needed 3 per cell while the interaction needed 14.