Single-Variable Testing

Change one thing at a time in a test, so you know what caused the result.

Change one variable per arm so the winner is attributable. Otherwise it's a guess.

Set up a test you can learn from

Version B of a landing page changes five things at once and wins by 4%, and nobody can say why. Choose what B should change in the next test.

How to apply it

If you must test combinations, use a multivariate (MVT) design with proper sample size.

Single-Variable Testing in practice: a concrete example

The test that can't tell you what won

You change the button colour to green and rewrite its label in the same variant, and conversion rises 9%. Which change did it? You can't say, because you moved two things at once, so the “winner” teaches you nothing you can reuse. Change one variable per arm and the result is attributable: the lift belongs to the thing you changed. If you genuinely need to test combinations, that is a multivariate test with a proper design and sample size, not a two-change guess.

Common Single-Variable Testing mistakes

Bundling several changes into one variant, so a win or loss can't be traced to any single cause.
Calling a confounded test a success and rolling the whole bundle out, carrying along changes that may have hurt.
Reaching for multivariate testing without the traffic to power all the combinations, which just spreads the sample too thin.

When Single-Variable Testing doesn't apply

Strict one-variable isolation is for learning why something worked. When you simply need the best of a few fully-formed designs and don't need to attribute the cause, a straight A/B between complete variants is fine. And a coherent redesign is often best shipped and measured as a whole, since splitting it into isolated variables would test a product nobody would actually build.

Practise it on a real challenge

Hero Test — three things changed at once

A team ships 3 variants of a hero with different copy, color, and image.

Put Single-Variable Testing into practice

SaaS Product Design Tracker — From feature backlog to launch — prioritise, test, QA, ship.