Testing price elasticity with a holdout group instead of guessing

Van Westendorp and conjoint surveys are useful for narrowing down a price range before launch, but they measure stated preference — what people say in a survey — not revealed preference, what people actually do when a real charge hits their card. A holdout price test closes that gap, and it's simpler to run than most teams assume.

The basic design: split a comparable audience into two or more groups, show each group a different price for the same product or plan, and hold everything else — messaging, design, timing — as close to identical as possible. Measure conversion rate and revenue per visitor (not just conversion rate alone, since a lower price will almost always convert better even when it produces less revenue) at each price point over a fixed test window.

Two design choices matter more than people expect. First, randomize at the visitor or account level, not the session level, so the same customer doesn't see two different prices across visits — a common and reputation-damaging mistake. Second, run the test long enough to cover a full purchase cycle for your product; testing a monthly subscription price for three days will catch impulse buyers and miss the more price-sensitive segment that takes two weeks to decide.

For products without enough traffic to reach statistical significance quickly, a sequential or Bayesian testing approach (rather than a fixed-sample classical A/B test) lets you monitor results as they accumulate and stop early if one price is clearly outperforming, without inflating your false-positive rate the way peeking at a classical test does. And treat the result as a point estimate for current conditions, not a permanent constant: elasticity shifts with the competitive set, the macro environment, and your own product's maturity, which is why the highest-performing pricing teams re-test major price points every few quarters rather than treating a single holdout result as settled forever.