CRO
Why Your CRO Program Isn't Working: Build a Research System, Not a Test Backlog
Most CRO teams ship A/B tests they invented in a meeting. The teams that actually compound revenue start somewhere else: a research system that surfaces the right experiments to run.
2026-02-18 · 10 min read · Rafat Afrad
Every CRO program we audit has the same problem: a backlog full of tests someone thought of in a brainstorm. Change the CTA colour. Try a sticky cart. Add urgency to the PDP. The tests run, most lose or go flat, the team gets demoralised, and CRO quietly gets defunded. The fix isn't better ideas, it's a research system that tells you what to test.
The four sources of test ideas
1. Quantitative analytics
Funnel data, segmented. Where do visitors drop, broken down by device, traffic source, returning vs new, and product category? Drop off concentration tells you where the friction lives. GA4 funnel exploration, Mixpanel, or a custom warehouse pull.
2. Session replay and heatmaps
Watch 30 sessions on your highest drop step. You'll see things data won't tell you: rage clicks on a non clickable element, scroll dead at 60%, focus on the wrong CTA. Hotjar, Microsoft Clarity (free) or FullStory.
3. Qualitative, surveys and interviews
On site exit intent polls: what stopped you from purchasing today? Post purchase surveys: what almost made you not buy? And ideally 5 to 10 actual customer interviews per quarter. Customers will tell you exactly what's broken, most teams don't ask.
4. Heuristic review
A senior CRO operator walks the site against frameworks (LIFT, MECLABS, Cialdini). Catches the things the team has gone blind to because they see them every day.
The synthesis: a friction map
All four sources feed into one document, the friction map. Each item is a hypothesised friction point with the evidence behind it. Items with multiple sources of evidence (quant + qual + replay all pointing at the same friction) jump the queue. Items from only one source go to the bottom.
From friction map to test backlog
Now you score the backlog. ICE works (Impact, Confidence, Ease) but the version we use adds R for Reach:
- Impact, estimated lift if it wins (1 to 10).
- Confidence, strength of evidence from the friction map (1 to 10).
- Ease, engineering effort to ship (1 to 10, higher = easier).
- Reach, % of sessions the test will affect.
How many tests should you actually run?
Fewer than you think. The math:
- Most ecommerce sites have enough traffic to run 2 to 4 well designed tests per month.
- Running more usually means under powered tests and false positives.
- Each test needs a pre declared sample size, decision rule and primary metric.
Statistical rigour
Use a frequentist or Bayesian approach consistently, don't mix. Pre declare sample size, never peek mid test and call it early on a fluke, and use revenue per session (not raw CVR) as your primary metric for ecommerce.
The win loss library
Every test, win, loss or flat, gets a one page write up: hypothesis, design, result, learning. After 12 months, you have a body of documented learning that survives team turnover and tells you which patterns work in your category. This is where CRO actually compounds.
What good looks like
A mature program ships 2 to 4 tests per month, wins 25 to 35% of them, and lifts revenue per session 2 to 8% per winner. Over a year that's 10 to 20% revenue lift from CRO alone, plus a library of learnings that informs every other channel. Without a research system, you'll never get there, you'll just churn through clever ideas that go nowhere.