A/B testing
Upcoming featureAI A/B testing: prove which fix wins
An upcoming feature for the paid plans. It opens to a few accounts first.
Minimum sample reached on day 12 of 14 · 4,812 visitors
Illustration on KYMA, a demo store. The feature is upcoming.
01 What it does
An A/B testing tool built on your CRO audit
01
From audit to test
02
An editor with AI
03
Your developer's variants too
04
A winner you can trust
02 Why it matters
An audit tells you what to change. A test proves it worked.
A CRO audit gives you five to seven changes, ordered by expected impact. Some are obvious and you just ship them. Others touch the price, the offer or the main button, and there you want proof before the change goes out to everyone.
A/B testing closes that loop inside the same tool: the suggestion becomes the variant, the page goal becomes the metric, and the report says plainly whether the change won, by how much, and how sure it is.
03 How it works
How to run an A/B test in three steps
- 01
Choose the change
Start from a suggestion in your audit, or describe the change you want to test in plain words. - 02
Shape the variant
See it on your real page, adjust the words and colours, and set the goal: a click, a form or an add to cart. - 03
Run and read
Real visitors see one version each. The report says which one wins, by how much and how sure it is.
04 An example
Example: testing the top fix from a CRO audit
Add to bag rate, cumulative
Illustration on KYMA, a demo store. The feature is upcoming.
From a product page test, 14 days
- 1The audit's top suggestion: make Add to bag the darkest thing on the page. Variant B does exactly that.
- 2Add to bag rate: 6.2% on the original, 7.9% on variant B.
- 3The first three days swung both ways, and the report kept them marked as too early to call.
- 4Result: variant B wins, with a 97% chance of beating the original once the minimum sample was reached on day 12.
05 Guide
What is A/B testing?
A/B testing, or split testing, shows two versions of a page to two groups of real visitors at the same time and measures which version reaches the goal more often: more add to carts, more forms sent, more sign-ups. Because both versions run side by side, a seasonal dip or a traffic spike affects them equally, and the difference you see comes from the change itself.
The hard parts are rarely the statistics. They are deciding what to test, building the variant and knowing when to stop. Read the A/B testing guide for the method in full.
06 Guide
From CRO audit to A/B test, without a blank page
Most A/B testing tools start empty: you bring the hypothesis, the design and the copy. EyeCaptain starts from your CRO audit. Each suggestion already names the element, the problem and the change, so it becomes a ready variant you can adjust with AI or by hand on your real page. Changes ranked highest by expected impact are the ones worth testing first.
For changes an editor cannot make, such as a new bundle or a different checkout step, your developer builds the variant and EyeCaptain splits the traffic, measures the goal and writes the result.
| Classic A/B testing tool | EyeCaptain A/B testing | |
|---|---|---|
| Where ideas come from | Your own backlog | Your CRO audit, ranked by impact |
| Building the variant | Visual editor or developer | Written for you, editable with AI, or by your developer |
| Calling a winner | You read the dashboard | Minimum sample and run time set in advance, result written in plain words |
| Tracking | A separate script | The same small script as Live heatmaps |
07 Guide
How long should an A/B test run?
Long enough to reach the sample size you set before it started, and at least one full week so every weekday is counted. Stopping as soon as one version pulls ahead is the most common way to call a winner that is not real: early results swing widely on small numbers. EyeCaptain fixes the minimum sample and run time when the test starts and marks the early days as too early to call. For the statistics behind it, see Bayesian vs frequentist A/B testing, and for how EyeCaptain compares with a dedicated testing suite, see EyeCaptain vs VWO.
08 Plans and cost
On the paid plans
Priced in credits per test. The cost is shown before a test starts, and it uses the same tracking script as Live heatmaps.
09 Questions
A/B testing questions
Start with a CRO audit today
A/B testing is an upcoming feature. Your audits run now, and they are what it will build on.
No card · Result in under 5 minutes · Desktop and mobile
