A/B testing

Upcoming feature

AI A/B testing: prove which fix wins

Pick a suggestion from a CRO audit, see it as a variant of your real page, adjust it with AI or by hand, and split test it on your real visitors with the same small script as Live heatmaps. EyeCaptain reports the winner only when the numbers hold. An upcoming feature.

An upcoming feature for the paid plans. It opens to a few accounts first.

A · Original
Add to bag rate6.2%
B · Ink buttonWINNER
Add to bag rate7.9%
Chance B beats A97%

Minimum sample reached on day 12 of 14 · 4,812 visitors

Illustration on KYMA, a demo store. The feature is upcoming.

01  What it does

An A/B testing tool built on your CRO audit

01

From audit to test

Each suggestion in a CRO audit can become a test, with the change already written for your page. No blank page, no brief to write.

02

An editor with AI

Click any part of the variant to rewrite it, recolour it or ask the AI for another version, and see it on your real page before anything goes live.

03

Your developer's variants too

For bigger changes, such as a bundle or a new step, your developer builds the variant and EyeCaptain splits and measures the traffic.

04

A winner you can trust

A minimum sample and run time are set before the test starts, so an early lucky streak is not called a win.

02  Why it matters

An audit tells you what to change. A test proves it worked.

A CRO audit gives you five to seven changes, ordered by expected impact. Some are obvious and you just ship them. Others touch the price, the offer or the main button, and there you want proof before the change goes out to everyone.

A/B testing closes that loop inside the same tool: the suggestion becomes the variant, the page goal becomes the metric, and the report says plainly whether the change won, by how much, and how sure it is.

03  How it works

How to run an A/B test in three steps

  1. 01

    Choose the change

    Start from a suggestion in your audit, or describe the change you want to test in plain words.
  2. 02

    Shape the variant

    See it on your real page, adjust the words and colours, and set the goal: a click, a form or an add to cart.
  3. 03

    Run and read

    Real visitors see one version each. The report says which one wins, by how much and how sure it is.

04  An example

Example: testing the top fix from a CRO audit

Add to bag rate, cumulative

AB
too early to callsample reachedday 1

Illustration on KYMA, a demo store. The feature is upcoming.

From a product page test, 14 days

  1. 1The audit's top suggestion: make Add to bag the darkest thing on the page. Variant B does exactly that.
  2. 2Add to bag rate: 6.2% on the original, 7.9% on variant B.
  3. 3The first three days swung both ways, and the report kept them marked as too early to call.
  4. 4Result: variant B wins, with a 97% chance of beating the original once the minimum sample was reached on day 12.

05  Guide

What is A/B testing?

A/B testing, or split testing, shows two versions of a page to two groups of real visitors at the same time and measures which version reaches the goal more often: more add to carts, more forms sent, more sign-ups. Because both versions run side by side, a seasonal dip or a traffic spike affects them equally, and the difference you see comes from the change itself.

The hard parts are rarely the statistics. They are deciding what to test, building the variant and knowing when to stop. Read the A/B testing guide for the method in full.

06  Guide

From CRO audit to A/B test, without a blank page

Most A/B testing tools start empty: you bring the hypothesis, the design and the copy. EyeCaptain starts from your CRO audit. Each suggestion already names the element, the problem and the change, so it becomes a ready variant you can adjust with AI or by hand on your real page. Changes ranked highest by expected impact are the ones worth testing first.

For changes an editor cannot make, such as a new bundle or a different checkout step, your developer builds the variant and EyeCaptain splits the traffic, measures the goal and writes the result.

Classic A/B testing toolEyeCaptain A/B testing
Where ideas come fromYour own backlogYour CRO audit, ranked by impact
Building the variantVisual editor or developerWritten for you, editable with AI, or by your developer
Calling a winnerYou read the dashboardMinimum sample and run time set in advance, result written in plain words
TrackingA separate scriptThe same small script as Live heatmaps

07  Guide

How long should an A/B test run?

Long enough to reach the sample size you set before it started, and at least one full week so every weekday is counted. Stopping as soon as one version pulls ahead is the most common way to call a winner that is not real: early results swing widely on small numbers. EyeCaptain fixes the minimum sample and run time when the test starts and marks the early days as too early to call. For the statistics behind it, see Bayesian vs frequentist A/B testing, and for how EyeCaptain compares with a dedicated testing suite, see EyeCaptain vs VWO.

08  Plans and cost

On the paid plans

Priced in credits per test. The cost is shown before a test starts, and it uses the same tracking script as Live heatmaps.

09  Questions

A/B testing questions

Showing two versions of a page to two groups of real visitors at the same time and measuring which one reaches the goal more often, such as more add to carts or forms sent.

Start with a CRO audit today

A/B testing is an upcoming feature. Your audits run now, and they are what it will build on.

No card · Result in under 5 minutes · Desktop and mobile