Skip to main content
An A/B test splits your shoppers between your live checkout and one or more other designs, then shows you how each one performs — so you can make the winner live with confidence instead of guessing.

​
Before you start

  • You need at least two published checkout templates: the one that is live, plus another one to test against it. Drafts can’t join a test.
  • Tests run on the checkout page. The thank-you page can’t be tested yet.
  • One test runs at a time.

​
Starting a test

  1. Open the checkout designer and click A/B test, next to the template picker.
  2. On the Setup tab, give the test a name. Live now shows the template shoppers see today.
  3. Under Also show shoppers, tick up to three more published templates. Use each slider to set the share of shoppers who see that template; your live template keeps whatever is left.
  4. Each template starts as All shoppers. Change that to Mobile shoppers, Desktop shoppers, or a country to show the template to that audience only.
  5. Choose a Goal — the number the test is judged on: Conversion rate, Revenue per shopper (the default, since it combines conversion and order size), or Average order.
  6. Choose Run for: Until I stop it, a set number of days, or until each template has seen a set number of shoppers.
  7. Choose When it ends: Tell me, or Make the winner live.
  8. Click Start test.

​
While a test runs

  • The A/B test button shows Test running and how many days it has been going. The template picker shows each template’s share of traffic.
  • Each shopper sees one template for their whole visit, and again if they come back, so nobody gets switched mid-checkout.
  • Your own previews are never counted.
  • Set live is unavailable on the toolbar while a test runs. To change the live checkout, end the test from A/B test.
  • You can publish changes to a template during a test; its row in the results is marked changed so you know the numbers span two versions. You can’t change the shares or the templates in a running test — stop it and start a new one.

​
Reading the results

Open A/B test and switch to the Results tab. The strip at the top shows how far the test has run, total shoppers, the goal, and — when the winner is set to go live automatically — the date that happens. The table shows, for every template in the test:
  • Shoppers, with the number who arrived from Meta, TikTok, or Google ads in brackets.
  • Conv. rate — the share of shoppers who completed a purchase.
  • Avg order — the average order value.
  • Rev / shopper — revenue per shopper, with each template’s difference from your live checkout shown beside it.
  • Status — Ahead when a template is clearly beating your live checkout on the goal with enough shoppers to trust, Too early until then.
Numbers refresh about once an hour, so a new test shows zero for its first hour.

​
Ending a test

  • Click Make live on any template to make it your live checkout and end the test. Shoppers already mid-checkout finish on the page they started with.
  • Click Stop test to end it and keep your current live checkout.
  • If you chose Make the winner live, the winner goes live at the end of the run only when it is clearly ahead. If no template is clearly ahead, your live checkout stays and you get an email so you can decide.
Ended tests stay listed under Past tests. Open one to see its final numbers and winner; you can still make any of its templates live from there.

​
Good to know

  • Refunds don’t change a test’s results.
  • The ad counts in brackets cover shoppers who clicked a Meta, TikTok, or Google ad on the same device they bought on. Everyone else is still counted in the total.
  • Starting, stopping, and making a winner live need the same permission as publishing checkout designs.