Shopify A/B Testing Best Practices: How to Test Your Store

A/B testing on a Shopify store means showing two versions of a page, a theme or an offer to different shoppers and keeping the one that sells more. The method is the same as on any other site. What changes on Shopify is where the test can run: inside Shopify itself, through a script in your theme, or through an app, and each of those reaches different parts of the store.

This guide covers the three ways to test on Shopify, which one fits which change, what each plan allows, how to run a test from start to finish, and what to test first. The general method, hypotheses and statistics included, is in our A/B testing guide. What to change on each page type is in our Shopify CRO guide.

Last updated October 12, 2026 by Devon Cox, Founder and President, ConversionTeam (16+ years in CRO). Shopify Help pages and app pricing were checked in October 2026.

TL;DR: A/B testing on Shopify in one minute

You A/B test a Shopify store with Rollouts Experiments on the Grow plan or higher, with a free tool such as Nantu through Google Tag Manager, or with a testing app from the Shopify App Store, and you judge the result on two full weeks of data.

• Rollouts Experiments are built into Shopify and test a theme change, a checkout and accounts configuration, catalog prices or discounts. They can’t change Liquid templates.
• A tool delivered through Google Tag Manager, such as Nantu, changes the homepage, collection pages, product pages and the cart in the shopper’s browser. It can’t change checkout on any plan.
• Set the sample size and the run length before launch, run whole weeks, and build the winner into the theme once the test has called it.

Bottom Line: pick the method by what you want to change and what your plan allows, then run the test long enough to trust it.

On this page:

• What is different about A/B testing on Shopify?
• What are the ways to A/B test a Shopify store?
• Which method should you use for which test?
• What can you A/B test on each Shopify plan?
• How do you run an A/B test on a Shopify store, step by step?
• What are the A/B testing best practices for Shopify?
• What should you A/B test first on a Shopify store?
• Can you A/B test Shopify checkout?
• Does A/B testing hurt Shopify SEO?
• FAQ

What is different about A/B testing on Shopify?

There are three ways to run a test on a Shopify store, and each one can change different parts of it.

An A/B test splits visitors between the current version of a page and a changed version, then compares a metric such as conversion rate between the two. The method, from writing the hypothesis to reading the statistics, is in our A/B testing guide. On Shopify, the question is where the test runs.

• Shopify’s own A/B testing is called Rollouts. You find it under Markets, then Rollouts, in your Shopify admin, and the A/B test type is an Experiment. Rollouts are available on the Basic plan or higher, and Experiments on the Grow plan or higher (Shopify Help). An Experiment can change your main theme, by editing it or replacing it, change your checkout and accounts configuration, set a catalog to active or draft, which changes the prices shoppers see, or activate and deactivate discounts (Shopify Help). It can’t change Liquid templates, doesn’t run on vintage themes, applies to the online store only, and can’t test a headless checkout.
• A tool delivered through Google Tag Manager, such as Nantu, changes the page in the shopper’s browser after it loads. On a Shopify store that means the homepage, collection pages, product pages and the cart page. It can’t change checkout on any plan: checkout.liquid is unsupported for the information, shipping and payment steps, so theme code doesn’t reach them (Shopify), and a container installed as a custom pixel runs in Shopify’s sandbox, which can’t display buttons, forms, banners or pop-ups (Shopify Help).
• A testing app from the Shopify App Store covers what its pricing page lists. Intelligems and Shoplift, the two apps in this guide, both list theme and template tests.

Rollouts is recent. Shopify opened it in early access with the Winter ‘26 Edition on December 10, 2025 (Shopify) and announced A/B testing for themes and checkout and customer account configurations on June 5, 2026 (Shopify changelog). The rest of this page is about choosing between the options and running the test well. For what to change on each page of the store, see the Shopify CRO guide.

What are the ways to A/B test a Shopify store?

Shopify’s built-in Rollouts Experiments, a tool delivered through Google Tag Manager such as Nantu, or a testing app from the Shopify App Store.

On a Shopify store, Nantu can test:

• the homepage
• collection pages
• product pages
• the cart page (layout and copy)

Nantu can’t test:

• the information, shipping and payment steps of checkout, on any plan
• the thank-you and order status pages

Rollouts Experiments. An Experiment compares a treatment against a control. By default, a new Experiment uses a 50% / 50% split and an end date 90 days after the start, and you can change both before you save it (Shopify Help). Shopify automatically creates a copy of your published theme or configuration for the rollout, so you can keep editing the live one independently of the experiment (Shopify changelog). A theme Experiment reports conversion rate, bounce rate, reached checkout rate and add to cart rate. A checkout Experiment reports checkout conversion rate. The metrics can’t be customized (Shopify Help). Several Experiments can run at once, each with its own dedicated share of traffic (Shopify Help). Shopify’s own A/B testing post points to Rollouts for theme tests.

Nantu. Nantu is ConversionTeam’s free, open-source A/B testing tool. It runs from a Google Tag Manager container installed once in your theme’s code. Each test is a tag in that container, so running a test needs no further theme edits, and the variation shows in the shopper’s browser without a theme change being published. Nantu records which variation each visitor saw as a GA4 user property, so results are read in the store’s own GA4 (Nantu Tracking). The install, and what it can and can’t reach on Shopify, is in Installing Nantu on a Shopify store.

Testing apps. The Shopify App Store has an A/B testing and experiments collection. Intelligems and Shoplift are the two apps this guide checked pricing for, as of October 2026. Every Intelligems plan starts with Smart Content, which covers content tests, theme and template tests and split URL tests; Build Your Own plans start at $69 a month at the lowest order volume tier, and price tests are a separate Smart Pricing module (Intelligems pricing). Shoplift from $99 a month on its Core plan, which covers template, theme and URL testing; price testing is on its Advanced plan, $399 a month (Shoplift on the App Store).

Which method should you use for which test?

Use Rollouts for a change you can make in the theme editor, a checkout configuration or a price; a Google Tag Manager tool for storefront copy, layout and element changes; and an app a store already pays for, for what its plan includes.

• Rollouts, on the Grow plan or higher, for a change you can build in the theme editor, a checkout and accounts configuration, a catalog price or a discount. It runs inside Shopify and reports its own metrics.
• A Google Tag Manager tool such as Nantu for storefront copy, layout and element changes on any plan, and for a change you want live without publishing a theme change. It changes the page in the browser, and Nantu costs nothing.
• An app the store already pays for, for what its plan includes, so the store doesn’t pay for or load two testing tools.

We’re testing-tool agnostic. We’ll use Nantu if you don’t have a testing tool, and we’re experienced with whatever testing tool you’re already using.

Whatever runs the test, the reading is the same: a sample size set before launch and whole weeks of data before you judge it. Those practices are below.

What can you A/B test on each Shopify plan?

Rollouts need the Basic plan or higher and Experiments need Grow or higher, changes to the checkout steps through apps need Shopify Plus, and a Google Tag Manager tool runs on the storefront from your theme’s code.

• Basic and higher: Rollouts, but not Experiments. A Launch publishes a change permanently, on a schedule or gradually, and an Event publishes it for a set period and rolls it back (Shopify Help). Standard checkout customizations, including the checkout and accounts editor and apps on the thank-you and order status pages, are available from the Basic plan up (Shopify Help). A Google Tag Manager tool such as Nantu runs on the storefront from the container in your theme’s code.
• Grow and higher: everything above, plus Experiments, which A/B test a theme edit or replacement, a checkout and accounts configuration, catalog prices and discounts (Shopify Help).
• Shopify Plus: everything above, plus apps built on checkout extensions that change the information, shipping and payment steps (Shopify Help).

The plan-by-plan table of what you can change in checkout, with the Shopify Help source for each row, is on our Shopify CRO agency page.

How do you run an A/B test on a Shopify store, step by step?

Pick the page from Shopify’s reports, write the hypothesis, choose the metric, set the sample size and run length, build and check the variation, launch, read the result once it has two full weeks of data, then build the winner into the theme.

Step 1: pick the page. Open Sessions by landing page and Conversion rate breakdown in Shopify’s analytics (Shopify Help). The first report shows which pages carry the traffic; the second shows where sessions drop out between adding to cart, reaching checkout and completing it. Test the page type with the most traffic and the biggest drop. How to rank what you find, and the research that tells you why shoppers leave, is in the Shopify CRO guide.

Step 2: write the hypothesis. One sentence: what you’re changing, what you expect it to do to which metric, and why. The format and the reasoning behind it are in our A/B testing guide.

Step 3: choose the metric. A Rollouts theme Experiment reports conversion rate, bounce rate, reached checkout rate and add to cart rate; a checkout Experiment reports checkout conversion rate; you can’t customize those metrics (Shopify Help). With Nantu, each visitor’s variation is recorded as a GA4 user property, so you read the result in GA4, split by that property (Nantu Tracking). Either way, decide the one metric that calls the test before it starts.

Step 4: set the sample size and the run length. Work out how many visitors each version needs to show the lift you’re after, and how many whole weeks that takes at the page’s traffic. The sample size math is in the A/B testing guide. Plan at least two full weeks, and run until the test reaches the sample size you set before launch.

Step 5: build the variation. For a Rollouts theme test, save the rollout, then edit your main theme in the theme editor from the rollout; the edits apply to that rollout only (Shopify Help), and Shopify’s copy of your published theme keeps the live theme editable independently of the experiment (Shopify changelog). For a Google Tag Manager test, the variation is a Nantu tag that changes the page in the browser: create a new test in the container, with the change as the variation’s code.

Step 6: check it before launch. We check every variation before launch, on a phone as well as a desktop, including adding a product to the cart. For a Nantu test, the QA tab lets you switch between the variations on the page before the test is live for shoppers. For a Rollouts change, review the edited theme in the theme editor before the Experiment starts.

Step 7: launch. A Rollouts Experiment starts with the traffic split you set, 50% / 50% by default. A Nantu test starts when you publish the container.

Step 8: read the result. Don’t judge the test until it has two full weeks of data and has reached the sample size you set. Read it where the tool reports it: a Rollouts Experiment in Shopify’s rollout analytics (Shopify Help), a Nantu test in GA4, split by the variation property.

Step 9: put the winner into the theme. For Rollouts, end the rollout and apply the changes permanently, or roll them back (Shopify Help). For a Nantu test, turn the winning variation on for all visitors until the change is coded into the theme, then remove the test. Then record the hypothesis, screenshots of both versions and the result, so the next test builds on this one instead of repeating it.

What are the A/B testing best practices for Shopify?

Test where the traffic is, change one thing or one deliberate bundle per test, run whole weeks, keep tests on the same page apart, check for script conflicts and flicker, and leave ecommerce tracking to Shopify’s Google integration.

The general method, from hypotheses to statistics, is in our A/B testing guide, and the reasons tests come back inconclusive are in why do my A/B tests always fail. These are the practices that apply on a Shopify store:

• Test where the traffic is. We prioritize pages with the most traffic and drop-off first.
• One change, or one deliberate bundle, per test. We set the sample size and the run length before launch and run tests in whole weeks, because a test called early is often a false winner. Test one change, or one deliberate bundle of changes, at a time.
• One test per page at a time, unless Rollouts keeps them apart. Several Experiments can run at once, each with its own dedicated traffic, so a visitor is in one of them at a time (Shopify Help). Two Google Tag Manager tests on the same page can’t be kept apart that way, so run them one after the other.
• Check the store’s apps and theme scripts before launch. A Google Tag Manager test adds a script to the page, so we keep test code small and use Nantu’s antiflicker so shoppers don’t see the original first. Open the page with the test running and watch for an app that redraws the section you changed.
• Leave ecommerce tracking to Shopify. We leave ecommerce tracking to Shopify’s Google integration and use Google Tag Manager for test code and Nantu’s test data only, because Shopify says adding Google Analytics through Google Tag Manager as well can duplicate tracking (Shopify Help).

What should you A/B test first on a Shopify store?

Test the page with the most traffic and the biggest drop-off first; the results below, grouped by page type, show what that has looked like on Shopify stores.

These are results from stores that were on Shopify when we tested them, as reported in each client’s progress report. Across all clients and platforms, 61.1% of our 2,288 audited tests won. The full list of what to change on each page, with more results, is in the Shopify CRO guide.

Homepage headline and hero copy. The first screen decides whether a new visitor keeps reading, and a headline is a quick build with any of the methods above. On a pet products store (2025), new homepage hero copy lifted conversion rate +72.4%. On a kids’ tech brand (2026), a homepage headline that says what the product does lifted conversion rate +8.2%.

Collection pages. Collection pages are where shoppers narrow the catalog, and the filters and the default sort are often left the way the theme set them. On a pet products store (2025), a redesigned best-sellers collection page lifted conversion rate +39.3%.

Cart. The cart is where shoppers check the total, and the copy around the cross-sell and the shipping line is cheap to change. On a pet products store (2025), rewriting the cart popup’s cross-sell headline lifted conversion rate +21.3%.

Sitewide fonts. Shopify themes usually set fonts in the theme settings, so a font test is quick to build and reaches every page. On a home goods store (2022), a new font pairing across the store lifted conversion rate +24.0%.

Price tests. A price test is a Rollouts catalog change on Grow or higher, or an app module. On a home improvement store (2018), a price test on a top product, where the lowest price won, lifted conversion rate +81.8%. A price test changes margin as well as conversion rate, so read revenue per visitor before you keep a lower price. An Experiment’s catalog change applies to the online store only, so other sales channels can show a different price while it runs (Shopify Help).

Can you A/B test Shopify checkout?

Yes, within limits: Rollouts Experiments compare two checkout and accounts configurations on the Grow plan or higher, apps built on checkout extensions change the information, shipping and payment steps on Shopify Plus only, and a tag manager can’t change checkout on any plan.

A checkout Experiment compares your current checkout and accounts configuration against a changed one and reports checkout conversion rate (Shopify Help). On Shopify Plus, the changed configuration can include apps that customize the information, shipping and payment pages, which Shopify allows on Plus only; on other plans, apps customize the thank-you and order status pages only (Shopify Help, Shopify Help).

A Google Tag Manager tool can’t reach any of that. checkout.liquid is unsupported for the information, shipping and payment steps (Shopify), so theme code doesn’t run there. Shopify’s Google Tag Manager help page points an upgraded checkout to a custom pixel (Shopify Help), and pixels run in a sandbox that can’t display buttons, forms, banners or pop-ups (Shopify Help). The thank-you and order status pages are the same: the additional-scripts box has been view-only since August 28, 2025, and script tags were retired on August 28, 2025 for Plus stores and August 26, 2026 for other stores (Shopify Help, Shopify).

So on Basic, Grow and Advanced, test the cart and the pages before it, and test checkout through a Rollouts Experiment on Grow or higher.

Does A/B testing hurt Shopify SEO?

No, as long as the test follows Google’s guidance: no cloaking, a canonical tag on any variation URL, temporary redirects, and the test ended once it has a result.

Google’s guidance on website testing is short. Show search engines the same content shoppers see. If a variation lives at its own URL, as in a split-URL test, point it at the original with a canonical tag and use a temporary redirect to send visitors there. End the test once it has a result, and keep the winner. A test delivered through Google Tag Manager changes the page in the browser and leaves the store’s URLs as they are, so there is no variation URL to manage.

FAQ

Does Shopify have built-in A/B testing?

Yes. Rollouts, under Markets in the Shopify admin, includes Experiments: A/B tests of a theme edit or replacement, a checkout and accounts configuration, catalog prices or discounts, on the Grow plan or higher. Rollouts can’t change Liquid templates, so layout and code changes on the storefront still run through Google Tag Manager or a testing app.

How much traffic do you need to A/B test a Shopify store?

As a rough minimum, about 5,000 users or 50 transactions a month on the page you want to test. Below that, focus first on higher-traffic pages, bigger changes, or research-led fixes that don’t need a test, so you aren’t waiting on an inconclusive result.

How long should a Shopify A/B test run?

At least two full weeks, and until it reaches the sample size you set before launch; a single test takes two to four weeks or more. Run whole weeks, because weekday and weekend shoppers behave differently, and don’t call a test early because it looks like it’s winning.

Is Nantu free on Shopify?

Yes. Nantu is ConversionTeam’s free, open-source A/B testing tool, and it runs on Shopify from a Google Tag Manager container installed once in your theme’s code. Install steps, what it can and can’t test on Shopify, flicker, tracking and QA are in Installing Nantu on a Shopify store.

Can you A/B test prices on Shopify?

Yes. On the Grow plan or higher, a Rollouts Experiment can set a catalog to active or draft, which changes the prices shoppers see. In the apps, price tests are a separate Smart Pricing module on Intelligems, and price testing is on Shoplift’s Advanced plan, $399 a month, as of October 2026. On a price test, we read revenue per visitor before keeping a lower price, because a lower price changes margin as well as conversion rate.

Does A/B testing slow down a Shopify store?

A Rollouts Experiment runs inside Shopify. A Google Tag Manager test adds a script to the page, so we keep test code small and use Nantu’s antiflicker so shoppers don’t see the original first. Check the page with the test running on a phone before launch, and remove tests once they’re done. For the rest of a store’s speed, see Shopify speed optimization.

Can you A/B test on Shopify Basic?

Yes, on the storefront. Rollouts are available on Basic, but Experiments need the Grow plan or higher, so on Basic a test runs through Google Tag Manager, with a tool such as Nantu, or through a testing app. Checkout can’t be tested on Basic: Experiments need Grow, and apps on the checkout steps need Shopify Plus.

What is the difference between Shopify Rollouts and an A/B testing app?

A Rollouts Experiment is built into Shopify, on the Grow plan or higher, and tests a theme edit or replacement, a checkout and accounts configuration, catalog prices or discounts, with fixed metrics. It can’t change Liquid templates. A testing app covers what its plans list: on Intelligems and Shoplift, content, theme and template tests on every plan, with price testing as a paid module or a higher plan.

Related reading

• Shopify CRO guide - what to change on each page of a Shopify store, with results from Shopify stores.
• Shopify CRO agency - how we run audits and A/B testing for Shopify stores, with the plan-by-plan checkout table.
• A/B testing guide - the method: hypotheses, sample size, statistics and scaling a program.
• Why do my A/B tests always fail? - the reasons tests come back inconclusive and what to do about it.
• Nantu, our free A/B testing tool - what it does and how to get it.
• Installing Nantu on a Shopify store - the install, what Nantu can test on Shopify, flicker, tracking and QA.
• A/B test win rate - how often tests win, across all our audited tests.

About the author

Devon Cox is the Founder and President of ConversionTeam, with 16+ years in conversion rate optimization. He holds a B.S. from Boston College and an MBA from the University of Colorado Boulder. More about Devon.