Blog · November 16, 2026 · Updated November 21, 2026
How to tell if a change to your Shopify store worked
You rewrote a product description, added a shipping line, or turned on a "Buy it now" button. A week later, sales are up a little. Was it the change?
Maybe. Sales go up and down for many reasons: a good weekend, a post that got shared, a payday. If you don't compare carefully, you can keep a change that did nothing, or undo one that was working. Here's a simple way to find out, without any special tool and without a statistics course.
Why this matters more on a small store
A big store can run proper A/B tests: half the visitors see version A, half see version B, and with thousands of orders the winner is clear. Most small stores don't have that traffic. Shopify doesn't split traffic for you either, and testing apps need a lot of visits to give a reliable answer.
So small stores compare before and after. It's less precise than an A/B test, but done carefully, it tells you what's worth keeping. The five rules below are what make it careful.
1. Change one thing at a time
If you change the theme, the prices and the photos in the same week, you'll never know which one mattered. Worse, a good change and a bad change can cancel each other out, and you'll conclude that neither did anything.
Pick one change, write down the date, and leave everything else alone for a week. If you have several fixes to make, put them in order and do them one week at a time. Not sure where to start? Find the step where shoppers stop first.
2. Pick the number that change is meant to move
Not every change targets sales directly. A better photo can't fix a payment problem. Match the change to the step it's about:
| Change | Number to watch |
|---|---|
| Better product photos or description | Visits that add to cart |
| Shipping cost shown on the product page | Visits that add to cart, and carts that reach checkout |
| Sold-out sizes removed or marked | Visits that add to cart |
| Reviews added, empty stars hidden | Visits that add to cart |
| Express checkout button | Carts that reach checkout |
| Clearer refund policy, About and Contact pages | Checkouts that become orders |
Shopify shows these steps in Analytics → Reports → Conversion rate breakdown. Compare the share at that step, not just the count: if visits doubled, carts will go up even if the page got worse.
Here's an example. Before your change, 400 visits and 12 added to cart: 3%. After, 600 visits and 15 added to cart: 2.5%. The count went up, but the share went down. The extra carts came from extra visitors, not from your change.
3. Compare the same length of time
Compare the 7 days before the change with the 7 days after. Keep the same days of the week on both sides: a week that includes a weekend sale is not comparable to a quiet week. If your store is very small, use 14 days on each side instead of 7.
Write both numbers down the day you make the change, so you aren't tempted to pick the comparison that looks best later. A simple note is enough:
- Date of change and what you changed
- The step you're watching
- The count and share for the 7 days before
A week later, add the same numbers for the 7 days after.
4. Be honest about small numbers
On a small store, one order more or less moves the numbers a lot. If you went from 3 orders to 4, that's good news, but it isn't proof. Treat it as a first read:
- If the number moved the way you hoped, keep the change.
- If it moved the wrong way, ask whether something else happened that week (a holiday, an ad that stopped, a sold-out product) before undoing it.
- If it barely moved, keep the change if it makes the store clearer anyway, and look at the next step.
With more visits, the answer gets clearer. Keep comparing week after week, and the pattern shows. A change that helps tends to keep helping; a lucky week doesn't repeat.
5. Watch for what else changed
Before you credit your change, check:
- Did your traffic source change? An Instagram post that took off brings different visitors, often more curious and less ready to buy.
- Did a product go out of stock, or come back?
- Did you run a sale or an ad at the same time?
- Was there a holiday or a big shopping day in one of the two weeks?
If any of these happened, the comparison is mixed. Note it and look again the next week. This isn't failure: it's knowing how much to trust a number.
Mistakes that fool almost everyone
- Looking every day. Daily numbers on a small store jump around. Checking each morning pushes you to react to noise. Look once, after the week.
- Stopping as soon as it looks good. If you stop the moment the number goes up, you'll often catch a lucky day. Decide the length (7 or 14 days) in advance and stick to it.
- Comparing with last month's best week. Compare with the week just before, not the best one you remember.
- Counting sales only. A change to the product page shows up first in add-to-cart, sometimes before orders move. Watch the step, not only the total.
What to do with the result
Keep a short list of what you changed and what happened. After a few weeks, you'll have something more useful than any tip list: a record of what works for your store and your shoppers. You'll also see which kinds of changes are worth your time, so the next fix gets picked faster.
Common questions
How long should I wait before judging a change on my Shopify store?
At least 7 days, so every day of the week is counted once. On a store with few visits, 14 days gives a steadier answer. Decide the length before you start.
Can I run an A/B test on a small Shopify store?
You can, with an app, but with few visits it can take weeks or months to get a clear winner. A careful before-and-after comparison on one change at a time is often more practical.
My sales went up after a change. Does that prove it worked?
Not on its own. Check the step the change targets, compare shares rather than counts, and make sure nothing else changed that week, like a sale, an ad or a holiday.
What if the numbers barely moved?
Keep the change if it makes the store clearer for shoppers, and move to the next fix. Small stores often see small moves that add up over several weeks.
Doing it every week
This is exactly the loop RevealShop runs. Each weekly scan gives you up to 5 actions. When you make one, you mark it as done, and your next scan shows the week before and the week after, side by side, for the step that action targets. Small numbers are labelled as an early result, so you know how much weight to give them.
Want to see what your store could work on first? The free audit checks 7 things from your public pages, with no login.