[et_bloom_inline optin_id=optin_3]
We are covering how to use split testing with Andrew Brown co-founder of the Splitly software tool. This episode covers when to use split testing, which products benefit most from split testing and why split testing improves your sales.
What is split testing?
- Website traffic is split 50/50 with half seeing one version of your page and the other half seeing the another page
- You then analyse the data to see which page performed better
How to use split testing in the Amazon platform
- Amazon does split testing all the time – you see changes
- Splitly – changes the listing in Seller Central back and forth over a 24 hour period
Why Split Test?
- Most of us in FBA want the ‘best’ images and copy
- If you can reliable improve your conversion rate by a couple of percent it translates to sales
- Optimising your listing can be cheaper than spending on advertising to get sales
- Only through split-testing can you tell if your change has brought more sales
- You can test hypothesis to see if it improves sales
The 80/20 Rule
- You have title, main image and price – plus the 5th bullet
- How to use split testing for the biggest benefit?
- Price is a key influencer on sales – which price is more profitable?
- At my lower or higher price point am I more profitable?
- The A9 search algorithm is a bit different to Google’s
- Amazon ‘knows’ if you’re making money and it rewards you in the search results where as Google doesn’t notice
- For Amazon’s search algorithm it’s about sales velocity and conversion rate
- For Google’s search algorithm is about relevance and organising information
- On Amazon you could be more profitable with a higher price, you think you’re making more profit but you may be damaging your ranking – long-term losing money
- With a lower price you could make more money with higher ranking and volume
- Amazon uses a rolling average over 30 days – so your most recent days are the most effective
- If you change your price you won’t know the full effect of that change for 30 days for it to feed through Amazon’s algorithm
- If you’re doing an A/B price test then you need to be aware of this
- It’s a dampening effect
Statistical Significance
- A mathematical term for how confident we can be that the change we’ve made is having the effect
- We’ve made a change and there’s a number of data-points (number of days) that we can analyse to check if the change is having the effect we want
- There is this big a difference between the conversion rate – if it’s over 90% then that’s a pretty good level of confidence
When a split test goes bad
- If the effect is not what you want – then the statistical significance will give you your answer faster
- If there’s no real statistical significance – then stop the test and start a different change to test again
Do you split test the products doing 100 per day or 5 per day?
- People try to rescue struggling listings or to test new launched products
- Split testing is the icing on the cake
- The products that are established have more sales, traffic and hence data so you can get the answer to your tests faster
- Also, if you test on a product that’s doing well and you move the needle 5% then you’ve made a lot more money
- If you do the test on a poor performing product then you’ve spent the same time and effort but getting little ROI
How often should you split test?
- Focus on one change at a time
- Do an A/B test with only two versions
- And change ONLY one thing so you know what has caused the effect
- Don’t complicate it
- Advanced users can then look at Multi-variate testing which is more complex
Software Tools – Splitly
- If you’re doing regular split-testing on a 24hr basis – it can be awkward to time it
- You need to make the change at mid-night Amazon time for the 24hr data slot every day – the tool makes the change at the correct time for you
- Software will gather the data for you and calculate the statistical significance for you – no mistakes
- Pricing A/B Test – the last 30 days of sales Amazon uses a rolling average – profit-week tool tests different price points and re-calculates it each day using a linear regression to assess today’s sales and price-points and build a model to look at which price-to-profit relationship is best for your product
- All automated and straight-forward using a tool
- You can do this manually but it’s very time intensive
What improvement of profitability are users seeing?
- Splitly work with Jungle Scout – ran a test on the baby towel with a hood
- Tested it with a baby in the image and without
- The test proved that it doubled sales with the baby in the image
- Test because the data is real evidence that you can base your decisions on
Contact
- Go to Splitly.com and everything you need is there
