CRO: Testing Atomic and Complex Changes

Conversion Rate Optimization (CRO) is a trendy, but complex subject. In order to increase your conversion rate, it is easy to start applying A/B testing tools and other web page testing techniques.

It’s easy to get lost in different Conversion Rate Optimization practices, raising a reasonable question: “In which cases should we test large changes, in which — atomic?”. “In which cases it is required to do a lot of testing, and in which — small changes and quick tests? What gives the best result in every particular case?”.

Let’s find out in our new article.

Photo by Daniele Levis Pelusi on Unsplash

Testing One Change in One Variation (Atomic changes)

Pluses:

  • By applying atomic changes you can easily track which change has led to a Conversion Rate (CR) increase or decrease, which serves as a basis on which you can develop the next test or a set of ideas of A/B split testing.
  • It is easier to develop quality ideas for the CRO test with atomic changes, as opposed to AB testing big changes that require high speed of generation. Applying atomic changes allows focussing on quality, not quantity.

Minuses:

Hence our InsightWhale recommendations — if you are applying atomic changes in one variation test only impactful hypothesis.

Testing Multiple Changes in One Variation (Complex changes)

Pluses:

  • You can get a large conversion rate increase when AB testing radical changes on the website, leading to a whole website redesign.

Minuses:

  • When it comes to web development, implementing 5 major changes on the website will take 5 times more time than it would take implementing a single atomic change, which means the test will be delayed. A radical redesign by AB testing big changes can take quite a long time.
  • The complex A/B test you’re conducting may show a negative result, but at the same time, one of the several CRO elements tested may actually have a positive effect on the Conversion Rate (CR), but it may simply be blocked by the negative effect of the remaining elements. As a result, an important element that does work will be lost along with elements that don’t if you do not test the elements participating in the test separately.
  • The complex CRO test can show a positive result, which will be implemented on the website, but in order to understand which change gave a positive conversion rate increase, and which actually decreased, you will need to conduct several more AB tests equal to the number of changes you’ve made. That means in the case of 5 new CRO elements being tested, a total of 6 new tests will be required — 5 for each CRO atomic change in particular and one complex tests that combined them all. In this scenario, you will spend time on one additional test, which is especially critical for websites where A/B split testing takes quite a while due to low traffic.
  • Due to testing several ideas in one variation simultaneously, it might become more and more difficult to develop new ideas. By choosing to apply complex CRO changes it becomes necessary to develop many more test ideas and provide a higher speed of generating ideas, due to which the quality of the proposed ideas and hypotheses may suffer.

Atomic vs Complex changes

Websites with low traffic should test 1 big impactful change or a few smaller changes. For websites with high traffic, we recommend testing 1 change at a time.

In order to get the fastest result, you can test several ideas in case you don’t yet have a single strong, significant hypothesis.

As a certified CRO agency, we do not recommend making any changes on the website before performing AB testing unless you are working on fixing a bug, repairing website errors, or implementing minor changes in design or content. The reason behind this suggestion is that you can’t be a hundred percent sure how the change will affect your website due to many factors.

Many factors play a role in increasing website conversion, especially the target audience, whose minds you can’t read, created UX/UI, and the quality of implementation made by developers. In this case, if something goes wrong the AB test will show it.

Another reason to avoid implementing website changes before A/B testing lays in the variability. If you make a change on the website and it ends up having a small positive or negative effect, it will be difficult to attribute correctly due to the variable nature of the data.

Conclusion

To avoid playing the dice with your company resources, test, test and test again. And call us while you’re at it 😉

Originally published at https://insightwhale.com on December 25, 2019.

Fully remote team of professionals providing Digital Analytics, Conversion Rate Optimization and Business Intelligence services for clients around the world.