ABn Testing

What is ABn testing?

ABn testing is a type of testing where 2 or more variations of a certain element on a web page are compared with each other to understand which variation will deliver the goal of an experiment, or prove the hypothesis.

The ‘n’ in ABn testing refers to the multiple variations in the test. By contrast AB testing is a simpler test, pitching just one variation against another.

How ABn testing works

Traditionally, traffic is split in equal proportion between the experiments, for example if you have 4 experiments each experiment will have 25% of the traffic.

Traffic is also divided randomly amongst each variation, to make the sample distribution arbitrary.

What can you use ABn testing for?

Some examples and ideas of what you can use ABn testing for are shown below:

• Hero images and promo banners
• Copy changes
• CTA buttons
• Price messaging
• Icons, signposting
• Various exit intent messages
• Hiding elements on the page e.g. fields in the form that are not being used
• Navigation
• Imagery on the page
• Different CSS styles
• New sales funnels

Considerations when ABn testing

Below are some key considerations and things to look out for when using ABn testing:

• ABn testing is great to be able to test more than one variation of a page or element. You should make sure that the test page has enough traffic to divide between each of the variations and that the conversion point is also achievable for the sample size otherwise the test will have to run for a long period of time in order to reach a high probability and statistical significance
• Look out for a number of transformations that will be made as part of the ABn test, if it is a radical re-style of the page it may be more efficient and reliable to build and host the new pages on a separate URL and set up a split test
• Sometimes, the changes may be simple but the you may want to understand which specific changes exactly are driving the lift/drop in conversions, in this case multivariate testing (MVT) is likely to be more appropriate
• Look at the version of the test that did not perform well. Understanding what causes a particular experiment to under-perform can help to evaluate the hypothesis and also to come up with additional tests, not specifically generated from assumptions only, but also learnings from phase 1 of the results
• When multiple variants are tested in the first page, it may be worth it to rerun an MVT of the winning version to understand what particular elements on the test helped to drive the increase in performance

ABn testing with Webtrends Optimize

Webtrends Optimize can run all of the examples and ideas provided above when ABn testing. We can also provide additional support and guidance with the strategy and process of deciding what type of tests best support the hypotheses and outcome you are trying to achieve, for example AB, ABn, MVT, Split tests, Server-side testing etc. All are available in the Webtrends Optimize platform as standard.