AB Testing

Make informed, data-driven decisions to get the most from your website by utilising AB testing software

What is AB Testing?

AB testing in digital marketing is a method for measuring the performance of two or more versions of a webpage or app. The results of AB testing are then analysed to establish which version works best for a defined conversion goal.

Webtrends Optimize uses market-leading technology to test any part of a website (such as landing pages or conversion funnels), and then measures user behaviour of a defined target audience. Using advanced statistical modelling, Webtrends Optimize can then accurately predict which test result will perform best.

Using AB testing to observe how an existing user experience performs against a new variation eliminates guesswork, helps you take more calculated risks and make informed decisions.

How does AB Testing work?

As a mini AB testing tutorial the following checklist is a good place to start:

  1. Start with your data and analyse/identify what area you would like to improve or change in your website
  2. Identify an element on the page you think will help achieve this and formulate a hypothesis to prove/disprove your change
  3. Create a variation based on this hypothesis
  4. Define your goals, making sure you are tracking the outcome of your test
  5. Decide who you want to see your AB test eg. all your visitors or just a specific subset (ie. those from a certain location, or using specific browser, or only return visitors etc.)
  6. Run your AB test with your new variation against the existing version (control)
Mock-up of a website with a red continue button to help show how you might implement A/B testing with the steps provided

To show how you might implement this, first take a look at this image

  1. You review your website analytics and see a large drop in traffic between this page and the next in your sales funnel (accessed by the red button)
  2. You decide the red button is the problem and hypothesise that changing the colour could make a difference
  3. You create an alternative version with a green button instead of red
  4. You implement tracking to make sure you can see customers getting through to the next step
  5. You decide this change is relevant to all your visitors
  6. You put your AB test live and monitor

Why do we do AB Testing?

The benefits of AB testing are huge and instilling a testing culture in your organisation can be a key foundation in driving business growth. AB testing tools can also help create a compelling business case for future development, investment and marketing campaigns. Below are a few other key benefits of AB testing:

Increase your ROI

This is one of the fundamental benefits for many when AB testing. You should see improved conversion rates, increased average order value (AOV) and generate higher online revenue.

Lower customer acquisition costs

Your existing marketing campaigns will start to convert better the more you optimise and refine your website with AB testing, particularly your landing pages.

Improve brand perception/loyalty

Good user experience (UX) builds trust with your customers. By AB testing different elements of your website you can refine and optimise that experience for your visitors.

Increase your customers lifetime value

Get the bigger picture of your customers' lifetime value by combining AB testing and personalisation techniques and tracking your customers interaction with your site.

Easy implementation/circumnavigate development pipeline

AB testing tools are simple to implement on your website which will then allow marketers (or anyone you authorise) to make changes quickly, without requiring a developers input.

What are the different types of AB tests?

The term AB testing is sometimes (incorrectly) used interchangeably with the term split testing. However, split testing is generally 2 distinct designs of an experience or page. Whereas in AB testing it is usually only one element of that page you are comparing with an alternative. Some common examples of areas of your website or app that you can look to AB Test are highlighted below.


Your headline is very important and can determine whether the visitor reads your content or bounces straight off the page/website altogether.

Call To Action (CTA)

The words/tone used, colours, positioning, size etc. of your CTA can make a huge difference to whether your visitor performs the desired engagement.

Images, Audio, and Video

Do different images/videos engage more than others? Does a video that plays automatically grab your visitors' attention and keep them on your page more than one where they have to interact with it first in order for it to play?


Your content might be excellent but if it's hard to digest your visitors may not stick around to read it. If you're selling something your visitor wants to purchase but it is unclear how they proceed to do so, you may lose them without making the sale.

Contact/Booking Forms

Is all the information you're asking for necessary? Is your form overly complicated? Is there a way to simplify or shorten the process such as AB testing facebook login or postcode lookup?

Content Length

Testing the length of your product descriptions, articles, blogs, FAQs etc. can all have a difference to obtaining the level of engagement you're trying to get from your visitors.

AB Testing statistics

Just as the AB testing story starts with the data, so it ends with the data too. The statistics/data/results you draw from your AB tests provide you with the confidence that the changes should go on to be permanently implemented on your core website.

Knowing that a change you have made has outperformed the existing experience and is statistically significant can be very useful when trying to champion your change request and get it prioritised by your development team. You don't have to theorise or speculate about the positive impact the change might have because you already know.

Get the most from your AB Testing

Know which content inspires action

Know which content inspires action

Even simple tests can provide useful insight and surprising results compared with your initial thoughts about what might work best. Changing the button colour or position of a call to action can have a significant impact on click-through rate and can increase conversion.

With AB testing, you'll get a statistically significant result for which button, message, design or any other element resonates best with your visitors, whatever the sample size.

Create more meaningful marketing

Create more meaningful marketing

Maximise your CRM and email marketing by breaking through the inbox noise. Webtrends Optimize allows you to test your emails to establish which design and messages encourage users to act.

Testing email creative can significantly increase engagement and click-through rate (CTR). Understanding user behaviour is important so our platform integrates seamlessly with your web analytics provider, including Google Analytics.

How else can you use AB Testing?

Your booking/sales funnel and landing pages are always a good place to use AB testing as these pages usually can have a very direct and immediate impact on your conversion and site goals, given their nature and objective.

AB testing your website personalisation features combines two very powerful conversion rate optimisation techniques and will help you identify the right approach specifically for your own visitors and potential customers.

Similarly, social proof and product recommendations engines can be very useful features, but AB testing the various approaches to this is important. What type of social proof works for your website/visitors? Indeed, does it work at all for you?

For product recommendations, how many products should you recommend? Where should you display these alternative items? What relationship should the additional product have to the one the user is currently exploring? All and more can, and should, reviewed using AB testing to ensure the experience you deliver is optimal.

Essentially AB testing is the glue that holds all these Conversion Rate Optimisation (CRO) techniques together to maximise their efficiency.

AB Testing vs Multivariate Testing

Understanding the different testing methods can be confusing, so what is AB testing as opposed to Multivariate testing (MVT)? Whereas AB testing is simply the choice between one version of a web page and a different version of a web page, with traffic split between the two, MVT is different because it allows multiple elements of a single webpage to be changed to establish which combination delivers the best result.

Both are very valuable features and which one to use depends on what your objectives are and what you are trying to understand in any given situation.

AB Testing with Webtrends Optimize

AB testing in Webtrends Optimize is incredibly easy. For those with coding skills our advanced editor is a very powerful tool.

However, our 'WYSIWYG' visual editor allows simple AB tests and complex variations to be quickly and easily created, even without any coding knowledge.

Check out this short video to see how quickly you can set up a simple but effective AB test.

What are the mistakes to avoid while AB Testing?

1. Invalid or incorrect hypothesis

When AB testing, you formulate your hypotheses before conducting a test. Every decision you make in your test depends on it, so getting it right is key! Understanding data before formulating hypotheses will allow you to build good tests and making sure your goals accurately represent success and failure will ensure you run a fair and valid test with meaningful results.

2. Not testing for the correct amount of time

4 conversions in a variation vs. 1 in the control is a 300% improvement, but meaningless at low volume. You can even reach statistical significance at very low volume if the observed difference in conversion is big enough (e.g. 20 vs 0), but experience tells us these are just indicative numbers.

Understanding sample size is key to running tests for long enough, but also not letting them run forever in hope of a result. Good planning around days also ensures you catch a variety of users who may behave differently – e.g., late night browsers vs. Sunday morning users.

3. Not re-running tests

Too often, marketers will only test once and believe the first result they receive. However, there is a possibility of a false positive for statistical significance – we’re all navigating probabilities. Therefore, it is important to retest at least once in a while so you can be confident that your result is correct.

4. Testing too many elements at once

If you are testing too many different elements at once, it is impossible to determine where the test’s success or failure comes from. You can always go back and test different elements later as a follow-up test or consider MVTs for more granular reporting to better determine high/low-performing components of your test.

Flexible support options

When using an AB testing tool you need to make a decision about which solution is right for your business. Do you take the technology/platform only or do you need some guidance and support? Our approach is flexible and bespoke. You might be considering conversion rate optimisation for the first time, maybe you have a dedicated internal CRO function, or perhaps you can deal with the AB testing basics but want help with something more complex.

Whatever your requirements, our comprehensive range of support options can be tailored to your specific needs. From a fully-managed service to a platform only SaaS model – and all points in between. Simply choose the level of support that works for you.

View Service Options

Get in touch

Want to know more about AB Testing, or how Webtrends Optimize can help you increase your conversion? Click below to send us a message or call us on 0333 444 5502.

Send message