Ok I admit it, I think baseline tests are dull!
For any of you who are unsure of what a Baseline test is, let me quickly explain. A baseline test is simply a tracking project, so no design changes are made to your website for customers to see which means you only have a control group. As a process it is often referred to as baseline analysis.
Baselines typically cover elements of tracking that you don’t find in a web analytics solution. However, you still need to plan and have a goal of what you want to achieve from building a Baseline. For example, why are people not proceeding to the next page? Is it due to specific form fields? Content not being viewed as expected by users?
Or it could be that over time a page has become full with additional content and banners and now you want to streamline the content, but you’re not sure what content is helping users reach your KPI. A baseline will help you to understand what content can be removed, if it’s generated no clicks or interest; tracking what users click on a specific page and what quirks your users are finding that you’re not aware of, the list goes on…
So, here’s why I think baseline tests are dull:
- They are uninteresting for the Optimisation Consultant – there’s no design transformations, they simply do not put all of their optimisation skills to work!
- They are unexciting for the client – there is no transformation that has optimised their website, therefore there’s no immediate gain. Deep down, they might feel time has been wasted instead of pushing an update they can see.
- They are monotonous for the developer – creating just conversion points simply feels like half project done! The exciting part of building projects is to be challenged to create transformations. Too boring!
HOWEVER, I also admit that baseline tests are the base pillar for many, many thoughtful and ‘bullseye’ test ideas!
The benefits of running a baseline
Once upon a time, on a poor performing page, a test idea was formulated based on my hunch. Honestly, it was a good hunch, I was convinced that I nailed the issue of the poor performance on the page. The button background to take the user to the next step of the funnel was a grey colour. Why grey I thought, it should stand out, right? So, a brighter, focus-capturing colour such as green would obviously have an awesome uplift – It’ll be a winner, no doubt!
And guess what?
The experiment did not perform as I expected! Sure, it did work slightly better than the grey button, but not the massive difference I was expecting.
Luckily, it was a fairly easy test that took not too much time to plan, build and QA, but imagine if I had used more resources for this test such as extra developers, additional Optimisation Consultant and Creatives team meetings… heavy QA… That would have been an unnecessary use of resources based on my ‘hunch’. The grey button was not the key issue and a baseline test would have told me that.
Another example
I remember another time when a test idea was put forward by a client. I wasn’t so sure. The idea was testing a key point page, with pretty big changes. It felt like a fairly elaborate test but I didn’t feel there was the data to back it up.
It would need a lot of careful planning to make sure we knew all possible test scenarios and eligibility rules to ensure that the test was bullet proof. We would be messing with API calls and would need to involve other third-party organisations.
Having learned my lesson previously by diving in feet first, I explained that it was a bit of a risk to test something so large when none of us were too sure of how many users would be impacted by the transformation. Without the data we just couldn’t be sure it was worth the time, effort and risk.
Conclusion, we decided to run a Baseline test!
What the Baseline showed us
During the Baseline test plan, we took the opportunity to track all elements, with the intention of capturing all possible behavioural scenarios. We also extended the Baseline test plan to capture custom data, to become more useful and far more informative. The findings were very interesting!
Even though the client did pinpoint the central element in their original idea, building the elaborate test wasn’t in fact necessary. The test would simply validate that yes, an internal fix would be needed to partially resolve the issue they were experiencing on that page. Nothing more.
When analysing the baseline results, we discovered other elements that allowed users to distract from the goal of that page. And these were elements the client had not thought of as being that harmful – but were especially important when we then analysed the projections of possible losses per year.
But the best part of the baseline results was the amount of incredible test ideas it generated. Good, solid, straight forward and data-driven test ideas which I can’t wait to get stuck into!
Conclusion
Baseline tests are as important as any AB test. We should always make sure that test ideas aren’t solely based on hunches.
Despite the wealth of analytics, recording and insights tools out there, there is often data you don’t have because you never knew you needed it. However, when running with a Baseline test, your tracking capabilities can be endless and precision perfect.
Ok I’ll admit it, I still find baseline tests dull to set up, but the findings are AWESOMELY EXCITING which makes it all worthwhile.
To find out more, or to ask any questions please get in touch!
