Use Baseline Analysis to capture data and track niche user behaviour before you start testing
A baseline analysis mimics analytics-style data capture for an account, as part of an experimentation product. It allows the user to capture various data, e.g. journeys, drop-offs, frustration points, interaction, etc., without having to run an Experiment, AA Test, or update their Tag Manager to facilitate that.
Data is useful wherever you manage to capture it. However finding the right people and time to facilitate Tag Manager updates is not always easy.
If you are familiar with your testing tool, and the data is already being captured there or would be useful to measure in the future, it becomes an efficiency to conduct the Baseline Analysis/research there too.
Before experimentation can occur, one often needs to know how users are performing on the website. Sometimes, this is general research to look at drop-off points in a journey, and sometimes it’s focused research on a single component e.g. Form.
Normally, high-level information is available from whichever Web Analytics tool you use. However, niche behaviour often isn’t, and either requires manual tagging work in a tag-manager (or similar), or an alternative mechanism.
One such mechanism is to run an AA test, whereby you are comparing two Control levels against each other. This is usually used to validate the statistics engine and initial setup when working with a new vendor.
However, if you have no need to compare a level against another (where data is divided across two levels), you can create a Baseline Analysis. This feature behaves as a tracking-only exercise and is reported on in a single-bucket with no variations.
Most Experimentation tools on the market don’t provide baseline analysis as an offering. Some have analytics/tag manager functionality built into their platforms, in which case you are still able to get to the data in a similar context.
Others, however, force you to use an AA-Test to collect this data. The difficulty with doing so is that your data is split into two groups, and you manually need to add these up. Alternatively, you could slant the traffic distribution between the two variations to 100%/0%, but overall the experience of doing so is sloppy – showing that the system is not designed for what you are looking for.
The first is a dedicated and purpose-built UI to facilitate baseline analysis. Doing so means the whole process is considerably more intentional, and so more painless.
The second is our ability to provide snippets to track niche behaviour. Nowadays, it’s clear that user behaviour is considerably more diverse than simply PPC vs. Organic traffic, or Desktop vs. Tablet vs. Mobile. The myriad of devices, costs, browser extensions, geography, etc. mean that there are numerous segments of people who often behave differently to each other. We therefore look into, and help our customers, to track niche behaviour such as:
These allow you to unlock segments that make sense in a modern context, and once finding them, personalise the experience for the user.