At Webtrends optimize, we work closely with brands to test their online content, discover what performs best, and target those results for an ongoing increase of conversion and revenue. With over 10 years of experience in optimization, we understand that a good test plan requires a foundation for good hypotheses. When beginning to test, developing a hypothesis can seem like a tedious first step, but it is essential to developing tests that actually move the needle. By testing against what you think will happen, you will prove or disprove assumptions and uncover new truths.
A hypothesis:
- Helps you understand and clarify the goals of your web site.
- Focuses the test criteria to what is important.
- Allows for an iterative pool of test ideas.
In order to develop a hypothesis that will help you build meaningful tests that drive conversion lift and insightful learnings, we recommend walking through the following seven steps.
1. Set goals: How does the page drive business value?
Every page has a goal, but have you mapped what that is? How does each page build on the others to drive conversion? What business value are these pages trying to deliver? For the pages you want to test, start by identifying the conversion goal. Your testing will uncover whether you can meet this goal—or exceed it.
2. Observe: Is your page meeting the goal?
Testing begins by taking a baseline measurement of how well a page is meeting its goal. You can review visitor behaviors on that page through analytics. Look for which actions visitors are taking. Map where they are coming from to the actions they take. And then ask, are these actions meeting our conversion goals?
3. Investigate: Why is the page not meeting its goal, and what’s in the way?
This is a good time to reset the assumption meter. Investigate visitor profiles and needs, and which functions the site should perform to meet those needs. Is there any friction on the page? Is the page layout or language helping visitors understand how to convert to the next step, or are they making conversion difficult?
Use your resources to understand these barriers. Analytics, click stream, conversion data and heatmaps can all shed light on blocks to conversion. Experienced optimization experts can help shed light on these barriers, and provide guidance on how to overcome them.
4. Propose: Structure your hypothesis with an action/reaction statement.
Write this down: “If I (do this), then (this) will happen.” As an example, “if I change the offer copy to focus on product benefits, rather than features, then more visitors will sign up.” This seemingly simple but methodical framework will formally frame your ideas, and reveal what you know about your web site—and what you still need to know. Use the insights gained during your investigation process.
5. Design: Focus your test to support or reject your hypothesis – only.
The test part of the hypothesis is, ” If I (do this)” and the expected result part of the hypothesis is, “then (this) will happen.” Make sure that your test content and the conversion metrics you are tracking directly tie to your expected result. If they do not, they will only distract from your insights. Testing tip: Test simple, but test big.
6. Measure: Sit back and let the data roll in.
Be sure to run your test until the results are statistically significant. An experienced testing team and good testing tool will help you predict how long the test will take and identify when significance has been reached. Major content differences are more likely to elicit more dramatic visitor responses (and a greater divergence of conversion lift) than smaller differences. And large differences in visitor response rates require dramatically less time to reach statistical significance.
7. Analyze: Failure is not an option.
Remember, you cannot fail. Your test is based on an informed hypothesis, and will show you whether you should accept or reject that hypothesis. Valuable insights can be derived even when a negative conversion lift, or no lift, occurs—as this is the response to your hypothesis, and you just learned something about your customers and how your web site drives business. You can take these valuable new insights and apply them to the next test, and the process starts all over again. Build a new hypothesis and continue down the path of optimization. When you learn what works, you continue to increase conversion over time.
Case study: online or downloaded training courses?
A global software company wanted to track the engagement levels of their training courses. The current method for providing training was through downloadable Power Point (PPT) presentations, which kept the company from tracking training progress. They wondered if offering training content online would increase customers’ willingness to complete training programs, and hypothesized that it would.
The company investigated the page usage patterns of presenting online-based training vs. PPT-based training. Conducting an A/B test comparing the two course delivery methods, they proposed the hypothesis, “If training is presented online, then conversion and engagement rates increase.”
Did tests show an acceptance or rejection of the hypothesis? After just eight days, tests revealed that the control page (online content) had a conversion rate of 40.30%, compared to the challenger page (PPT content) of 36.20%, a difference of -10.17%. Engagement rates were also in favor of the control page: 23.50% vs. just 11.53%—a huge difference of 103.86% between online vs. PPT training.
While there was a more favourable outcome for online training methods, analysis provided the company with an additional discovery. Whether customers begin training online or after downloading courses, a majority of them are choosing not to move forward with training. And that presents another opportunity for testing and optimizing.
Testing is hypothesis driven – every time
Whether you want to know something simple—if a green call to action button gets better results than a red button—or something more complex—if online courses are more engaging than downloadable courses—the test always starts with a hypothesis: “If I (do this), then (this) will happen.” It’s both the beauty and challenge of optimizing your digital properties so they meet the goals that bring business value.
