Webtrends Optimize believes that the recent release of The Forrester Wave™: Online Testing Platforms, Q3 2015 from Forrester Research makes a number of things clear including the fact that this is a mature market where all but one vendor is positioned as a Strong Performer or a Leader. For clients, this means you can feel comfortable evaluating vendors on their ability to meet your specific goals for your optimisation program. For vendors, however, this poses a special problem: How to differentiate within a relatively homogeneous solution set. Luckily, if you read between the lines, the Wave analysis makes Forrester’s view of the Optimisation market clear and goes on to suggest that automation will be the key to future differentiation.

Self-service versus managed service
There is a tension within the Wave analysis between solutions that focus on client-based testing versus vendor-assisted testing. The obvious evidence of this is the advancement of the so-called self-service solutions in the Wave diagram. All but one of the vendors in the Leaders band deliver self-service focused solutions while those in the Strong Performers band excel in vendor-assisted client success. The tension between these two models is underscored by these Forrester insights:

  • About the self-service solutions: “Many firms tell Forrester they prefer to manage their own online testing programs, but with very few exceptions, these firms depend extensively on the vendor’s services team for success. Online testing platform vendors must offer their customers a broad range of enterprise-class professional services —from implementation to managed (or strategic) services.”
  • About the managed service solutions: “Online testing platforms must make complex, powerful, self-service functionality safely available through well-designed user interfaces that users find intuitive and easy to use.”

With these statements, self and managed services simply become two-sides of the same coin. The tension, then, is really in the observation that solutions which once differentiated based on how they deliver value to the client are being advised to close the gaps with each other. This increases homogeneity and vendors lose some of their differentiation.

Continuous optimisation and the rise of the machines
Multiple times the phrase “continuous optimisation” appears within the Forrester Wave. The definition appears to be “applying analytics-driven approaches at every possible opportunity to evolve and leverage the understanding of customers as they interact across digital touchpoints.” It is within the following requirements for this strategy, as envisioned by Forrester, that the clues to future differentiation within optimisation lay:

  • Campaign management
  • Multichannel deployment
  • Self-service and usability
  • Enterprise services
  • Automation
  • Digital intelligence

The first two requirements are basics for any optimisation solution – whether tests are deployed by clients or vendors. As I demonstrated above, the self-service and enterprise services requirements that used to help differentiate vendors are now expected to co-exist. This leaves automation and digital intelligence as the battlegrounds for the future of optimisation.

Automation makes continuous optimisation possible
As optimisation programs scale to hundreds of test per month, the complexity in managing all the tests, targets, and assets makes it nearly impossible to run a successful program without strict guardrails to prevent overlaps and collisions. Sophisticated and strategy-driven enterprises manage these scale-related challenges with a rigorous program composed of expertly developed processes. At a lower level, it is even harder to perform the basics of test planning, segmentation, creating content variations, deploying the tests, evaluating results, and then acting on the insights. These tactical execution needs frequently result in a build or outsource discussion as enterprise optimisation programs look to build their team.

The goal of automation is not to remove these steps, but to use data mining and algorithms to “smarten” the optimisation system. Test plans can be suggested based on metadata from previous tests, segments can be automatically discovered and used to target content, content variations can be suggested by how previous content types have performed, testing types (A/B, MVT, or split) can be inferred through the number of factors and levels in play, and winners can be automatically selected and deployed.

Digital intelligence provides insights that drive decisions
More often than not, clients will opt for simpler tests with little or no segmentation and act based on lift alone. Simpler tests allow less experiences practitioners to maintain the pace needed for continuous optimisation. Lost in this analysis and report is all the valuable insight that comes from running more complex multivariate and funnel tests that allow you to determine what elements on a page contribute the most to a conversion or to keep users engaged with content.

One of the challenges for optimisation practitioners is the multiple data sources they must contend with. While optimisation solutions track visitors included in tests, they rely on analytics solutions to evaluate user behaviour throughout a site and across sessions. Segmentation poses another challenge, since sessionised data is often held in some kind of relational database outside the aggregate data-store. Where, then, is the source of record? And, how do you reconcile these multiple versions of truth?

Our vision for optimisation is intelligent automation
According to The Forrester Wave™: Online Testing Platforms, Q3 2015 from Forrester Research, “Webtrends is a Strong Performer because its strategy focuses on an automated optimisation vision to create relevant and engaging digital customer experiences.”

Our vision for enterprise optimisation harnesses the efficiencies and insights gained from a unified digital intelligence backbone and echoes many of the key directional elements identified by Forrester. The tension between managed and self-service delivery models and the imperative to meet in the middle is clear. Webtrends Optimize firmly believes that independent testing at scale within the organisation is the ultimate goal but the foundational benefits of an expertly managed service crafting a strategic, coordinated optimisation program is the key to driving long-term success. This belief is supported by the great successes our clients have realised, developing enterprise optimisation programs like Microsoft. As Forrester notes in the report: “Mastering the practice of online testing is not a one-off project for implementation and training.” Indeed, testing isn’t a one-time event, it is a program that needs to be methodically developed and in conjunction with the right technology that will support those processes.

In the coming months, you will hear more from Webtrends around our strides in common data collection and data storage across analytics, segmentation, and optimisation solutions and the insights this common view of users provides. As well, our vision for automation will manifest as we roll out new capabilities to our clients built on years of experience with testing and targeting across industries. The resulting automation capabilities, built upon a common view of users throughout their journeys as customers, will be unique within the industry and will provide Webtrends Optimize customers with a powerful platform for insight and action.