I am not a Trekkie… sporting a pair of pointed ears at Comic Con, never quite held it for me. But I do love the original 60’s series. That tension between William Shatner, as the emotional, instinctive and very human Kirk clashing with the resolute, Leonard Nimoy as the “Vulcan” Spock has been a surprisingly reliable reference, in understanding and managing diverse teams. Am I the only one who thinks “What would Kirk do?”. Go boldly where no man has gone before.
When it comes to my data, it’s more “What would Spock think?”. However, not being a Vulcan – I have dedicated a fair bit of time to understanding my decision bias and keeping them in check.
Wikipedia lists 141 distinct categories of human bias. Some, such as “Loss aversion” and “Confirmation Bias” you might be familiar with. Being in the CRO game, I’ve listed out a few of the more obscure ones, that resonate, particularly when it comes to AB testing and improving conversion rates.
(Setting phasers to stun).
The “Look Elsewhere Effect”
So first up is “Look elsewhere effect”, which is a form of confirmation bias, this is linked to a human predisposition to pay more attention to evidence that supports our hypothesis than evidence that that doesn’t. So where an expected result goes unchallenged. Unexpected results will often be scrutinised to an inch of their lives.
The temptation for the analysts is to go on a data treasure hunt. Cycling through all the segment permutations, crunching the data to an inch of its life – until “voila” captain, “Klingons from Alpha Centuri, do significantly convert more if presented with that stunning overlay banner.”
Being more like Spock – So this segment may be true, but by mining data to such a large extent introduces a decision bias. Effectively it’s playing 1000’s of rounds of Poker and then declaring that you can beat the house because of 1 winning hand. Exploring segments post-test has its place, however exercise caution. Challenge both expected and unexpected results and re-test if necessary to fully validate.
Rhyme-as-reason effect
Spock would never fall for this, but this is the tendency for statements that rhyme to be perceived as being more truthful. The often quoted example is the OJ Simpson trial where the defence presented the jury with the line, “If the glove doesn’t fit… You must acquit!”. (they did).
Being more like Spock
When copy testing and seeking to improve acceptance of your message, give consideration to the aesthetic as well as the meaning. “Clunk, Click with every trip” probably saved more lives than “Put your seatbelt on”.
Decoy Effect
So this is the decision bias that when presented with 2 choices, consumers will change their selection based on the introduction of a third choice where it is inferior in one attribute. This technique has been used to exhaustion in supermarkets and car washes with Bronze, Silver and Gold cycle options.
Being more like Spock
As a consumer – when presented with 3 choices, be aware of the presence of a potential decoy and its influence. It might be 50p more, but are you really going to eat all that popcorn?
As a marketer when presenting visitors with choices, always look to present value clearly and asymmetrically so that visitors can quickly arrive at a decision.
Dunning–Kruger effect
This is the decision bias of overestimating ones ability. Professor Dunning notes…
“the knowledge and intelligence required to be good at a task are often the same qualities needed to recognise that one is not good at that task. If one lacks such knowledge and intelligence, one remains ignorant that one is not good at that task.”
Perhaps best captured by the quote “The more I learn, the more I realise how much I don’t know.” – Albert Einstein.
Dunning and Kruger suggest that as experience with a subject increases, confidence typically declines to more realistic levels. As people learn more about the topic of interest, they begin to recognise their own lack of knowledge and ability. Then as people gain more information and actually become experts on a topic, their confidence levels begin to improve once again.
Being more like Spock
With AB testing, there is no hiding place. The data will judge.
My advice for new entrants into CRO is to…
- Keep learning and seeking
- Involve other people
- Dispassionately, keep questioning what you know
Courtesy & Acquiescence bias
Is a type of response bias that occurs when some individuals tend to not fully state their unhappiness with a service or product as an attempt to be polite or courteous, or opt for the more positive response. This can introduce a systematic error.
It is a common bias in qualitative research methodology and it can vary culturally. In the UK the classic example is in a restaurant with the “Is everything ok with your meal?” question.
Being more like Spock
Feedback forms captured anonymously with an open text field will provide greater insight than a fixed set of ratings. Not only will visitors tell you what you perhaps already know – they will alert you to issues that you might be unaware of and inspire initiatives. Even if the average flatters, trends and direction in sentiment should be reliable. Pay attention though to any campaigns that might affect the cultural balance.
Live long and prosper!
