The general election poll fiasco demonstrates that retailers should be wary of research that claims to forecast how consumers will behave.
Following last week’s humiliation of all pollsters and pundits it would be understandable, and indeed probably appropriate, if no piece of consumer research were ever again digested without a hypertension-inducing slug of salt.
But before we write off statistics and fall back to divination or chief executive opinion, we should make sure we draw the right lessons.
“The poll fiasco reminds us that people are unreliable at predicting what they are going to do”
Michael Jary, OC&C
Of course, any business decision that hinged on a swing of a few percent of the market would be best avoided. Company leaders have the advantage over politicians that they need not bet their careers on such slim margins for error.
The poll fiasco reminds us that people are unreliable at predicting what they are going to do, even a few days ahead. However as the broadcasters’ impressive exit poll showed, they can report what they have done.
We should be sceptical of research that claims to forecast how consumers will behave, and concentrate on understanding what they are doing now.
Accuracy of tracker surveys
This is becoming more possible because of what Dixons Carphone boss Sebastian James calls the “digital exhaust” of consumer data.
The exit poll was in effect a tracker survey: it sampled an identical set of constituencies in consecutive elections, and used movements to extrapolate changes in actual voting patterns. Trackers can be very accurate and give early warnings of changes in brand health or loyalty. Retailers tend not to use them enough.
One reason intention data is fickle is that people solidify their choices very late: whether it’s the 18% of voters who apparently made up their mind on election day, or the 28% of shoppers who make their final brand choice in-store.
And retailers, just like political parties, suffer from poor turnout: some 10% of firm shopping intentions are never realised.
One pollster, Survation, claims to have conducted an eve of polling survey that showed 37% support for the Conservatives; almost exactly the actual result. But no doubt to their eternal regret they chickened out of publishing.
Even statisticians suffer from confirmation bias, preferring the safety of the pack to their own analysis. We should avoid the temptation to explain away outliers and search for the insight they may contain.
Some sources did better. Bookies had the Conservatives as the largest party throughout. Prediction markets that aggregate the informed guesswork of the many got even closer, consistently calling a greater than 50% probability of Cameron being Prime Minister.
It seems that combining data with the crowd’s intuition and experience produces an improved result to data alone. Many businesses are beginning to use employee prediction markets to guide decisions, for example, on product launches or advertising campaigns.
Perhaps the ideas of a representative panel, stable segmentation and target profile may be dissolving as people become more individualistic and less brand loyal. The steadily declining shares of vote held by the two major parties mirror the difficulties of a Tesco or a Marks & Spencer in retaining a broad, middle-market position.
The increasing regionalisation of voting patterns echoes a fragmentation of consumer preference. Consumers no longer limit their switching to the big four grocers, or hunt back and forth between Next and M&S, any more than voting can be explained with a BBC swingometer.
Brand choice is more complicated and promiscuous. New entrants and disruptors challenge the incumbents. Traditional affinities are breaking down.
Real-time behavioural data needs to be collected, processed, contextualised and interpreted to understand this more complex world. “Opinion polls” are no longer going to cut it.
- Michael Jary is partner at OC&C Strategy Consultants