Here’s a thought experiment for those of you intrigued by the world of data and business.
We travel, you and I, to an island where 100 people live. All I’ve told you in advance is that 10 of the 100 people on the island like eating apples—the other 90 hate them.
When we get to the island and get off our boat we run into an islander, apparently at random.
The first question in our thought experiment is this—what is your estimate of the probability that this islander likes apples?
I suspect your answer will be a straightforward one. Since 10 out of 100 islanders like apples, and we have nothing else to base our estimate on, it seems reasonable that the change of this islander being one of the apple-eaters is 10 out of 100, or 10 per cent.
If that is where you ended up, I agree with you that it seems the most reasonable conclusion.
Now two things happen. Firstly, I give you a new bit of info. It turns out that 15 of the 100 people on the island like eating oranges (the other 85 hate them) and all of the 10 apple lovers also love oranges—they just can’t get enough fruit. Secondly, the islander we’ve just met pulls an orange out from his bag and starts eating it.
Next question—has your estimate of the probability that this islander likes apples changed, and what is it now?
The critical thing turns out to be that this new observation about oranges has changed our perspective. In the first stage, we could only see our islander as just a random one of the 100 people on the island. Now we know better—we’ve narrowed down to the fact that he is one of the 15 people on the island who like oranges. And of those 15 people, there are 10 who like apples (because we said that all the apple-lovers also liked oranges).
So our probability estimate now? For our islander to be an apple-lover he’d have to be one of 10 out of the possible 15 people in the set we know he belongs to. So the probability has risen from our first estimate of 10 per cent to a new figure of 10/15, or 67 per cent.
Why all this fruit-based quizzing? Well, it turns out that the math that underpins this quiz, is hugely relevant to our business lives now.
Suppose I operate a Web site that you are visiting. I want to show you ads on my Web pages or promotions for my products. But there are lots of ads I could show you and lots of promotions I could run. Which ones should I put in front of you, specifically?
Let’s take the plausible example that one of my ads is for a new mobile phone.
Overall in the population only a relatively small number of people are in the market for a phone at any one time (for this example, let’s say one per cent). So how do I decide whether you are interested in a new phone? Given that I don’t know anything about you then I can’t really say that the chances of you being interested in a new phone are any higher or lower than the one per cent average across the population.
Except hold on a minute. I do know some things about you. I know what browser you are using to connect to my Web site. I know whether you are connecting from a mobile or a laptop.
And it is just possible that some of those things are somehow correlated with being interested in mobile phones. Imagine I’ve historically observed that one per cent of the people I show the phone ad to click on it (just like the overall population average) but that if I split my traffic down by browser type it turns out that five per cent of nerdy Chrome users click on the ad but only 0.1 per cent of old-fashioned Edge users do. That suggests that the population of people who are going to click on the ad is “biased” towards Chrome users.
In this scenario, where the two statistics (browser use and phone-ad clicking) are not independent but instead skewed together (just like orange- and apple-eating on our island) the browser I see you using is a clue. If you are using a Chrome browser, I can conclude you are more likely to react positively to my mobile phone ad.
The idea that you can refine your probability estimate of something (phone buying) based on other bits of data that skew towards or away from that core statistic is called Bayes Theorem, and Bayesian methods are used over and over again in a whole variety of data science analyses. Specifically, the example we’ve just worked through is a real one—many systems for serving ads in real time on Web sites are Bayesian in nature.
It turns out there really is profit to be made from data science! This is the last in this series of articles based on The Average is Always Wrong, but there is plenty more to explore for the business-minded and data-curious—happy hunting.
Ian Shepherd is a CEO and CMO who has held senior roles in a range of world-class consumer brands over the last 25 years including Sky, Vodafone, Game, and Odeon. Ian has launched loyalty programmes, built new digital revenue streams for traditional retailers, and turned declining market share into stellar growth—all based on a keen practical understanding of the consumer and of the power of data and customer insight.
Get your copy of Mr Shepherd’s book The Average is Always Wrong.