The Is/Ought Problem
“If it’s simple, it’s always false. If it’s not, it’s unusable.” — Paul Valéry
Imagine you're a hominid going about your business on the plains of Africa a few thousand years ago. It's daylight, so you're out hunting and gathering food or wood for the fire—all in the hope of feeding yourself, keeping warm, and surviving long enough to reproduce and pass on your genes. The simple life.
Imagine you hear a noise in the long grass up ahead. A rustling sound. What's that, you wonder? Is it just the wind blowing through the grass? Or could it be a lion about to attack? You don't hang around to find out and run away.
If you'd assumed it was a lion and it turned out it was just the wind, that's a Type 1 error (a false positive). Essentially, you detected an effect that isn't present; you thought A was connected to B, but you were wrong. Nevertheless, you came to no harm.
On the other hand, if you'd assumed the rustle in the grass was just the wind and it was actually a lion, that's a Type 2 error (a false negative). You failed to detect an effect that is present, and, unfortunately, you're now the lion's dinner, and your genes won't be getting passed on any time soon.
Back in the day, it was probably best to just assume all rustles in the grass were predators, as there was no big cost to a Type 1 error, but there was a much higher cost for making a Type 2 error. Indeed, as humans, the basis of our survival over thousands of years has been our ability to see such patterns and causal events in our environment.
Unfortunately, modern life isn't so simple. In fact, it's infinitely more complex! So what does this mean for the way we learn now?
David Hume's Is/Ought problem describes how we often observe the way the world is and, from that, derive how things should be.
In recent years, FC Barcelona - with their admiral record of youth development and tiki-taka brand of football, and British Cycling - with their "marginal gains" fuelled hauls of Olympic medals have been held up as examples of what we "ought" to do in sport, as has the eponymous 10,000 "rule," thanks to some dubious journalistic interpretation of Anders Ericsson's research on expertise.
Similarly, the Finnish, Chinese, Korean, or wherever else's models have all been put forth as examples of how it "should" be done in education. These recommendations are inevitably influenced by a range of cognitive biases, including confirmation bias and hindsight bias (or the knew-it-all-along effect).
Consider another example: "talent" scouts in football. The media is full of emotive tales of scouts who "always believed X would make it" or "knew X had what it took from day one." Really? When first glimpsing a 6-year-old child on a muddy football pitch one rainy Sunday morning, did they know that child would go on to become a professional footballer some 12+ years later?
What you don't hear in those tales is how many times that scout has thought or said the same thing about numerous other young players who ultimately didn't go on to make it. Our memory is very selective, it seems, and, ultimately, it's only after the fact (with the benefit of hindsight) that we can draw such conclusions.
Most recently, it's been the turn of Leicester City FC, thanks to their magnificent (and odds-defying) Premier League title victory. The media and social networks have been awash with supposed insights into how and why they were able to upset the apple cart and do what they did.
For example, a state-of-the art technical scouting and performance analysis department has been cited. After all, they were able to spot and pluck eventual player of the year Riyad Mahrez from the French 2nd division for a reported £350k, peanuts in the modern game. However, those narratives tend to brush over the £9 million club record signing of striker Andrej Kramarić, signed by the same people and now out on loan in Germany after scoring just two goals.
Similarly, Ranieri's infamous "pizzas" have made for a nice narrative and have been heralded as a key moment in the season (he promised the squad he would take them out for pizzas if they kept a clean sheet, something they had struggled to do up until that point). Many have interpreted this as being a key way to encourage team spirit and bonding. What is often missed, however, is that upon arriving at the restaurant, the players were told they had to make their own pizzas.
The message being conveyed by Ranieri is that nothing comes easy, and we will have to work for everything.
While both of these examples have clear relevance and undeniably played a role in Leicester's success, the overall picture is far too complex to boil down to definitive black-and-white answers. Bonini's paradox explains how it is very difficult or impossible to understand some systems due to their complexity, but by trying to reduce this complexity to simplified models or rules, the complexity is lost, and the model is therefore useless in explaining things.
Points for reflection
While there are insights to be garnered from the Leicester story, what if random chance played the biggest role? The stars aligning in one crazy moment, the butterfly effect in all its glory—the complexity of which is just impossible to nail down to definitive reasons or a nice, neat model? cf. The fragility of cause and effect and The Texas Sharpshooter Fallacy
Is there anything you currently infer you ought to do, just because that's the way it is elsewhere?