Most important take-away: don’t take ‘experts’ too seriously, especially not the famous ones. They are very prone to looking at the world through the lens of a so-called hedgehog: there is only one truth and reality has to adapt to it.
Tetlock has even demonstrated an inversed correlation between fame and accuracy. The more famous the forecaster, the worse his predictions. Sadly, the media have a preference for hedgehogs, as they have ‘one big idea’ and usually are great storytellers. And as every media trainer will teach you (as mine did as well): keep it simple. Combine this with firm statements and we have great television, don’t we?!
It’s this bad forecasting that causes trouble, as people use this ill advice as basis for their decisions: financial losses, missed opportunities, unnecessary sorrow and even war are the result. Which doesn’t mean that all forms of forecasting are useless, so argues Tetlock. There is a remedy: just add a little more doubt to the mix.
Another difficulty is judging predictions on their value – it’s harder than commonly accepted: for the simple reason – and yes this is quite a cliche – that the truth is elusive. Vague descriptions as ‘very likely’ and ‘most probable’ are, according to research, subject to different interpretations. Especially media are extremely skilled in the usage of vague wording. The solution is to be more precise, through the use of numbers. Although even then one cannot rule out misunderstandings. Classic example being the meteorologist forecasting a 70% chance of rain. Only to find out that millions of people actually feel deceived by him because they could have done without their umbrella all day.
Now, let’s turn to Tetlock’s solutions. The art of good forecasting can be learned and it all comes down to combining the different insights and knowing when you’re fooled. As is the case with heuristics: replacing a difficult question by a simpler one. Think WMD in Iraq: in judging the Bush’ administration, common sense tells us that their judgement was incorrect, whereas the question (if assessing the value of the judgement is your goal) should be if the judgement was reasonable. And at that time, with the information known at that precise moment, it was.
Tetlock has worked with hundreds of people which he calls ‘superforecasters’; ordinary men and women with above average intelligence (but not more), a love for numbers and quite some time on their hand. They are intelligent and knowledgeable, but that’s not their USP. It’s what they do with it. The result is astonishing: they constantly outperform professional experts on predicting the future.
The big study (the Good Judgment Project) they participated in, and the basis of this book, demonstrates that superforecasters always start with looking for the outside picture of any given question. They really zoom out: what do the general data say? What’s the basis assumption we can work with? Furthermore, they are very conscious of the risks of group think and go to great lengths to incorporate all the information they can find. And, superforecasters don’t try to make predictions on the huge problems; they take a huge problem and break it down into smaller, more manageable problems, making their forecasts on those smaller units. Finally, they adjust their answers accordingly, in an perpetual loop of forecasting – measuring- adjusting.
That’s the best recipe, as simple as that….
Tetlock’s ten commandments for clear thinking should be put up in every board room:
- Triage
- Break seemingly intractable problems into tractable sub-problems
- Strike the right balance between inside and outside views
- Strike the right answer between under- and overreacting to evidence
- Look for the clashing causal forces at work in each problem
- Strive to distinguish as many degrees of doubt as the problem permits but no more
- Strike the right balance between under- and overconfidence, between prudence and decisiveness
- Look for the errors behind your mistakes but beware of rear view mirror hindsight biases
- Bring out the best in others and let others bring out the best in you
- Master the error-balancing cycle
DANIEL PEPIJN VAN DIJK 