Learn the Characteristics of Superforecasters for Highly Effective Thinking
I was intrigued when I heard of Superforecasters. These are a group of individuals in the Good Judgement Project who accurately make over 100 predictions about current affairs. From forecasting wars to predicting election winners, they have an uncanny ability to get the answers right.
The curious learner in me wanted to know more about these individuals. I wondered, “Do they have tips, tricks, and traits that I can model to help me learn more effectively and speed up my learning rate?”
A quick google search led me to the book, Superforecasting: The Art and Science of Prediction. In it, I learned some surprising characteristics about Superforecasters and how they go about making predictions. It turns out, you and I can apply the things they do in our own personal lives. But first, let’s start with their traits.
Characteristics of Forecasters
1. They have above average intelligence…
But they’re not geniuses. The Superforecasting authors say that if you can read the book with little to no effort, you have a shot at becoming a superforecaster.
2. They come from all walks of life
This was surprising to me because I thought they were the academic type. It turns out, they come from different backgrounds and different industries. They’re farmers, lawyers, movie producers, engineers, etc.
3. They don’t give up on hard problems
Growth mindset is a common trait among these individuals. The harder the problem, the grittier they get.
4. They’re humble, non-deterministic, and cautious
They know that there is no such thing as certainty. The world is filled with unknown variables which would affect their forecasts. They don’t let ego get in the way so they err on the side of caution. That said…
5. They update their beliefs based on new information
They care more about being right than believing they’re right. If new evidence comes out, they update their predictions for a more accurate forecast. My favorite quote in the book is, “Beliefs are hypotheses to be tested, not treasures to be guarded.”
6. They can suspend their biases to make an accurate prediction
They know that their perspective and opinions is just one angle of seeing a problem. So, they know that being emotionally attached to their beliefs isn’t an optimal way to make predictions.
7. They’re comfortable with numbers
Probability and statistics are their friend.
8. They work in teams
Teammates poke holes in their logic and reasoning so they can fortify them even more. They are also open to hearing different perspectives because it allows them to view a problem from multiple angles which leads them to accurate predictions.
So how do they make predictions?
First of all, They Keep Score Of Their Forecasts
Unlike pundits who use vague verbiage to make forecasts and interpret the outcome in their favor when an event happens, Superforecasters use a number called Brier Score. There’s no room for interpretation with a Brier Score. Either their forecast is correct or not.
The Brier Score is set up to penalize a forecaster if their prediction is wrong and reward them for getting it right. You can read more about it on the Good Judgment Open page.
They then Start With An Outside View
For example, let’s say the question asked in the tournament is this: “Will there be a robbery in LA in the next 30 days?”
You would be tempted to say something along the lines of, “Well, everyone’s leaving that place, businesses are boarded up, and people are getting desperate. So yeah, there will be a robbery in the next 30 days”
A Superforecaster would start differently. He’ll most likely say, “In the past 10 years, there was an average of 190 robberies per year. Divide 190 by 12, you’ll get 15.85 robberies per month. There are still 30 days left until the end of this forecast so 30 days/360 days x 15.85 robberies = 1.320. In other words, there’s a 100.32% chance of a robbery happening in the next 30 days (someone, correct my math if I got this wrong :).
They Then Proceed with an Inside View Analysis
Using an outside view analysis is just their starting point – their baseline if you will. Once they have that, they’ll proceed to do their research. They can be ultra news junkies because they know that having information from multiple perspectives can help them calibrate their forecasts. Depending on the info they get, they will nudge their prediction up or down.
They Ask Feedback from Their Teammates
Superforecasters are phenomenal by themselves, but their forecasts become more accurate when they team up. They practice constructive disagreement which helps them view their problems from different viewpoints.
They Update their Forecasts Based on New Information
The forecasting questions they get have an end date. Superforecasters are free to change their forecasts until that date. It’s actually crazy how much updating they do. One forecaster mentioned in the book updated his forecast about 50 times which made his prediction more calibrated.
Predicting the big stuff like wars and pandemic outbreaks is cool, but I find forecasting more helpful in everyday situations. Here are some examples:
- Based on what I know about my boss, what’s the probability that he will approve my idea?
- It’s Tuesday and it’s 7:30am. What’s the probability that there will be a crash on the highway as I drive to work?
- If I launch a course, what’s the probability that I’ll gain at least 50 students? How can I improve those odds?
In a way, this is an exercise that teaches hindsight in the moment kind of thinking. It helps you make optimal decisions and even helps in preparing for the obstacles that you might encounter.
It also allows you to wisely allocate time and resources on the levers with the highest probability of success. And, if a predicted outcome is not desirable, you could find variables that will nudge that probability up.
Although it takes a lot of brain power to think like a Superforecaster, the valuable lessons learned from doing so makes it worth it.