This is some general advice on how to make a good prediction.
Gut instincts are incredibly valuable when it comes to making a prediction, the best predictors often heavily rely on their gut instincts, but remember that your gut can be flawed. Your instinct is exactly that, an instinct, so any cognitive or emotional biases you have could impede your predictive success.
The trick is to not rely 100% on what your gut instincts tell you but to always question them: subject them to critical appraisal, think about any biases that might be effecting your objectivity.
Its useful to be aware of some of the most common cognitive biases, thinking short cuts which can corrupt our metal calculations.
Of all these biases the most significant one you should be aware of is confirmation bias. Selectively using evidence to support your point of view.
Remember your emotions have a tendency to force you to believe much more in evidence that supports your point of view than evidence that does not support your point of view. The more emotionally involved you are in the outcome the less reliable your predictions are likely to be as a result of this.
To illustrate how significant this issue, is here are the results of a survey conducted in January 2016 where a group of US panellist were asked to predict who would win the US election. You can see that Democrats were pretty certain a democrat would win and Republications pretty certain a republications would win.
When making predictions try to really stand back from what you feel. But also remember you can also over compensate for this and be overtly negative about an outcome sometimes.
Start by having a good think about the problem you are trying to solve. Making a prediction is essential a problem that you are trying to solve. What do you need to know to solve the problem, where are you going to find that out, what issues do you need to consider, what could effect things? Build up a map in your mind.
For complex geopolitical predictions assembling as much evidence as you can is vital. Look out for conflicting points of view, understanding differences in opinions will help you pin down the level of certainty on any one topic.
We often over think about how unique a situation is and forget to look at evidence from the past that could help understand issues. Always think about whether there have been any similar situations in the past or from around the world that could help guide you.
Often with complex predictions you have to break it down in to lots of sub factors that are easier to make predictions about.
The more ways you can look at the problem the better. We have a tendancy to start looking at problems from the inside out. For example, elections we would consider reactions to the leadership and specific policies, an outside in approach would be to consider things like historical voting habits of a community and factors like the overall state of the economy. A good predictor will evaluate and weigh up things from as many points of view as they can.
This sounds a bit complicated but its not really. Its just the name given to the process of adding up the answers from different outcome scenarios. All you do is think about the topic from each specific point of view and predict the outcome based solely on that factor e.g. if the economy is doing well this tends to favour the incumbent political party so with that way of looking at things the incumbent part would win. You then look at the next point of view and the next and then add up how many point towards one choice or the other. If from 10 points of view choice A comes up 6 times and choice B 4 times, then this suggests there is a 60% chance of A being the outcome.
For example if you are left leaning you should read right leaning publications and sources of information to get a sense of the message and feelings from the other side of the fence and vica versa.
When making any forecasts it is very important once you have made up your mind not to give up thinking about it, but from that point in time to focus on trying to dissuade yourself. This will help you test out how robust your prediction is. If you can easily dissuade yourself or find your opinions being shaken it’s important to take notice of these feelings.
Dogmatism is the enemy of good prediction, never fix your prediction in stone, always be prepared to adjust your level of confidence in what the likely outcome will be and to be ready to completely change your mind.
You need to be in a ready state to jump ship at any time, and be constantly alert to over confidence. A good predictor, I am afraid to say, lives in a somewhat paranoid mindset.
What if something happens like an earthquake in the next 3 weeks? Unlikely, but a good predictor will go though and consider all the things that might happen that could influence the outcome of a predictions and take this into consideration.
Markets tend to over-react to news emotionally, and as a result you often see massive spikes in stock market prices when company results are announced or economic figures are released and then within a few days they settle back to where they were before.
It’s a natural process to overestimate the impact of any one piece of news, be considered when processing news information. My advice would be to sleep on it and think about how you will react to it the next day.
It helps if you are asked to predict how certain you are that something will happen to understand the basic odds of things happening. For example, if there are 2 people in a race and you are 50% certain one candidate will win, means you don’t know who will win. If there are 3 candidates in a race and you are 50% certain one particular candidate will win, this means you think this candidate is 50% more likely to win than the other 2 candidates, though you don’t know if they will win overall.
Having a good understanding of your certainly level can help ensure you make more considered predictions and also identify where there are holes in your knowledge.
Much of this advice is taken from these 2 book and if you don’t read both of these books you might find it tough to be a good forecaster.
An example: working out the result of the UK EU referendum
Imagine you were trying to predict which way the UK will vote in the upcoming EU referendum. Below are some examples of the range of things you could think about and research to help you make a better prediction.
Unpacking the problem:
1. What are the polls saying?
2. We cannot entirely rely on what the polls are saying can we, why?
3. Phone polls and telephone polls are telling a different story, what does that mean?
4. A lot of people are undecided which way they likely to vote?
5. Who are the people who cannot be reached by polls and what are they thinking?
6. Who will be most motivated to vote?
7. What impact could news events have as they unfold in the next few months?
8. What will be the impact of debate and discussions?
9. What impact will the personalities involved have on the debate?
10. What will be the impact of the press media and social media conversations have on the debate?
Gathering evidence:
1. What are all the polls saying right now? How much do they vary?
2. Why did the polls get it wrong last time? What can we learn from this?
3. How have the polls moved so far?
4. What evidence do we have about who the hard to poll groups are and what they are thinking?
5. What evidence do we have about voting intention amongst the undecided?
6. What was the latent sentiment of the UK public before the referendum process started?
7. What historical data is available from other referendums or independence votes?
Looking at it from different angles:
1. In other votes for independence how did public sentiment evolve in the run up to these votes?
2. Do British people feel they will be economically better off inside out outside the EU?
3. How important are immigration issues in the mind’s eye of votes compared to economic issues?
which issue will have the final say?
4. Who do the British voter trust more David Cameron or Boris Johnson?
5. What other big issues are driving people’s feelings on which way to vote?
6. With 3 major newspaper groups lobbying for an exit, how much influence do newspapers have?
7. How passionately do the different sides of the debate feel about the issue?
8. Will the levels of passion change as we get close to the vote?
9. How will the levels of passion change if the polls are leaning heavily one way or the other?
10. Loss aversion, will this impact on our decision?
Scenario planning:
1. What would happen to voter’s sentiment if a terrorist attack occurs the UK before the election?
2. What would happen if there was a collapse in the value of the pound before the election?
After you have read and thought about this the most important thing perhaps to do is think about what has not been considered in these list that might have an impact?
1. Have an intelligent conversation with your gut instinct!
Gut instincts are incredibly valuable when it comes to making a prediction, the best predictors often heavily rely on their gut instincts, but remember that your gut can be flawed. Your instinct is exactly that, an instinct, so any cognitive or emotional biases you have could impede your predictive success.
The trick is to not rely 100% on what your gut instincts tell you but to always question them: subject them to critical appraisal, think about any biases that might be effecting your objectivity.
Its useful to be aware of some of the most common cognitive biases, thinking short cuts which can corrupt our metal calculations.
2. Dis-aggregate what you want to happen with the process of predicting what will happen
Of all these biases the most significant one you should be aware of is confirmation bias. Selectively using evidence to support your point of view.
Remember your emotions have a tendency to force you to believe much more in evidence that supports your point of view than evidence that does not support your point of view. The more emotionally involved you are in the outcome the less reliable your predictions are likely to be as a result of this.
To illustrate how significant this issue, is here are the results of a survey conducted in January 2016 where a group of US panellist were asked to predict who would win the US election. You can see that Democrats were pretty certain a democrat would win and Republications pretty certain a republications would win.
When making predictions try to really stand back from what you feel. But also remember you can also over compensate for this and be overtly negative about an outcome sometimes.
3. Unpack the problem
Start by having a good think about the problem you are trying to solve. Making a prediction is essential a problem that you are trying to solve. What do you need to know to solve the problem, where are you going to find that out, what issues do you need to consider, what could effect things? Build up a map in your mind.
4. Gather as much evidence as you can from as diverse a sources as you can
For complex geopolitical predictions assembling as much evidence as you can is vital. Look out for conflicting points of view, understanding differences in opinions will help you pin down the level of certainty on any one topic.
5. A sage piece of advice “You don’t need to look into a crystal ball to see the future, just read the history books”
We often over think about how unique a situation is and forget to look at evidence from the past that could help understand issues. Always think about whether there have been any similar situations in the past or from around the world that could help guide you.
6. Break out the problem into sub-predictions
Often with complex predictions you have to break it down in to lots of sub factors that are easier to make predictions about.
7. Look at it from more than one angle – inside out and outside in, more helicopter viewpoints and in meta dimensions:
The more ways you can look at the problem the better. We have a tendancy to start looking at problems from the inside out. For example, elections we would consider reactions to the leadership and specific policies, an outside in approach would be to consider things like historical voting habits of a community and factors like the overall state of the economy. A good predictor will evaluate and weigh up things from as many points of view as they can.
8. Use a “Montecarlo” style approach to aggregate the answers from the different points of view.
This sounds a bit complicated but its not really. Its just the name given to the process of adding up the answers from different outcome scenarios. All you do is think about the topic from each specific point of view and predict the outcome based solely on that factor e.g. if the economy is doing well this tends to favour the incumbent political party so with that way of looking at things the incumbent part would win. You then look at the next point of view and the next and then add up how many point towards one choice or the other. If from 10 points of view choice A comes up 6 times and choice B 4 times, then this suggests there is a 60% chance of A being the outcome.
9. Enter the minds eye of the people with the opposing point of view and try to dissuade yourself!
For example if you are left leaning you should read right leaning publications and sources of information to get a sense of the message and feelings from the other side of the fence and vica versa.
When making any forecasts it is very important once you have made up your mind not to give up thinking about it, but from that point in time to focus on trying to dissuade yourself. This will help you test out how robust your prediction is. If you can easily dissuade yourself or find your opinions being shaken it’s important to take notice of these feelings.
10. Be prepared to change your mind over and over again
Dogmatism is the enemy of good prediction, never fix your prediction in stone, always be prepared to adjust your level of confidence in what the likely outcome will be and to be ready to completely change your mind.
You need to be in a ready state to jump ship at any time, and be constantly alert to over confidence. A good predictor, I am afraid to say, lives in a somewhat paranoid mindset.
11. Think though all what if scenarios
What if something happens like an earthquake in the next 3 weeks? Unlikely, but a good predictor will go though and consider all the things that might happen that could influence the outcome of a predictions and take this into consideration.
12. Understand market behaviour
Markets tend to over-react to news emotionally, and as a result you often see massive spikes in stock market prices when company results are announced or economic figures are released and then within a few days they settle back to where they were before.
It’s a natural process to overestimate the impact of any one piece of news, be considered when processing news information. My advice would be to sleep on it and think about how you will react to it the next day.
13. Understand certainty predictions
It helps if you are asked to predict how certain you are that something will happen to understand the basic odds of things happening. For example, if there are 2 people in a race and you are 50% certain one candidate will win, means you don’t know who will win. If there are 3 candidates in a race and you are 50% certain one particular candidate will win, this means you think this candidate is 50% more likely to win than the other 2 candidates, though you don’t know if they will win overall.
Having a good understanding of your certainly level can help ensure you make more considered predictions and also identify where there are holes in your knowledge.
14. Read Superforecasting by Philip Tetlock & Dan Gardner and The Signal & the Noise by Nate Silver:
Much of this advice is taken from these 2 book and if you don’t read both of these books you might find it tough to be a good forecaster.
Putting these ideas into practice...
Imagine you were trying to predict which way the UK will vote in the upcoming EU referendum. Below are some examples of the range of things you could think about and research to help you make a better prediction.
Unpacking the problem:
1. What are the polls saying?
2. We cannot entirely rely on what the polls are saying can we, why?
3. Phone polls and telephone polls are telling a different story, what does that mean?
4. A lot of people are undecided which way they likely to vote?
5. Who are the people who cannot be reached by polls and what are they thinking?
6. Who will be most motivated to vote?
7. What impact could news events have as they unfold in the next few months?
8. What will be the impact of debate and discussions?
9. What impact will the personalities involved have on the debate?
10. What will be the impact of the press media and social media conversations have on the debate?
Gathering evidence:
1. What are all the polls saying right now? How much do they vary?
2. Why did the polls get it wrong last time? What can we learn from this?
3. How have the polls moved so far?
4. What evidence do we have about who the hard to poll groups are and what they are thinking?
5. What evidence do we have about voting intention amongst the undecided?
6. What was the latent sentiment of the UK public before the referendum process started?
7. What historical data is available from other referendums or independence votes?
Looking at it from different angles:
1. In other votes for independence how did public sentiment evolve in the run up to these votes?
2. Do British people feel they will be economically better off inside out outside the EU?
3. How important are immigration issues in the mind’s eye of votes compared to economic issues?
which issue will have the final say?
4. Who do the British voter trust more David Cameron or Boris Johnson?
5. What other big issues are driving people’s feelings on which way to vote?
6. With 3 major newspaper groups lobbying for an exit, how much influence do newspapers have?
7. How passionately do the different sides of the debate feel about the issue?
8. Will the levels of passion change as we get close to the vote?
9. How will the levels of passion change if the polls are leaning heavily one way or the other?
10. Loss aversion, will this impact on our decision?
Scenario planning:
1. What would happen to voter’s sentiment if a terrorist attack occurs the UK before the election?
2. What would happen if there was a collapse in the value of the pound before the election?
After you have read and thought about this the most important thing perhaps to do is think about what has not been considered in these list that might have an impact?
EmoticonEmoticon