Edge Master Class 2015 with Philip Tetlock
— A Short Course in Superforecasting —

| Class 1 | Class 2 | Class 3 | Class 4 | Class 5 |

Philip Tetlock:  If you turn to session six, slide 117-118, you’re going to see a little piece on the seductive power of scenarios. Imagine you’ve got one of these between subjects designs in which half of the people read the top slide, half of the people read the bottom slide, then they make a judgment about the plausibility or probability of this outcome.

Edge Master Class 2015 with Philip Tetlock
— A Short Course in Superforecasting —

| Class 1 | Class 2 | Class 3 | Class 4 | Class 5 |

Philip Tetlock:    What I want to do today is three things: I want to clear up some confusion from last time about counterfactual reasoning and how it’s intertwined with superforecasting; I want to link up some of the things that are in the final sets of slides on sacred values and taboo cognition and superforecasting; Then, I want to give you some examples of superforecasting in action and talk about condensing it all into four big problems and one killer app solution.

Edge Master Class 2015 with Philip Tetlock
— A Short Course in Superforecasting —

| Class 1 | Class 2 | Class 3 | Class 4 | Class 5 |

Philip Tetlock:   I was thinking over lunch about how to summarize or capture the essence of what went on this morning. There were certain key ideas I wanted to get across and for the most part they’ve gotten across, but some may have been miscommunicated a bit based on some feedback I’ve gotten.

You can look at the glass as either one-third full or two-thirds empty. Forecasting tournaments are in their infancy as a scientific method and as a tool for improving policy debate. We’ve gone through the first generation of tournaments, and we’ve made tangible progress. We’ve learned how to keep score; we have shown that it’s possible to measure the accuracy of probabilistic judgments, of messy real world events; we have shown that it’s possible to improve accuracy through a combination of selecting the right people, training them, teaming them, and using the right types of aggregation algorithms. Those are all achievements, but we’re still far short of what I’m going to call the “Bob Axelrod ideal” of a mechanism that can guide policy with evidence based precision. I don’t know if that’s a fair characterization of where you would want us to be, but we’re at least two-thirds short of that. We have made some progress, and we’re on the right path.

Edge Master Class 2015 with Philip Tetlock
— A Short Course in Superforecasting —

| Class 1 | Class 2 | Class 3 | Class 4 | Class 5 |

Philip Tetlock:    I thought it might be a good idea before we go into the quality of questions that are input into tournaments to say a few more words about the nature of the Good Judgment Project and what it meant to win the forecasting tournament, how it won the forecasting tournament, and what inferences you might or might not want to draw from victory in forecasting tournaments in general.

If you were to turn to slide twenty-eight in the book, it raises the question, how much can the Good Judgment Project improve foresight?