Superforecasting

Philip E. Tetlock and Dan Gardner draw on forecasting research and the Good Judgment Project to discuss probabilistic predictions, updating, and keeping score. The publisher's book page describes that research setting. The forecast below is an original practice example, not a result from the project.

Turn “soon” into a resolvable question

“The library will reopen soon” is hard to score. Try: “Will the library's public website announce that the main reading room is open by 6 pm on 30 November?” Specify the year, time zone, source, and what counts as open before recording a probability. Partial access to a different room should not quietly become a successful prediction afterward.

Begin with relevant comparisons if you have them: how often have similar repairs finished within the announced window? Then note this project's circumstances. A completed safety inspection and an outstanding furniture delivery may point in different directions. Without reliable comparison data, say that your starting estimate is a judgment, not a measured frequency.

Keep an update trail

Suppose your first estimate is 60%. A dated announcement confirms the inspection has passed, so you move to 75%. Record both numbers and the reason for the change. Do not erase the first entry. A later delay might justify moving back down; updating is useful only when it responds to information rather than discomfort with uncertainty.

When the deadline arrives, record the outcome using the agreed source. One 75% forecast that fails does not establish poor forecasting: failures are part of what 75% means. Over many comparable predictions, check whether events assigned roughly that probability occurred roughly that often. Keep the number of forecasts visible; a handful gives little basis for a firm calibration judgment.

Where this approach stops

A forecast estimates what will happen. It does not decide what should happen or how much a bad outcome matters. A high chance of a small benefit may be unacceptable if the alternative is a serious loss. Read Thinking in Bets alongside this book to separate forecasts, choices, and outcomes.