Forecast scoring correction
correction ·
The earlier score used the odds saved with each result. It did not account for how long earlier odds had stood.
The score now follows changes in odds over time. Forecasts with no measurable open period use their original recorded odds. Both versions below use the same 9 forecasts and outcomes.
What changed
Each forecast’s squared probability error is averaged from first recording until its first recorded outcome or the end of its window, whichever came first. Each scored forecast then counts equally in the overall score. A later correction changes the outcome used, keeping the original period.
A forecast first recorded at or after its window end, or recorded and checked at the same instant, uses its original odds. This also changes the earlier method: Oracle’s backlog forecast uses 72% instead of 85%, and Amazon’s cloud-growth forecast uses 60% instead of 70%. These two changes raise this snapshot’s score; time averaging alone would lower it slightly.
Missing or inconsistent history makes a score unavailable. It is counted and excluded from the average. No statement, probability or outcome was changed by this scoring correction.
Preserved scores
| Brier score | Previous | Corrected |
|---|---|---|
| All scored forecasts | 0.075800 | 0.088579 |
| Renting an H100 gets cheaper than it was in March 2025. | 0.014400 | 0.014400 |
| A Chinese company builds more than 1,000 humanoid robots. | 0.014400 | 0.014400 |
| Oracle’s contracted backlog jumps by $20B or more in one quarter. | 0.022500 | 0.078400 |
| The US puts a tariff on imported chips. | 0.078400 | 0.078400 |
| Nvidia’s next rack posts verified public test results. | 0.090000 |