The Lab · The math · updated 2026-10-22
The Brier score — how to grade a probability, not a pick
A pick is right or wrong. A probability is something subtler, and grading it takes a different tool. Here's the Brier score, the market's own score in four sports, and why it settles in hundreds of games what a win–loss record needs thousands for.
The Brier score grades a forecast by how far its probability landed from what happened, squared and averaged. Lower is better; always saying 50% scores 0.25. It uses every game's probability instead of just the pick, so it can tell two forecasts apart long before a win–loss record can.
Record figures marked as today's come live from the public record, refreshed as games are graded — as of Oct 30, 2026. Numbers we published on earlier dates stay as written, dated, so you can check them against today's.
A pick is right or wrong. A probability isn't either. If a forecast says 70% and the team loses, was it wrong? Not necessarily — 70% means losing three times in ten. The question is whether it says 70% about the right games, and whether those games come in about 70% of the time.
Grading that takes a different tool. This is the one the Lab uses for every test.
The score
For each game, take the forecast's probability, subtract what happened (1 if it happened, 0 if it didn't), and square it. Then average over every game.
| Forecast | Happened? | Score for that game |
|---|---|---|
| 70% | Yes | (0.70 − 1)² = 0.09 |
| 70% | No | (0.70 − 0)² = 0.49 |
| 50% | Either | (0.50 − 1)² or (0.50 − 0)² = 0.25 |
| 95% | No | (0.95 − 0)² = 0.9025 |
That's the Brier score. Lower is better. Zero would be perfect — certainty, always right. A forecaster who says 50% about everything scores 0.25 no matter what happens, so 0.25 is the bar for knowing anything at all.
The squaring does two jobs. It punishes confident misses hard (the 95% call that missed scores ten times worse than a 70% call that came in, and nearly twice as badly as a 70% call that missed), and it rewards honesty: on average, the best score comes from saying exactly the probability you believe. Shading a forecast up or down to look bolder or safer costs you.
The market's own score
Here's the betting market's score on every game of four seasons, 2021 through 2024–25 — the consensus price at least an hour before the start, fee removed:
| Sport | Games | Market's Brier | Always 50% | Knows only who's favored |
|---|---|---|---|---|
| Baseball | 9,270 | 0.2395 | 0.2500 | 0.2425 |
| Hockey | 5,126 | 0.2301 | 0.2500 | 0.2350 |
| Basketball | 4,887 | 0.2074 | 0.2500 | 0.2175 |
| Football | 1,077 | 0.2124 | 0.2500 | 0.2207 |
"Knows only who's favored" is a forecaster who picks the same favorite as the market and gives every one the sport's average win rate — 58.7% in baseball, say — instead of the market's game-by-game number. The market beats it everywhere. That gap is what the market knows beyond who's better: how much better, game by game.
Two things to notice. First, the scores sit close to 0.25. Sports are noisy, and the differences that matter live in the third decimal place. Second, the sports can't be compared with each other. Basketball's 0.2074 doesn't mean the basketball market is smarter than baseball's. Basketball games are more lopsided, so they're easier to call (how often favorites win, by sport). A Brier score only means something beside another forecast of the same games.
Our board publishes the market's own probability, fee removed, so its public score is the market's score on every game it has graded: 0.2348 across 1,595 games, five sports mixed together. It's the baseline any forecast claiming to beat the market has to beat, on those same games — not a number to set beside the table above.
Why not just count wins?
Because a win–loss record throws away almost everything. A pick at 52% and a pick at 75% count the same when they win. To see a one- or two-point edge in a win rate takes thousands of graded games (the arithmetic).
A Brier comparison keeps the probability, and it can be paired: score two forecasts on the same games and look at the difference, game by game. Most of each game's noise — the upset, the fluke — hits both forecasts almost equally and cancels. What's left is how they differ, and that settles far faster. Two examples from our own tests:
- Our NFL props model against FanDuel's price, on 592 props: 0.2653 against 0.2399. Paired, the model was worse by more than four standard errors. One look, and the question was closed. (The test.)
- A line-movement signal against the closing price, on 338 games: worse by 0.00136, with a standard error of 0.00068 — about two standard errors. A difference in the fourth decimal place, resolved in a few hundred games. (What it means.)
How to read a Brier difference
- Same games, always. Pair the forecasts; never compare scores from different samples.
- Look at the standard error, not the size. 0.001 sounds like nothing. With a standard error of 0.0003 it's a real difference; with 0.003 it's noise.
- Count your tries. Test twenty ideas and one will clear two standard errors by luck. A difference found by looking needs a fresh sample before anyone believes it — which is why our tests are written down before their games are played.
- Cluster what moves together. Props from one game rise and fall together, so they aren't independent tries. Our Frankenstein test clusters its standard errors by game for exactly this reason.
For the numbers people: what's inside the score
The Brier score splits into three parts (Murphy's decomposition):
Brier = reliability − resolution + uncertainty
- Uncertainty is how hard the events are: p̄(1 − p̄) for the base rate p̄. Nothing a forecaster does changes it. It's why basketball scores lower than baseball.
- Reliability is calibration error: within each band of forecasts, how far the outcome rate sits from the forecast, squared and weighted. A calibrated forecast scores near zero here — the market's bands land within a point or two of their prices in all four sports.
- Resolution is how much the forecasts separate games that happen from games that don't — how far, and how correctly, they dare to move away from the base rate.
A forecast that always says the base rate is perfectly calibrated and has zero resolution. The market's edge over "knows only who's favored" in the table above is almost all resolution.
For a paired comparison, the statistic is the mean of dᵢ = (pᵢ − yᵢ)² − (qᵢ − yᵢ)² over games i, with standard error sd(d)/√n — or, when several bets share a game, the standard error of the per-game sums. The Brier score is a proper scoring rule: its expected value is minimized by reporting your true belief, which is the formal version of "shading costs you."
What this doesn't tell you
Whether a bet makes money. Brier grades the probability, not the price. A forecast can beat the market's Brier and still lose money if the improvement is smaller than the fee — and every one of ours so far has scored worse than the market, not better.
What happens in any one game. A better Brier score over a thousand games says nothing about tonight. A 70% forecast loses three times in ten, however good it is.
Which forecast to trust on a different sample. Scores are about the games they were measured on. That's why we write the test down first and score it forward, on games nobody had seen.
Model output is informational and entertainment content, not betting or financial advice. If you bet, bet what you can afford to lose.
Log what you actually bet — units and dollars — and see it graded beside a model that publishes every call it makes, wins and losses alike. Free, no card.
Start a free tracker