The Lab · The math · updated 2026-10-22
Reading a null result properly — the form signal, in full
A p-value of 0.373, an R-sq of 0.41%, and a negative adjusted R-sq. Here is the complete output on our strongest shadow signal, and why "not significant" is the least interesting thing in it.
A result that's "not significant" doesn't mean nothing is there — it can mean the test was too small to see anything. Here's a real test of our form signal, read the way an engineer reads a regression: the noise first, then the power, then the p-value. A sharper question later got an answer.
This article is for Jake.
Everything else in the Lab is written for someone who has never seen a regression table. This one isn't. If you have ever stared at a Minitab session window wondering whether to believe the bottom line, this is the same problem with baseball in it.
The question, stated so it can be tested
We publish a probability for every game. That probability is the de-vigged median of the sportsbooks we poll (and, since October, one betting exchange) — we have no private information and claim none. The open question is whether anything cheap and public is left over: does a team's recent form predict what the closing price already priced in?
If the market is efficient, form is already in the number. If it isn't, form should predict the part of the outcome the price failed to anticipate.
That leftover is the dependent variable, and choosing it correctly is most of the work:
Surprise = actual result − what the market's price implied
Positive means the home side beat expectations.
Scoring against outcomes would be the naive choice and it is enormously less efficient — you are trying to detect a two-point effect through the variance of who won a baseball game. Scoring against the price removes everything the market already knew and leaves only the residual, which is what a signal has to move to be worth anything.
The predictor is a five-game form differential, computed identically to the shadow signal that runs in production. The subgroup below is every graded game where form leaned toward the home side.
The output
Nothing here is illustrative. This is our real backtest, N = 194, from the
admin panel:
One-Sample T: Surprise (pts), form leans home
N Mean StDev SE Mean 95% CI
194 3.17 49.59 3.56 (-3.85, 10.19)
Null hypothesis H0: mu = 0
Alternative H1: mu != 0
T-Value P-Value
0.89 0.373
Analysis of Variance
Source DF Adj SS Adj MS F-Value P-Value
Regression 1 1,949 1,949 0.79 0.373
Error 193 474,525 2,459
Total 194 476,474
Model Summary
S R-sq R-sq(adj)
49.59 0.41% -0.11%
A one-sample t on a residual is algebraically a regression with an intercept
only, which is why F = T² = 0.79 and the two p-values agree exactly. That
identity is worth keeping in your head; it is the same test wearing different
clothes, and a lot of apparent disagreement between methods is really this.
Reading it in the right order
Do not start at the p-value. Start at S = 49.59.
That is the residual standard deviation: the typical game lands about fifty points away from what the price implied. Against that noise, we are hunting an effect somewhere in the low single digits. Everything else in the table follows from that ratio, and no amount of statistical technique rescues it.
R-sq = 0.41%. Form explains four tenths of one percent of the variation in
surprise. Ninety-nine and a half percent of what happens in a baseball game,
after the price has spoken, is something else.
R-sq(adj) = -0.11%. Negative, which surprises people the first time they
see it. Adjusted R-sq penalises each term for the degrees of freedom it costs,
and it goes negative when a predictor explains less than a random one would.
Minitab prints it rather than clamping at zero, which is the correct behaviour:
it is telling you the term is not paying rent.
P-Value = 0.373. We fail to reject the null. Note the phrasing — fail to
reject, not accept. The distinction is the entire subject of the next
section.
The confidence interval is the honest summary. (-3.85, 10.19). The data is
consistent with form being mildly harmful, with form being worth ten points, and
with form being worth exactly nothing. That interval, not the p-value, is what
we actually learned.
The number nobody reports
Here is the question almost no betting analysis asks: if the effect were real, would this test have found it?
| true effect | power at N = 194 | N for 80% power |
|---|---|---|
| 2 pts | 8% | 4,819 |
| 3 pts | 13% | 2,142 |
| 5 pts | 29% | 771 |
| 8 pts | 61% | 301 |
| 10 pts | 80% | 193 |
Read the top row again. If recent form is genuinely worth two points of
surprise, this test had an 8% chance of detecting it. Ninety-two times out of
a hundred we would have run this exact analysis, on a real effect, and reported
p = 0.373.
So p = 0.373 is not evidence that form does nothing. It is evidence that our
sample cannot tell. Those are wildly different claims and they get conflated
constantly — usually by people who wanted the null.
The design is powered to detect a ten-point effect. A ten-point effect would be enormous; nothing that large survives in a market this liquid. We built a test that could only find something that cannot exist.
Type I, Type II, and which one costs you
A Type I error is claiming a signal that isn't there. A Type II error is missing one that is.
Betting analysis is obsessed with Type I — it is why we run 2-standard-error gates before promoting anything, and it is the right instinct when a false positive means publishing a pick that loses money in public. But at 8% power the error we are overwhelmingly more likely to be making is Type II. We are far better protected against fooling ourselves than against missing something real.
That is a deliberate asymmetry and we would choose it again. It just should be stated rather than hidden behind a p-value that reads like a verdict.
What would settle it
To resolve the observed +3.17 at 80% power takes 1,918 games. We have 194 — about a tenth of the way. The signal runs on baseball only, so the rest is most of another season.
Which is why the signal is neither promoted nor killed. It sits in shadow, scored on every graded game, accumulating. The production model applies no adjustment from it, so nothing you see on the board has been touched by a result with a 0.373 attached to it.
What happened next
Update, September 5. We kept scoring, and asked sharper questions. Two answers came back, pointing different ways — the most instructive thing this signal has done.
- Form leads the closing line. From the moment it fires, the market moves its way by about 0.39 points of win probability by first pitch: z +3.13 across 341 games. Form knew something the morning price didn't.
- Form doesn't improve the closing probability. Added to our published number, it made the Brier score slightly worse: +0.00144 ± 0.00126, z +1.14 — the wrong direction, short of significant.
The first answer fits the interval above leaning positive. The second is why form will not be promoted: by first pitch the price has already absorbed what form knew. Signals that predict the line, not the winner works through how both can be true.
Notice what changed. Not the sample — 341 games against 194 — but the question. "Does form predict the surprise in the result?" needs about two thousand games to see a three-point effect, because results are noisy. "Does the price move the way form points?" got an answer past three standard errors in 341, because prices move far less than results do. A sharper question is worth more than a bigger sample.
Four things worth taking away
Report S before R-sq. The residual standard deviation tells you what
size of effect is even in play. R-sq without it is decoration.
A negative adjusted R-sq is information, not an error. It means the term costs more degrees of freedom than it earns back.
Never read a p-value without its power. "Not significant" from a well-powered test and "not significant" from an 8%-powered test are opposite findings that print identically.
Prefer the confidence interval. (-3.85, 10.19) says everything the p-value
says and adds the magnitude, which is the part you can actually act on.
The unglamorous truth is that most honest sports-betting analysis ends here: a
wide interval, a small R-sq, and a sample that will take years to say anything.
The alternative — treating p = 0.373 as proof of absence, or a lucky subgroup
as proof of presence — is how people end up confident and wrong in both
directions at once.
Model output is informational and entertainment content, not betting or financial advice. If you bet, bet what you can afford to lose.
Log what you actually bet — units and dollars — and see it graded beside a model that publishes every call it makes, wins and losses alike. Free, no card.
Start a free tracker