The Lab · The math · updated 2026-08-16

Reading a null result properly — the form signal, in full

A p-value of 0.373, an R-sq of 0.41%, and a negative adjusted R-sq. Here is the complete output on our strongest shadow signal, and why "not significant" is the least interesting thing in it.

This article is for Jake.

Everything else in the Lab is written for someone who has never seen a regression table. This one isn't. If you have ever stared at a Minitab session window wondering whether to believe the bottom line, this is the same problem with baseball in it.

The question, stated so it can be tested

We publish a probability for every game. That probability is the de-vigged median of ten sportsbooks — we have no private information and claim none. The open question is whether anything cheap and public is left over: does a team's recent form predict what the closing price already priced in?

If the market is efficient, form is already in the number. If it isn't, form should predict the part of the outcome the price failed to anticipate.

That leftover is the dependent variable, and choosing it correctly is most of the work:

Surprise = actual result − what the market's price implied

Positive means the home side beat expectations.

Scoring against outcomes would be the naive choice and it is enormously less efficient — you are trying to detect a two-point effect through the variance of who won a baseball game. Scoring against the price removes everything the market already knew and leaves only the residual, which is what a signal has to move to be worth anything.

The predictor is a five-game form differential, computed identically to the shadow signal that runs in production. The subgroup below is every graded game where form leaned toward the home side.

The output

Nothing here is illustrative. This is our real backtest, N = 194, from the admin panel:

One-Sample T: Surprise (pts), form leans home

  N     Mean   StDev   SE Mean        95% CI
194     3.17   49.59      3.56   (-3.85, 10.19)

  Null hypothesis        H0: mu = 0
  Alternative            H1: mu != 0

  T-Value    P-Value
     0.89      0.373

Analysis of Variance

Source        DF     Adj SS    Adj MS   F-Value   P-Value
Regression     1      1,949     1,949      0.79     0.373
Error        193    474,525     2,459
Total        194    476,474

Model Summary

     S    R-sq   R-sq(adj)
 49.59   0.41%      -0.11%

A one-sample t on a residual is algebraically a regression with an intercept only, which is why F = T² = 0.79 and the two p-values agree exactly. That identity is worth keeping in your head; it is the same test wearing different clothes, and a lot of apparent disagreement between methods is really this.

Reading it in the right order

Do not start at the p-value. Start at S = 49.59.

That is the residual standard deviation: the typical game lands about fifty points away from what the price implied. Against that noise, we are hunting an effect somewhere in the low single digits. Everything else in the table follows from that ratio, and no amount of statistical technique rescues it.

R-sq = 0.41%. Form explains four tenths of one percent of the variation in surprise. Ninety-nine and a half percent of what happens in a baseball game, after the price has spoken, is something else.

R-sq(adj) = -0.11%. Negative, which surprises people the first time they see it. Adjusted R-sq penalises each term for the degrees of freedom it costs, and it goes negative when a predictor explains less than a random one would. Minitab prints it rather than clamping at zero, which is the correct behaviour: it is telling you the term is not paying rent.

P-Value = 0.373. We fail to reject the null. Note the phrasing — fail to reject, not accept. The distinction is the entire subject of the next section.

The confidence interval is the honest summary. (-3.85, 10.19). The data is consistent with form being mildly harmful, with form being worth ten points, and with form being worth exactly nothing. That interval, not the p-value, is what we actually learned.

The number nobody reports

Here is the question almost no betting analysis asks: if the effect were real, would this test have found it?

Read the top row again. If recent form is genuinely worth two points of surprise, this test had an 8% chance of detecting it. Ninety-two times out of a hundred we would have run this exact analysis, on a real effect, and reported p = 0.373.

So p = 0.373 is not evidence that form does nothing. It is evidence that our sample cannot tell. Those are wildly different claims and they get conflated constantly — usually by people who wanted the null.

The design is powered to detect a ten-point effect. A ten-point effect would be enormous; nothing that large survives in a market this liquid. We built a test that could only find something that cannot exist.

Type I, Type II, and which one costs you

A Type I error is claiming a signal that isn't there. A Type II error is missing one that is.

Betting analysis is obsessed with Type I — it is why we run 2-standard-error gates before promoting anything, and it is the right instinct when a false positive means publishing a pick that loses money in public. But at 8% power the error we are overwhelmingly more likely to be making is Type II. We are far better protected against fooling ourselves than against missing something real.

That is a deliberate asymmetry and we would choose it again. It just should be stated rather than hidden behind a p-value that reads like a verdict.

What would settle it

To resolve the observed +3.17 at 80% power takes 1,918 games. We have 194 — about a tenth of the way, and at roughly 15 graded games a day that is several seasons.

Which is why the signal is neither promoted nor killed. It sits in shadow, scored on every graded game, accumulating. The production model applies no adjustment from it, so nothing you see on the board has been touched by a result with a 0.373 attached to it.

Four things worth taking away

Report S before R-sq. The residual standard deviation tells you what size of effect is even in play. R-sq without it is decoration.

A negative adjusted R-sq is information, not an error. It means the term costs more degrees of freedom than it earns back.

Never read a p-value without its power. "Not significant" from a well-powered test and "not significant" from an 8%-powered test are opposite findings that print identically.

Prefer the confidence interval. (-3.85, 10.19) says everything the p-value says and adds the magnitude, which is the part you can actually act on.

The unglamorous truth is that most honest sports-betting analysis ends here: a wide interval, a small R-sq, and a sample that will take years to say anything. The alternative — treating p = 0.373 as proof of absence, or a lucky subgroup as proof of presence — is how people end up confident and wrong in both directions at once.

Model output is informational and entertainment content, not betting or financial advice. If you bet, bet what you can afford to lose.

← Back to The Lab · How the model works · The record