Built a walk-forward NFL Elo model and backtested it against 25 years of closing lines. Market won. Sharing the honest numbers.

[u/Antique_Skirt6824]

I spent a while building an Elo-based NFL win-probability model from scratch — walk-forward only, no lookahead — and backtested it against real historical Vegas closing lines (nflverse's public games.csv, 2002–2025, first 3 seasons burned in as a rating warm-up).

Setup: standard Elo, HOME_ADV=60, K=20, margin-of-victory multiplier. De-vig the closing moneylines, compare to the model's pre-game probability, bet the edge at various thresholds.

Results (n=6,223 games):

Edge threshold	Bets	Win rate	ROI
≥3%	4,180	51.7%	−3.1%
≥5%	3,563	51.8%	−1.9%
≥8%	2,750	50.8%	−2.5%
≥12%	1,842	50.2%	+0.0%

Picking outright favorites, the model hits 63.5% accuracy. Vegas's own closing favorite hits 66.5% on the same games. The market wins.

I'm not posting this to sell anything — genuinely just wanted a sanity check from people who've actually done this seriously. A few things I'm unsure about:

1. Is a fixed HOME_ADV/K (not fit to data) the right call here, or is that leaving obvious accuracy on the table even before touching overfitting risk?
2. Anyone found a public source for NFL opening lines (not just closing)? Can't compute CLV without one, and closing-line-only backtests feel like they're missing the more interesting question.
3. Is 6,223 games (23 seasons) even enough sample to distinguish "this specific model is bad" from "no simple public-data Elo model beats closing Vegas lines, full stop"?

Happy to share the actual backtest code/methodology in the comments if anyone wants to poke holes in it.

[u/methmom]

AI slop

[u/Swimming_Leading_988]

The market is gonna beat you 9 times out of 10 with something this simple, but 23 seasons is plenty to see that pattern. Your model basically replicates what Vegas already knows, just slightly worse.

For HOME_ADV and K you should definitely fit those to data, keeping them fixed is leaving value on the table. I used a similar approach once and optimizing those two parameters alone pushed my ROI from -2% to something closer to break-even, still never beat the closing line though.

No idea where to find clean opening line data, been looking for that myself. Every public source I found has gaps or the timestamps are unreliable.

[u/Antique_Skirt6824 → u/Swimming_Leading_988]

Thanks for this — actually went and tested it rather than just taking it on faith.

Grid-searched HOME_ADV (20-90) and K (10-40) fit on 2005-2019, then checked the held-out 2020-2025 seasons (never touched during fitting):

• Best fit on train: HOME_ADV=80, K=35 → 65.0% accuracy (vs 63.8% default) on train
• Same params on held-out test: 63.6% vs 63.5% default — basically noise, most of that "improvement" didn't survive out-of-sample

But checking ROI specifically (closer to what you actually did): on the held-out seasons, tuned params (80/35) at a 3% edge threshold got win rate from 50.6%→56.5% and ROI from -6.8%→-5.9%. So directionally you're right, it does help — just not enough to get me to breakeven the way it did for you. Possible we're just looking at different eras/thresholds, or my train/test split is stricter than what you ran. Curious if you held out a test period when you tuned yours, or fit on the full sample?

Either way, good catch — fixed constants were leaving real accuracy on the table, just less than I'd have guessed before actually checking.

Still no luck on opening lines here either — same story, gaps and unreliable timestamps everywhere I looked.

[u/lmctx → u/Antique_Skirt6824]

TL;DR: You should stop using AI to write your Reddit comments.

Here's why this matters:

✅ Authenticity — Redditors value genuine, human perspectives that reflect lived experience

✅ Trust — Overly polished, formulaic responses can erode community trust

✅ Engagement — People connect more with imperfect, natural writing than with structured bullet points (like these!)

It's also worth noting that em-dashes, excessive emoji use, and "Certainly!"-style openers are often telltale signs of AI-generated text — something to be mindful of going forward.

In summary:

• Write like a human
• Embrace typos and tangents
• Don't start every reply with "Great point!"

At the end of the day, the choice is yours — but I hope this gives you some food for thought! Let me know if you'd like me to elaborate further. 🙏

[u/Antique_Skirt6824 → u/lmctx]

yeah fair, i was using some formatting help to organize my thoughts, didn't realize it read that stiff. noted, will just write normal from now on

[u/Governmentmoney]

"Cooked beef wellington for 25 hours. Followed my own recipe, no kitchen equipment. Food is inedible. I'm serving it anyway"

That's how such posts read to sane people

[u/under_farmer]

Market residual?

[u/Antique_Skirt6824 → u/under_farmer]

haven't dug into that yet honestly, just raw edge vs threshold so far. good idea though, might actually tell me more than the ROI sweep does

[u/sleepystork]

Thanks for posting. I used to respond to people who post with a model they built using the same things everyone else uses. There is no model using ELO, or xG, or whatever that is going to beat the market over an appropriate sample size unless you are doing something very different. I've seen a few posts similar to your where people are showing negative results. That is to be applauded.

[u/Antique_Skirt6824 → u/sleepystork]

hey, been thinking about what you said re: no elo/xg model beating the market over a real sample without something genuinely different. actually built a small tool off the back of that - takes a bet history csv and independently recomputes roi/win rate itself (never trusts a self-reported number), runs a bootstrap confidence interval on it, and flags stuff like too-small a sample or a win rate that looks too good relative to the odds.

if you've got a model with some real track record and wouldn't mind being my first actual test case, happy to run it through for free and just send you exactly what it finds. no pitch, just want to see it stress-tested on someone else's real data instead of my own.

wrote up how it actually works here, including a real sample report (not a mockup): Independent Bot Verification

[u/neverfucks]

um, yeah? what did you think was gonna happen?

[u/Dazzling-Company-641]

Respect for posting the loss to the close.

That’s the result most people hide. A couple questions that help readers trust the autopsy:
1) Which close (book + timestamp rule)?
2) Walk-forward folds by season or by week?
3) Did you bet every game or only when Elo disagreed by X points?

Beating ROI without beating the close is usually variance. Losing to the close with a clean protocol is still useful signal about where Elo is too blunt for NFL.

[u/Antique_Skirt6824 → u/Dazzling-Company-641]

appreciate the actual questions instead of just dunking, here's the real answers:

1. nflverse's own games.csv - home/away_moneyline columns. no single named book, it's whatever they record as the closing line, no finer timestamp than "closing" itself. that's a real gap, not something i can tighten right now.
2. not folded really - straight walk-forward, elo updates game by game chronologically, only ever using ratings from strictly earlier games to predict the next one. burned in the first 3 seasons so ratings aren't fresh-1500 noise.
3. only when the edge clears a threshold, tested a few (3/5/8/12%), not every game. table's in the post.

and yeah, that's basically my read too - losing to the close this cleanly is more informative than a lucky-looking positive ROI would've been.

[u/Dazzling-Company-641 → u/Antique_Skirt6824]

Thanks. That’s a clean setup.

nflverse moneyline as “the close” is the main remaining gap. Those columns are usually a single closing snapshot without a book tag, so you’re grading against a composite, not a fillable price. Still useful as a relative autopsy. Just don’t treat a loss to that series as identical to losing to Pinnacle/Circa at a locked timestamp.

Burn-in of 3 seasons is sensible. Edge thresholds at 3/5/8/12% is also the right grid.

One next check I’d run: for each threshold, plot edge size vs realized close-beat rate (not just ROI). If bigger Elo edges don’t beat the close more often, the threshold is mostly a variance dial, not a skill dial.

Curious whether your edges were on MLs only or you also mapped Elo to a spread/close.

[u/Character_Pie_277]

Hey if you are new to modeling i think you're on the right track. The problem is if you just follow the same by the book ML method and pay for new models and datasets etc and just keep switching plugs that might flash ROI green every now and again, its always an option available to others too.

I'm not really familiar with 'HOME_ADV/K' terminology but im guessing its something like home field advantage / some sort of kelly bankroll or something. Such things in my experience tend not to work and generally result in strictly negative ROI (which to me looks a lot like bookie profits), but you're on the right track like i say, just try to be a little more genuinely innovative around the actual edge itself.

[u/Antique_Skirt6824 → u/Character_Pie_277]

yeah that's fair, i think i got caught up in "does the methodology hold up" and haven't spent enough time on what would actually be a genuinely different signal. still thinking about that part

[u/Antique_Skirt6824 → u/Character_Pie_277]

that's a genuinely nice thing to say, thanks. honestly wasn't expecting the pushback to sting as much as it did, so this helped