Two years ago, after being introduced to the computer program MATLAB at work, I conceived the brilliant plan of writing a computer program to pick the winners of football games against the spread. Armed with a semester’s worth of statistics expertise and a powerful processor, I was confident that I could beat the odds. It seemed so simple to outsmart all those silly rubes making bad bets based on their “gut instincts.” What fools! A clever fellow like me could harness the power of a “little magic box” and elevate myself to the financial status of a king.

My original theory was that Vegas set the point spreads for each game based on public perception – that they were angling to have about 50% of the money on each side. I figured that I could use the statistics from previous games to determine what the “true” spread should be, and it would be an easy way to pick winners. I entered five seasons’ worth of data by hand – point spreads, rushing yardage, turnovers, etc. – and set the computer loose to choose its own algorithm based on what would have worked best the previous several weeks. Based on a large-scale simulation, I thought its odds of working would be pretty good.  Unfortunately, all the program really did was pick underdogs – the bigger the better. And in a season where favorites beat out underdogs 127-116 (including a Patriots squad that covered increasingly outlandish spreads the first eight weeks in a row), it turned into a pretty expensive lesson of what happens when you “overfit the curve.” (more…)