Last week, I wrote what I thought was a pretty interesting post about how I believe that Vegas is mispricing some of the Major League Baseball teams. According to that article, there is some sort of edge to going long the Dodgers and the Cub while shorting the Pirates, Yankees, Braves, and Cardinals this coming season.
There was a lot of thinking done by a lot of people who are far smarter than I am for this research. At the end of the day, I said that I believed this to be a good bet, but the exposure to a small sample size meant that the risk of ruin was too great to invest real capital. These are good gambles, but not smart investments.
Despite being fairly convicted that I would not risk my own money (at least not with an expectation of a positive outcome) on this strategy, I shared it with the New Traders, Rich Traders facebook community that is run by Steve Burns. In the comments for that post, Travis James suggested that I look into backtesting the model on historical data.
Backtesting Baseball Futures
I was able to locate an article from February of 2013 that had the PECOTA projections for each team’s total wins as well as the over/under numbers from Vegas at the time. I copied those numbers into a spreadsheet and then added the number of games that each of the teams actually won in 2013.
The results weren’t good:
As you can see, using three games as the threshold for a significant difference would have given us three long positions and six short positions last season. That would have resulted in 3 wins, 5 losses, and one push. Even if we were able to get better odds through shopping around to turn the push into a win, we still would have lost money on the season.
My initial thought was that maybe three games isn’t a large enough difference to be significant. However, both there were four teams projected to five games or more below their over/under and those bets would have produced 1 Win, 2 Losses, and a push.
Going Further Back
I wasn’t able to find any PECOTA projections or over/unders from further back than 2013 online. I do have the evidence from Trading Bases though.
In the book, Joe decided to use five games as his threshold for a significant difference between his projections and the over/under. This gave him a total of nine bets, four longs and five shorts.
Of those nine bets, Joe produced 6 winners and only 3 losers.
Small Sample Size
The problem here is that none of these backtests mean anything. Using trades from just 2011 and 213 is nowhere near enough data to make an accurate determination.
With that sample size idea in mind, there is no way that you can project to have consistent success with this betting approach year after year. The bets would probably indicate an edge if you were to combine 10-20 years of bets into one backtest, but there is nothing to prevent wild swings in profitability from year to year. These wild swings could produce a black swan that completely wipes out an account in any single year.
For that reason, we need to abandon this futures idea and focus on a daily strategy that will generate enough trades to give us some statistical significance and repeatability.