Performance By Week of Season

Python has some nice built-in plotting capabilities, so with a little work I was able to analyze the performance of my predictor by week of the season. This is what that looks like:

The red dotted line is the predictor's average long-term performance.  As you might expect, performance starts off very poorly but improves quite rapidly during the early part of the season.  It gets to average performance after about 4 weeks, but I typically start predicting after 2 weeks (800 games), when the error is still around 12 RMSE.  The more interesting observation is that prediction accuracy gets steadily worse at the end of the season.  Most of the games in those last three weeks of the season are tournament games:  the various conference tournaments followed by the NCAA Tournament.  I assume my predictor is just worse at tournament games.

It's also interesting to look at a different measure of performance: Wins Against the Spread (ATS).  This is the percentage of games where the predictor would have placed a winning bet against the Las Vegas closing line. 

(Again, the red dotted line shows the average performance.)  This shows a different shape than the RMSE plot.  In the late part of the season, the predictor's performance ATS gets very good -- even though its RMSE performance is getting worse.  There's a similar although less obvious trend in the early part of the season, too.  (I'm not sure what's going on in Week 6.)  Apparently my predictor performs poorly in the late season -- but the Vegas lines perform even worse.