The Recency (Non-)Effect

Our final RPI tweak (unless I'm lying again) will look at the impact of recent results on predictability.  It's reasonable to suppose that a team's level of performance might change during the season -- that is, it might get better or worse as the season goes along.  In this case,recent games might be a better predictor of future performance than older games

To test this notion, we can modify our RPI calculations so that they take into account only the last "N" games.  For some representative values of "N", that yields these results:

Predictor	N	% Correct	MOV Error
1-Bit		62.6%	14.17
RPI (nw, 15+15+70)		76.8%	11.46
RPI (nw, 15+15+70)	4	67.6%	12.52
RPI (nw, 15+15+70)	8	72.8%	11.89
RPI (nw, 15+15+70)	16	74.2%	11.47

Restricting the RPI calculations to the most recent games has a strong negative impact on predictive power.  Of course, this method is fairly drastic: it gives 100% value to the most recent games and 0% to anything older.  A more nuanced approach would count the most recent games more, but not discount entirely the older games.  Something like a weighted moving average would be ideal, but it isn't entirely obvious how to apply that to RPI.  Instead, we'll take the approach of counting the most recent games more than once.  That is, we'll treat each team as if it played it's most recent games multiple times (with the same results).  This will have the effect of weighting those games correspondingly more.

This table shows the impact of repeating some number of recent games some number of times (in addition to counting all games once):

Predictor	N	Repeats	% Correct	MOV Error
RPI (nw, 15+15+70)	4	1	72.8%	11.63
RPI (nw, 15+15+70)	8	1	74.2%	11.56
RPI (nw, 15+15+70)	16	1	74.6%	11.49
RPI (nw, 15+15+70)	4	1/3	73.6%	11.64
RPI (nw, 15+15+70)	8	1/3	75.4%	11.58
RPI (nw, 15+15+70)	16	1/3	75.2%	11.51
RPI (nw, 15+15+70)	2	2	70.6%	12.49
RPI (nw, 15+15+70)	1	2	66.2%	12.82

In no case that I could find did weighting recent games improve performance over the baseline.  Putting emphasis on a small number of recent games is particular bad; this suggests that if teams do change performance over the course of the season it is only slowly.

Next time (unless something shiny distracts me again), we'll sum up the various tweaks we've tried with RPI.