MOV Cutoff Filter

The next tweak we'll look at for RPI goes back to our previous discussion about the limits of prediction.  We noted there that the last possession of the game could swing the final game score by 6 points.  Even if it's hard to quantify exactly, there's a certain random component in final game scores.  That's particularly a concern for rating systems that rely only on won-loss records, because a swing of a few points in a close game could change a game from a win to a loss.  Intuitively at least, we might want to discount close games when calculating RPI, under the theory that they're not really good evidence that one team was better than the other.

The easiest way to do this is to filter out all games where the final MOV was less than some threshold when computing a team's RPI.  Making that change and applying it with various thresholds to our current best "% Correct" RPI variation gives these results:

Predictor	% Correct	MOV Error
1-Bit	62.6%	14.17
RPI (unw,15+15+70)	75.4%	11.49
RPI (nw, 15+15+70, mov-cutoff=1)	76.8%	11.46
RPI (nw, 15+15+70, mov-cutoff=3)	75.4%	11.56
RPI (nw, 15+15+70, mov-cutoff=8)	73.4%	11.62
RPI (nw, 15+15+70, mov-cutoff=12)	70.0%	11.98

Filtering out all games that were decided by 1 point provides a big improvement in "% Correct" and a small improvement in "MOV Error". It's also interesting to note how resilient RPI is to removing games. MOV cutoff = 12 removes more than 40% of the games and only introduces a few percent more error.

We can try the same technique with our current best "MOV Error" variation (the infinitely deep RPI):

Predictor	% Correct	MOV Error
1-Bit	62.6%	14.17
RPI (improved)	74.6%	11.33
RPI (improved, mov-cutoff=1)	74.2%	11.31
RPI (improved, mov-cutoff=3)	74.6%	11.43
RPI (improved, mov-cutoff=8)	73.4%	11.36
RPI (improved, mov-cutoff=12)	72.0%	11.79

Again, an MOV cutoff of 1 provides a (small) improvement in the MOV Error.

It turns out I lied in the last post when I said the MOV cutoff would be the last tweak we examined for RPI.  In the next posting, I'll take a quick look at calculating RPI using a rolling window that only counts the last "N" games, to see if there's a usable "recency" effect in teams' performances.