One Bad (Good) Game

As mentioned in my previous posting, I recently looked at the effect of dropping a football team's best game (highest Margin of Victory) and their worst game (lowest MOV).  The intuitive notion is that everybody has bad days, where everything goes wrong, and good days, where everything goes right, and maybe those days don't tell us anything useful about the real strength of a team.  If that's so, then dropping those games might give us ratings that are more accurate.

To test this hypothesis I implemented this "drop the worst score" grading system for a couple of the rating systems I use for football and measured performance in the usual way.  Here are the results for one of the rating systems:

Predictor	% Correct	MOV Error
BGD Baseline	73.7%	16.52
BGD w/o blowouts or lowouts	72.6%	16.77
BGD w/o lowouts	72.9%	16.69
BGD w/o blowouts	73.6%	16.62

Here I'm using the whimsical "lowout" to indicate the worst loss for a team.

As this shows, eliminating the blowouts/lowouts hurts predictive performance.  For what it's worth, the losses seem to be more important than the wins.  (I saw the same effect in basketball when I looked at this last year.)

The Impact of MOV Cutoffs in Football Ratings

I was prompted to start my football predictions by a discussion on an email list of the value of MOV cutoffs in rating systems.  Roger Dendy believed that capping the MOV in blowout victories improved his rating system.  My testing of MOV cutoffs in basketball has shown just the opposite -- that no matter how big the blowout, there's always information in the margin of victory.  Capping MOV at any level (in both blowouts and nailbiters) always reduces the prediction value of a rating.

Of course, just because that's true in basketball doesn't mean it's true in football.  I was pretty sure it was true, but I believe in "trust but verify."  So I put together the football predictor and tested a couple of different rating systems both with and without MOV caps.

I have many rating systems that use MOV, so I picked one and measured it's performance with a 100-fold X-validation  across my archive of college football scores from 2005 to date.  It had a RMS of 16.78 and predicted 71% of the games correctly.

Then I experimented with adding a cutoff to the MOV.  I set the cutoff to 32 points, so that all the games where the MOV exceeded 32, it would be treated as 32.  I just picked 32 arbitrarily as a good figure for a blowout win.  The performance degraded to RMS=17.19 and 69%.  I then bumped up the cutoff to 48 points, and the performance was RMS=17.01 and 70%.

The other rating system showed a similar pattern of performance.

What this shows -- at least for the two rating systems I tested and these performance metrics -- is that even huge margins of victory have value in assessing future performance.  People argue intuitively that there's "no difference between winning by 48 and winning by 52" but that appears not to be true.

Recently I got to wondering if it might not make more sense to drop a blowout victory entirely.  This would be like "drop your lowest score" grading in high school.  The intuitive notion here is that sometimes teams just have a bad day -- a few unlucky bounces and worse goes to worse.  Or lucky bounces and better goes to better, from the other side of the coin.  More on that notion next time.

Football Predictions

Here are college football predictions for this week.  I discovered a couple of different bugs in my input data since last weeks predictions; these should be somewhat better.  Apologies as always for the old-school formatting, and heed my Disclaimer as well.

+--------------------+--------------------+--------------------+
|Hname               |Aname               |prediction(mov)     |
+--------------------+--------------------+--------------------+
|ohio state          |wisconsin           |-11.7               |
+--------------------+--------------------+--------------------+
|western michigan    |ball state          |15.5                |
+--------------------+--------------------+--------------------+
|washington          |arizona             |7.4                 |
+--------------------+--------------------+--------------------+
|duke                |virginia tech       |-4.3                |
+--------------------+--------------------+--------------------+
|utah                |oregon state        |1.5                 |
+--------------------+--------------------+--------------------+
|ucla                |california          |1.7                 |
+--------------------+--------------------+--------------------+
|central florida     |memphis             |26.0                |
+--------------------+--------------------+--------------------+
|tulsa               |southern methodist  |5.4                 |
+--------------------+--------------------+--------------------+
|texas tech          |iowa state          |18.5                |
+--------------------+--------------------+--------------------+
|texas a&m           |missouri            |16.1                |
+--------------------+--------------------+--------------------+
|texas               |kansas              |22.7                |
+--------------------+--------------------+--------------------+
|southern california |stanford            |-10.0               |
+--------------------+--------------------+--------------------+
|texas-el paso       |southern mississippi|-8.1                |
+--------------------+--------------------+--------------------+
|tennessee           |south carolina      |-.1                 |
+--------------------+--------------------+--------------------+
|san diego state     |wyoming             |20.3                |
+--------------------+--------------------+--------------------+
|rutgers             |west virginia       |2.5                 |
+--------------------+--------------------+--------------------+
|penn state          |illinois            |5.0                 |
+--------------------+--------------------+--------------------+
|oregon              |washington state    |24.9                |
+--------------------+--------------------+--------------------+
|oklahoma state      |baylor              |7.3                 |
+--------------------+--------------------+--------------------+
|notre dame          |navy                |17.6                |
+--------------------+--------------------+--------------------+
|indiana             |northwestern        |-5.7                |
+--------------------+--------------------+--------------------+
|north carolina      |wake forest         |5.3                 |
+--------------------+--------------------+--------------------+
|new mexico state    |nevada              |-6.2                |
+--------------------+--------------------+--------------------+
|nebraska            |michigan state      |-6.6                |
+--------------------+--------------------+--------------------+
|kentucky            |mississippi state   |-11.1               |
+--------------------+--------------------+--------------------+
|michigan            |purdue              |22.1                |
+--------------------+--------------------+--------------------+
|miami (ohio)        |buffalo             |1.0                 |
+--------------------+--------------------+--------------------+
|maryland            |boston college      |6.8                 |
+--------------------+--------------------+--------------------+
|marshall            |alabama-birmingham  |14.2                |
+--------------------+--------------------+--------------------+
|louisville          |syracuse            |-5.3                |
+--------------------+--------------------+--------------------+
|louisiana tech      |san jose state      |11.6                |
+--------------------+--------------------+--------------------+
|louisiana-monroe    |western kentucky    |-6.2                |
+--------------------+--------------------+--------------------+
|middle tennessee    |louisiana-lafayette |4.7                 |
|state               |                    |                    |
+--------------------+--------------------+--------------------+
|kansas state        |oklahoma            |-4.0                |
+--------------------+--------------------+--------------------+
|minnesota           |iowa                |-13.5               |
+--------------------+--------------------+--------------------+
|idaho               |hawaii              |-8.0                |
+--------------------+--------------------+--------------------+
|florida             |georgia             |2.4                 |
+--------------------+--------------------+--------------------+
|florida state       |north carolina state|13.2                |
+--------------------+--------------------+--------------------+
|east carolina       |tulane              |9.9                 |
+--------------------+--------------------+--------------------+
|nevada-las vegas    |colorado state      |1.3                 |
+--------------------+--------------------+--------------------+
|georgia tech        |clemson             |-4.4                |
+--------------------+--------------------+--------------------+
|akron               |central michigan    |-1.8                |
+--------------------+--------------------+--------------------+
|kent                |bowling green state |-5.2                |
+--------------------+--------------------+--------------------+
|auburn              |mississippi         |11.5                |
+--------------------+--------------------+--------------------+
|arkansas state      |north texas         |15.2                |
+--------------------+--------------------+--------------------+
|vanderbilt          |arkansas            |-5.3                |
+--------------------+--------------------+--------------------+
|arizona state       |colorado            |26.3                |
+--------------------+--------------------+--------------------+
|new mexico          |air force           |-16.8               |
+--------------------+--------------------+--------------------+
|florida             |troy                |11.7                |
|international       |                    |                    |
+--------------------+--------------------+--------------------+
|pittsburgh          |connecticut         |11.4                |
+--------------------+--------------------+--------------------+
|brigham young       |texas christian     |-10.1               |
+--------------------+--------------------+--------------------+
|miami (florida)     |virginia            |15.8                |
+--------------------+--------------------+--------------------+
|houston             |rice                |21.5                |
+--------------------+--------------------+--------------------+

Predicting the Oblong Ball

I was recently challenged by some friends to predict NCAA college football, so I gathered up some historical data from this archive and adapted some of the better rating systems I've investigated to create a predictor.  It's hard to judge the performance.  It does not perform as well as the systems reported here according to my standard cross-validation testing, but my implementation of Sagarin's ELO also underperforms the reported performance.  Since my implementation of ELO tracks the Sagarin performance very well in basketball, I suspect there's a systemic difference in how performance is measured.

At any rate, I don't intend to spend a lot of time on this, but just for amusement, here are the predictions for this weeks games:

alabama over tennessee by 10.6
arkansas over mississippi by 14.8
ball state over central michigan by 1.8
boise state over air force by 25.7
california over utah by -11.4
central florida over alabama-birmingham by 21.2
clemson over north carolina by 5.1
florida atlantic over middle tennessee state by -12.4
florida state over maryland by 5.9
hawaii over new mexico state by .5
houston over marshall by 12.9
illinois over purdue by 11.0
iowa over indiana by 6.5
kansas state over kansas by 18.7
louisiana state over auburn by 14.4
louisiana-lafayette over western kentucky by 6.5
miami (florida) over georgia tech by -15.4
navy over east carolina by 5.2
nebraska over minnesota by 14.4
nevada over fresno state by 7.8
north texas over louisiana-monroe by 2.7
northern illinois over buffalo by 3.1
notre dame over southern california by 2.5
ohio over akron by 21.1
oklahoma state over missouri by 9.9
oklahoma over texas tech by 9.4
oregon over colorado by 22.8
penn state over northwestern by 10.9
rutgers over louisville by 12.0
south florida over cincinnati by -6.9
southern mississippi over southern methodist by -4.4
stanford over washington by 17.7
temple over bowling green state by 17.2
texas a&m over iowa state by 14.5
texas christian over new mexico by 21.9
texas-el paso over colorado state by -1.8
toledo over miami (ohio) by 16.4
tulane over memphis by 10.2
tulsa over rice by 4.4
ucla over arizona by 1.6
utah state over louisiana tech by -2.9
vanderbilt over army by 5.1
virginia tech over boston college by 15.6
virginia over north carolina state by -4.4
wake forest over duke by -2.3
washington state over oregon state by 5.2
west virginia over syracuse by 3.2
western michigan over eastern michigan by 13.7
wisconsin over michigan state by 7.0

Apologies for the awful formatting -- I put this together in 3 days and didn't put much effort in to making pretty.

The Usual Disclaimers apply:  Use this information at your own risk; it is not intended for gambling purposes and the Net Prophet does not encourage or recommend gambling on sports events.