tag:blogger.com,1999:blog-59026203365096470502024-03-12T19:48:48.997-04:00Net ProphetExploring algorithms for predicting NCAA basketball games.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.comBlogger266125tag:blogger.com,1999:blog-5902620336509647050.post-28084769707169034062017-04-04T12:56:00.000-04:002017-04-04T12:56:09.357-04:002017 Machine Madness WinnerThe 2017 NCAA season ended with a win by UNC over Gonzaga in a title game marred by both poor shooting and excessive officiating. I didn't have much time/enthusiasm for college basketball this year, but I did want to take a moment to congratulate the winner of the <a href="http://www.ultimatebracketchallenge.com/Pool/AutoJoinPoolPage.aspx?Pool=1247&PoolPassword=machines">Machine Madness competition</a>... me. :-)<br />
<br />
My entry barely squeaked out a win over <a href="http://erikforseth.com/">Erik Forseth</a>'s entry. The Net Prophet entry to tournament pools takes the base prediction out of the prediction model and then picks some number of upsets (usually 8). Two of those upsets this year were Oregon to the Final Four, and UNC over Gonzaga in the final game. (Like most machine models, I had Gonzaga stronger than UNC at the start of the tournament.) Over at the ESPN Tournament Challenge, the Net Prophet entry finished with 1500 points, good enough for 19,407th out of 13+ million brackets. Not bad.<br />
<br />
Lest my head swell too much, I have to point out that over in the Kaggle competition I finished in 416th place. This year I didn't have much time to spend on the contest, so I ran the code for last year with some quick hacks in place to get it working and although it generated a legal entry, I had no faith in the results, so I'm not surprised to see the poor showing. But my model has always done worse on Kaggle than in bracket competitions, so perhaps this was not a surprising result.<br />
<br />
More notably, Monte McNair and the aforementioned Erik Forseth finished 3rd and 4th in the Kaggle contest. That's tremendously good results for both of them, congratulations!Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com8tag:blogger.com,1999:blog-5902620336509647050.post-65415561752724815472017-03-09T20:02:00.004-05:002017-03-09T20:02:59.068-05:002017 Machine Madness CompetitionIf anyone is still reading this blog, you've no doubt noticed the lack of posts this year. I've been busy with other things, and to be honest, the predictor has performed poorly for the last couple of seasons, which reduces my motivation. Monte McNair, on the other hand, is still quite active and has just opened up the 2017 Machine Madness Competition. If you're interested in joining, you can find the pool <a href="http://www.ultimatebracketchallenge.com/Pool/AutoJoinPoolPage.aspx?Pool=1247&PoolPassword=machines">here</a>. Good Luck!Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com7tag:blogger.com,1999:blog-5902620336509647050.post-23520723054997827702016-04-08T14:20:00.002-04:002016-04-08T14:20:44.889-04:002016 Machine Madness WinnerI've been a little slow in getting around to this, but I want to congratulate "SDSU Fan" on winning the <a href="http://www.ultimatebracketchallenge.com/Pool/PoolHomePage.aspx?Pool=1225">2016 Machine Madness</a> contest! In real life, SDSU Fan is Peter Calhoun, a graduate student in Statistics at (no surprise) San Diego State University. We had a very large pool of entrants this year (40!) so Peter deserves some congratulations for beating the masses. Peter was trailing by a significant amount after the Round of 32, but strong performances in the later rounds (and especially the FF) resulted in big lead by the end.<br />
<br />
Peter's model modified the Logistic Regression/Markov Chain (LRMC) approach proposed by Kvam and Sokol to use random forests. Peter also finished in fiftieth on Kaggle -- a very strong performance all around.<br />
<br />
Despite the large number of entries, nobody had Villanova winning it all. I think that makes the Villanova win a "true upset". I know in my model, Villanova played considerably better than predicted.<br />
<br />
Speaking of my model, it follows a strategy in pool-based contests of picking some "likely" upsets to try to maximize the chance of winning. (This is probably more important in a larger pool.) This year, it picked Purdue to make it to the Championship Game. Not only didn't that happen, Purdue was upset in the first round by #12 Little Rock. I'm adding a special "Purdue Rule" to the Net Prophet model so that mistake is never again repeated. :-)<br />
<br />
Congratulations again to Peter on great performance! Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com1tag:blogger.com,1999:blog-5902620336509647050.post-53710221308771049602016-04-08T13:47:00.000-04:002016-04-08T13:48:55.063-04:00Paper Reviews<div class="title mathjax">
These papers have been added to the paper archive available through the Papers link on the sidebar. Links are also provided for direct download of the papers.<i><span style="font-weight: normal;"> </span></i><br />
<br />
<i><span style="font-weight: normal;">Dubbs, Alexander, "Statistics-Free Sports Prediction", <a href="http://arxiv.org/abs/1512.07208">arXiv.org</a></span></i></div>
<div class="title mathjax">
<span style="font-weight: normal;"></span></div>
<blockquote class="tr_bq">
<div class="title mathjax">
<span style="font-weight: normal;">The author builds logistic regression models for MLB, NBA, NFL, and NHL games that use only the teams and scores. This works best for basketball, and the author concludes that "in basketball, most statistics are subsumed by the scores of the games, whereas in baseball, football, and hockey, further study of game and player statistics is necessary to predict games as well as can be done."</span></div>
<div class="title mathjax">
<span style="font-weight: normal;"><br /></span></div>
<div class="title mathjax">
<span style="font-weight: normal;">COMMENT: I'm not sure the results of this paper say anything deeper than "Compared to the other major sports, NBA has a long season and the teams don't change much from year to year." </span></div>
</blockquote>
<div class="pub-title" id="yui_3_14_1_1_1459974117718_1219" itemprop="name">
<i>Clay, Daniel, "Geospatial Determinants of Game Outcomes in NCAA Men’s Basketball," <a href="https://www.researchgate.net/publication/272507595_Geospatial_Determinants_of_Game_Outcomes_in_NCAA_Men's_Basketball">International journal of sport and society 02/2015; 4(4):71-81.</a></i></div>
<div class="pub-title" id="yui_3_14_1_1_1459974117718_1219" itemprop="name">
</div>
<blockquote class="tr_bq">
<div class="pub-title" id="yui_3_14_1_1_1459974117718_1219" itemprop="name">
The authors build a logistic regression model for 1,648 NCAA Tournament games that include features for distance travel, time zones crossed, direction of travel, altitude and temperature. They conclude "We found that traveling east reduces the odds of winning more than does traveling west, and this finding holds when controlling for strength of team, home region advantage and other covariates. Traveling longer distances (>150 miles) also has a dramatic negative effect on game outcomes..."</div>
</blockquote>
<blockquote class="tr_bq">
COMMENT: This paper shows that travel distance and direction has a statistically significant impact upon game results in the NCAA Tournament, but I want to add a few caveats to this conclusion. First, it isn't clear that the authors understand and control for the fact that there are many more basketball programs (and arguably stronger basketball programs) on the East Coast than elsewhere in the nation. For this reason, it's likely that teams moving west to play in the Tournament are stronger than teams moving east. Since the authors don't control for the strength of teams, it's impossible to say whether the claimed impact of direction of travel means anything. Second, the magnitude of these effects may not be huge. I don't understand how the authors calculate their "Odds Ratio" but factors like strength of team are several orders of magnitude more significant in determining outcome. Third, the authors are measuring strength of team by seed, which has several problems. It's a very coarse measure, it doesn't distinguish between teams with the same seed, and it's often poorly correlated with the actual team strength (i.e., teams are commonly mis-seeded). In my experience, many factors with low significance vanish when team strength is more accurately estimated. I think distance and direction of travel probably do have an impact on Tournament games, but I suspect the true effect is smaller than this paper would indicate.</blockquote>
Clay, Daniel, "Player Rotation, On-court Performance and Game Outcomes in NCAA Men's Basketball", <a href="https://www.researchgate.net/publication/264090235_Player_Rotation_On-court_Performance_and_Game_Outcomes_in_NCAA_Men%27s_Basketball">International Journal of Performance Analysis in Sport · August 2014</a><a href="https://www.researchgate.net/publication/264090235_Player_Rotation_On-court_Performance_and_Game_Outcomes_in_NCAA_Men's_Basketball"></a><br />
<br />
<blockquote class="tr_bq">
The authors look at the relationship between the size of rotation (how many players play at least 10 minutes in a game) and statistics such as rebounding, shooting percentage, etc. The authors conclude that teams with deep rotation tend to rebound better, particularly on the offensive end. They also have more steals. By contrast, smaller rotation teams tend to shoot the ball better, both field goals and free throws, and they are more effective at taking care of the ball, resulting in fewer turnovers. In general, a larger rotation improves the chance of winning.</blockquote>
<blockquote class="tr_bq">
COMMENT: There's quite a bit of interesting material in this paper, and I recommend reading it and drawing your own conclusions. I have reservations about some of the conclusions in this paper because the authors have not controlled for number of possessions in the game for many of the statistics. Since I'd expect (for example) that both the number of offensive rebounds and the depth of rotation to increase with more possessions, I'm not sure I immediately accept that teams with deeper rotations rebound better. The authors do control for possessions in two of the statistics (offensive and defensive rating) and those conclusions are more convincing. However, as far as I can tell the authors did nothing to control for overtime games, and that may also be affecting the results. </blockquote>
<blockquote class="tr_bq">
From the specific viewpoint of predicting game outcomes, the authors don't make use of any kind of strength rating, so it isn't clear whether depth of rotation has any predictive value that wouldn't already be covered by a good strength metric.</blockquote>
Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com1tag:blogger.com,1999:blog-5902620336509647050.post-71265174146284281452016-03-28T20:57:00.000-04:002016-03-28T20:57:09.993-04:00Sorry About That!I have to apologize to anyone who Stole My Entry over on Kaggle, because the Net Prophet predictor has made a hash of it this Tournament, and is mired low in the Leaderboard and well below the median entry. A number of the upsets have been very improbable according to the Net Prophet predictor and it has suffered accordingly.<br />
<br />
It's worth noting that some others have been suffering too: Monte McNair has done better than Net Prophet but not by a whole lot. Ken Massey entered for the first time and is very low on the Leaderboard (apparently because he gambled rather heavily on 2-15 matchups). The most interesting story is ShiningMGF, who started poorly (perhaps because their first-round predictions are influenced by the Vegas lines?) but have been climbing steadily and are now in tenth place. Top Ten finishes three years running is almost certainly a good indication that they know something the rest of us don't!<br />
<br />
Over at the <a href="http://www.ultimatebracketchallenge.com/Pool/PoolHomePage.aspx?Pool=1225">Machine Madness</a> contest, Net Prophet isn't doing any better, being one of the many entries that predicted Kansas as the eventual champion. It looks like "SDSU" has the win locked up already. "Predict the Madness" is likely to finish second unless North Carolina loses the next game. Beyond that it gets a little murky, but all the entries with UNC winning it all have an obvious advantage.<br />
<br />
But regardless of who wins, it's been a great turnout for the contest (40 entries!) and I want to give my sincere thanks to everyone who entered. It's really great to see so much interest and participation!<br />
<br />
<br />Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com3tag:blogger.com,1999:blog-5902620336509647050.post-17025716086004520812016-03-22T20:29:00.000-04:002016-03-23T13:33:50.794-04:00What Would a Perfect (Knowledge) Predictor Score in the Kaggle Competition?It isn't possible to have a perfect predictor for NCAA Tournament games, because the outcome is probabilistic. We can't know for sure who is going to win a game. But we could conceivably have a predictor with <i>perfect knowledge</i>. This predictor would know the true probability for every game. That is, if Duke is 75% likely to beat Yale, the perfect knowledge predictor would provide that number. (Because predicting the true probability results in the best score in the long run.) What would such a predictor score in the Kaggle Contest?<br />
<br />
The Kaggle contest uses a log-loss scoring system. In this system, a correct prediction is worth the log of the confidence of the prediction, and an incorrect prediction is worth one minus the log of the confidence of the prediction. (And for the Kaggle contest the sign is then swapped so that smaller numbers are better. <br />
<br />
Let's return to our example of Duke versus Yale. Our perfect knowledge predictor predicts Duke over Yale with 0.75 confidence. What would this predictor score in the long run? (I.e., if Duke and Yale played thousands of times.) Since the prediction is also the true probability that Duke will win, that number is given by the equation:<br />
<br />
<div style="text-align: center;">
`0.75 * ln(0.75) + (1-0.75) * ln(1-0.75)`</div>
<br />
that is, 75% of the time Duke will win and in those cases the predictor will score ln(0.75), and 25% of the time Yale will win and the predictor will score ln(0.25). This happens to come out to about -0.56 (or 0.56 in Kaggle terms).<br />
<br />
<div style="text-align: left;">
</div>
<div style="text-align: left;">
So we see how to calculate the expected score of our perfect knowledge predictor given the true advantage. If the favorite in all the Tournament games was 75% likely to win, then our perfect predictor would be expected to score 0.56. But we don't know the true advantage in Tournament games, and they're all different advantages. Is there some way we can estimate this?</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
One approach is to use the historical results. We know how many games were upsets in past Tournaments, so we can use this to estimate the true advantage. For example, we can look at all the historical 7 vs. 12 matchups and use the results to estimate the true advantage in those games. (One problem with this approach is that in every Tournament, some teams are "mis-seeded". If we judge upsets by seed numbers, this adds some error.)</div>
<div style="text-align: left;">
<br /></div>
Between this <a href="https://en.wikipedia.org/wiki/NCAA_Men's_Division_I_Basketball_Championship_Upsets">Wikipedia page</a> and this <a href="http://espn.go.com/mens-college-basketball/story/_/id/14943099/upsets-seeds-teams-college-basketball">ESPN page</a> we can determine the win percentages for every possible first-round matchup. There have been a reasonable number of these matchups (128 for each type of first-round matchup) so we can have at least a modicum of confidence that the historical win percentage is indicative of the true advantage:<br />
<div style="text-align: center;">
<br /></div>
<table class="tableizer-table" style="margin-left: auto; margin-right: auto; text-align: left;">
<thead align="center">
<tr class="tableizer-firstrow"><th>Seed</th><th>Win Pct</th></tr>
</thead><tbody>
<tr align="center"><td>1 vs. 16</td><td>100%</td></tr>
<tr align="center"><td>2 vs. 15</td><td>94%</td></tr>
<tr align="center"><td>3 vs. 14</td><td>84%</td></tr>
<tr align="center"><td>4 vs. 13</td><td>80%</td></tr>
<tr align="center"><td>5 vs. 12</td><td>64%</td></tr>
<tr align="center"><td>6 vs. 11</td><td>64%</td></tr>
<tr align="center"><td>7 vs. 10</td><td>61%</td></tr>
<tr align="center"><td>8 vs. 9</td><td>51%</td></tr>
</tbody></table>
<br />
Using the win percentage as the true advantage, we can then calculate what our perfect knowledge predictor would score in each type of match-up:<br />
<br />
<center>
<table class="tableizer-table">
<thead>
<tr class="tableizer-firstrow"><th>Seed</th><th>Win Pct</th><th>Score</th></tr>
</thead><tbody>
<tr><td>1 vs. 16</td><td>100%</td><td>0.00</td></tr>
<tr><td>2 vs. 15</td><td>94%</td><td>-0.22</td></tr>
<tr><td>3 vs. 14</td><td>84%</td><td>-0.45</td></tr>
<tr><td>4 vs. 13</td><td>80%</td><td>-0.50</td></tr>
<tr><td>5 vs. 12</td><td>64%</td><td>-0.65</td></tr>
<tr><td>6 vs. 11</td><td>64%</td><td>-0.65</td></tr>
<tr><td>7 vs. 10</td><td>61%</td><td>-0.67</td></tr>
<tr><td>8 vs. 9</td><td>51%</td><td>-0.69</td></tr>
</tbody></table>
</center>
<br />
Since there are equal numbers of each of these games, the average performance of the predictor is just the average of these scores: -0.48.<br />
<br />
This analysis can be extended in a straightforward way to the later rounds of the tournament, but since there are fewer examples in each category it's hard to have much faith in some of those numbers. But I would expect the later round games to make the perfect knowledge predictor's score worse, because more of those games are going to be close match-ups like the 8 vs. 9 case.<br />
<br />
So 0.48 probably represents an optimistic lower bound for performance in the Kaggle competition.<br />
<br />
UPDATE #1:<br />
<br />
Here's an rough attempt to estimate the performance of the perfect predictor in the other rounds of the Tournament.<br />
<br />
According to the <a href="https://en.wikipedia.org/wiki/NCAA_Men's_Division_I_Basketball_Championship_Upsets">Wikipedia page</a>, there have been 52 upsets in the remaining rounds of the Tournament (a rate of about 2%). If we treat all these games as having an average seed difference of 4 (which is a conservative estimate), then our log-loss score on these games would be about -0.66. (Intuitively, this is as we would expect -- with most of the low seeds eliminated, games in the later rounds are going to be between teams that are more nearly equal in strength, so our log-loss score will be correspondingly worse.) Since there are as many first round games as all the other rounds, the overall performance is just the average of -0.48 and -0.66: 0.57.<br />
<br />
UPDATE #2:<br />
<br />
Over in the <a href="https://www.kaggle.com/c/march-machine-learning-mania-2016/forums/t/19720/what-would-a-perfect-predictor-score/112764#post112764">Kaggle thread</a> on this topic, <a href="https://www.kaggle.com/goodspellr">Good Spellr</a> pointed out that if you treat the first round games as independent events with a normal distribution, you can estimate the variance as well:<br />
<br />
<center>
`variance = (1/n^2) sum_(i=1)^n p_i*(1 - p_i)*(Log[p_i/(1 - p_i)])^2`</center>
<center>
<br />
</center>
which works out to a standard deviation of about 0.07. That means that after the first
round of the tournament, the perfect prediction would fall in the range
[0.34, 0.62] about 95% of the time.<br />
.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com4tag:blogger.com,1999:blog-5902620336509647050.post-47251078907971944702016-03-20T20:45:00.001-04:002016-03-20T20:45:27.352-04:00A Quick UpdateI'm still in Brooklyn watching games (well, we're done watching now -- had a couple of fun games) and have been too busy to do more than minimum checking of email, but I found time to check on the <a href="http://www.ultimatebracketchallenge.com/Pool/PoolHomePage.aspx?Pool=1225">Machine Madness</a> contest. I see that we have an amazing 40 contestants -- presumably most found us through the Kaggle Contest, but it's great to see the participation. What's not so great is that the Net Prophet entry is doing poorly both here and at the Kaggle Contest, but that's a post for another day :-) Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com1tag:blogger.com,1999:blog-5902620336509647050.post-35507623262832863982016-03-15T23:47:00.000-04:002016-03-15T23:47:25.764-04:00Year End RankingsI'm not really into ranking teams that much (because match-ups depend on many more factors), but I came up with a new (and I think better) rating system today and here's how it ranks the Top Twenty:<br />
<br />
<center>
<table class="tableizer-table">
<thead>
<tr class="tableizer-firstrow"><th>Rank</th><th>Team</th><th>Rating</th></tr>
</thead><tbody>
<tr><td>1</td><td>North Carolina</td><td>131.6</td></tr>
<tr><td>2</td><td>Kansas</td><td>129.6</td></tr>
<tr><td>3</td><td>Michigan State</td><td>126.9</td></tr>
<tr><td>4</td><td>West Virginia</td><td>125.3</td></tr>
<tr><td>5</td><td>Virginia</td><td>117.9</td></tr>
<tr><td>6</td><td>Villanova</td><td>114.9</td></tr>
<tr><td>7</td><td>Oregon</td><td>112.1</td></tr>
<tr><td>8</td><td>Xavier</td><td>110.4</td></tr>
<tr><td>9</td><td>Purdue</td><td>109.3</td></tr>
<tr><td>10</td><td>Louisville</td><td>108.9</td></tr>
<tr><td>11</td><td>Arizona</td><td>106.1</td></tr>
<tr><td>12</td><td>Duke</td><td>105.4</td></tr>
<tr><td>13</td><td>Kentucky</td><td>105.2</td></tr>
<tr><td>14</td><td>SMU</td><td>104.0</td></tr>
<tr><td>15</td><td>Indiana</td><td>103.9</td></tr>
<tr><td>16</td><td>Oklahoma</td><td>103.7</td></tr>
<tr><td>17</td><td>Miami Florida</td><td>99.9</td></tr>
<tr><td>18</td><td>Maryland</td><td>97.9</td></tr>
<tr><td>19</td><td>Baylor</td><td>97.7</td></tr>
<tr><td>20</td><td>Wichita State</td><td>97.4</td></tr>
</tbody></table>
</center>
<br />
I'm not entirely sure what I think of this. The top of the rankings isn't too surprising, although I think most folks wouldn't have UNC ahead of Kansas and MSU. Oklahoma is much lower than the #2 seed they received. Wichita State is also a surprise at 20 -- although they seem to be handling Vanderbilt tonight so maybe there's something to that.<br />
<br />
And I guess you could conclude that it's a bad year for Louisville and SMU to be on probation -- they were both very solid this year.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com1tag:blogger.com,1999:blog-5902620336509647050.post-9833810712297700522016-03-14T09:19:00.004-04:002016-03-14T09:19:46.250-04:00Does Coaching Experience Matter?One of the things I investigated in the run-up to the Tournament this year was whether coaching experience matters. My approach was pretty simplistic -- I offered my prediction model information on how a team/coach had performed the previous year in the Tournament to see if that information had any predictive value. It didn't -- at least for my model.<br />
<br />
Over at <a href="http://harvardsportsanalysis.org/2016/03/for-coaches-previous-experience-in-march-madness-may-not-mean-much/">Harvard Sports Analysis Collective</a> (worth reading, by the way), Kurt Bullard takes a better look at the same question. He looks at how coaches perform relative to their seeding over their coaching lifetime. If experience matters, you'd expect coaches with more experience to do better. But that's not the case -- there's no correlation between how well a coach does and how much experience he has. (Alternatively, it could be that his experience is factored into the seed his team gets, although I'd argue that's probably not the case.)<br />
<br />
At any rate, you might want to be leery of analysts who say that "Michigan State is going to do well in the Tournament because Coach Izzo has more experience than anyone in the Tournament." Michigan State probably <b>is</b> going to do well -- but that's because the Committee mis-seeded them, not because of Coach Izzo's experience.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com1tag:blogger.com,1999:blog-5902620336509647050.post-32174374860703659352016-03-09T17:21:00.000-05:002016-03-09T17:21:52.542-05:00That's Not Really A NumberSuppose that you're competing in the <a href="https://www.kaggle.com/c/march-machine-learning-mania-2016">Kaggle competition</a> and you're using team win ratios and average scoring for the season to predict who is going to win a game. Your input to your model might look something like this:<br />
<br />
<center>
<table class="tableizer-table">
<thead>
<tr class="tableizer-firstrow"><th>Team</th><th>Win Ratio</th><th>Ave. Score</th><th>Team</th><th>Win Ratio</th><th>Ave. Score</th></tr>
</thead><tbody>
<tr><td>Michigan</td><td>0.75</td><td>86.5</td><td>UCLA</td><td>0.73</td><td>81</td></tr>
</tbody></table>
</center>
<br />
Your results are mediocre, so you decide to improve your model by adding more information about each team. The seeding of the team -- the NCAA Tournament committee's assessment of team strength -- seems like it would be useful for prediction, so you add each team's seeds to your inputs:<br />
<br />
<center>
<table class="tableizer-table">
<thead>
<tr class="tableizer-firstrow"><th>Team</th><th>Win Ratio</th><th>Ave. Score</th><th>Seed</th><th>Team</th><th>Win Ratio</th><th>Ave. Score</th><th>Seed</th></tr>
</thead><tbody>
<tr><td>Michigan</td><td>0.75</td><td>86.5</td><td>5</td><td>UCLA</td><td>0.73</td><td>81</td><td>14</td></tr>
</tbody></table>
</center>
<br />
You've just made a mistake. Do you see what it is?<br />
<br />
The way you've added the seeding information, many machine learning tools / models are going to treat the seed as a number<a href="https://draft.blogger.com/blogger.g?blogID=5902620336509647050#1" name="top1"><sup>1</sup></a>, not any different from the Win Ratio or the Average Score. And that's a problem, because the seed is not really a number. It's actually what statisticians would call a <a href="https://en.wikipedia.org/wiki/Categorical_variable">categorical variable</a>, because it can take one value out of a fixed set of arbitrary values. (Machine learning types might be more likely to call it a categorical feature.) If you're not convinced about that, imagine replacing each seed with a letter -- the #1 seed becomes the A seed, the #2 seed becomes the B seed and so on. This is perfectly reasonable -- we could still talk about the A seeds being the favorites, we'd know in the first round the A seeds play the P seeds and so on. It wouldn't make any sense to try to do that with a numeric feature like (for instance) the Average Score.<br />
<br />
Another difference between numeric and categorical features that merely look like numbers is that real numeric features have a fixed scale, while categorical features have an arbitrary scale. The difference between an Ave. Score of 86.5 and an Ave. Score of 81 is 5.5 points, and that's the same amount as the difference between an Ave. Score of 78.8 and 73.3. But the difference between a 16 seed and a 15 seed might be quite different than the difference between a 1 seed and a 2 seed. (Not to mention that the difference between a 16 seed and a 15 seed might be quite different than between the 16 seed and a different 15 seed!)<br />
<br />
So if I've convinced you that team seeds are not really numbers but just look like numbers, then how should your represent them in your model?<br />
<br />
The basic approach is something called "<a href="http://stackoverflow.com/questions/17469835/one-hot-encoding-for-machine-learning">one hot encoding</a>" (the name derives from a type of <a href="https://en.wikipedia.org/wiki/One-hot">digital circuit design</a>). The idea behind one hot encoding is to represent each possible value of the categorical feature as a different binary feature (with value 0 or 1). For any particular value of the categorical feature, one of these binary features will have the value 1 and the rest will be zero. (Hence "one hot".) To represent the seeds, we need 16 binary features:<br />
<br />
<table class="tableizer-table">
<thead>
<tr class="tableizer-firstrow"><th>Team</th><th>Win Ratio</th><th>Ave. Score</th><th>Seed_1</th><th>Seed_2</th><th>Seed_3</th><th>Seed_4</th><th>Seed_5</th><th>...</th><th>Seed_16</th><th>Team</th><th>Win Ratio</th><th>Ave. Score</th><th>Seed_0</th><th>...</th></tr>
</thead><tbody>
<tr><td>Michigan</td><td>0.75</td><td>86.5</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>...</td><td>0</td><td>UCLA</td><td>0.73</td><td>81</td><td>0</td><td>...</td></tr>
</tbody></table>
<br />
This representation gives your model much more flexibility to learn the differences between the seeds. For example, it could learn that being a 5 seed is every bit as valuable as being a 4 seed. If you represented the seed as a number, it would be impossible to learn that.<br />
<br />
One drawback about this representation is that it has a tendency to rapidly increase the number of input features. Seeds added 32 features to the model -- adding the Massey ordinals to your model could add many thousands of features! That can be bad for several reasons. First, the increased dimensionality will slow down the training of your model. This may or may not be a problem, depending upon the overall size of your training data and the computational power you have for training. A more significant problem is that it will encourage your model to overfit. Even if you use the Tournament data all the way back to 1985 (which I do not encourage) there may be only a few games for unlikely pairings such as (say) a #8 seed plays a #16 seed. That may cause your model to learn a rule for those games that is too specific.<br />
<br />
Now it may be that you disagree with the very premise of this posting. You really do think that seeds can be treated like numbers. The good news for you is that it is very easy to do that -- just include the seed as both a numeric feature and a categorical feature. You can look at the size of your model coefficients or other components to see whether treating the seed as a number has any value or not.<br />
<br />
<hr width="80%" />
<span style="font-size: small;">
<a href="https://draft.blogger.com/null" name="1"><b>1 </b></a>This is admittedly a simplification. Some machine learning models (such as tree-based models) may be flexible enough to treat numeric variables in ways that mimic categorical variables. But many models (and notably linear and logistic regressions) will not.<a href="https://draft.blogger.com/blogger.g?blogID=5902620336509647050#top1"><sup>↩</sup></a>
</span>
Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com1tag:blogger.com,1999:blog-5902620336509647050.post-54072307943818007952016-03-08T19:43:00.001-05:002016-03-08T19:43:23.523-05:00Machine Madness 2016This is just a quick post to announce the return for the 7th year of the Machine Madness contest, for machine predictors to compete in a traditional bracket-style March Madness competition. Details can be found on <a href="http://netprophetblog.blogspot.com/p/machine-madness-2015.html">this page</a>. Last year, Dr. Amanda Schierz (aka "Bluefool") won the competition and ESPN immediately (*) sent a film crew all the way to England to interview her. Don't miss out on your chance to be a media star!<br />
<br />
(*) If by immediately you mean a year later.<br />
<br />
Please come compete, and let me know if you have any questions!Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com5tag:blogger.com,1999:blog-5902620336509647050.post-41613790973449321242016-03-04T14:27:00.002-05:002016-03-04T14:27:42.982-05:00Scoring the Kaggle Contest In this <a href="http://netprophetblog.blogspot.com/">previous post</a>, I talked briefly about whether competitors in the <a href="https://www.kaggle.com/c/march-machine-learning-mania-2016">Kaggle March Madness</a> contest should "gamble" with their entries. The short answer is "yes" -- if your goal is to win money, then your best strategy is to gamble with at least some of your game predictions. (How many games you should gamble with is an interesting question for another time.) In my opinion, that's a sign that the contest is broken. Rather than testing who can make the best predictions about the NCAA Tournament, the contest is testing who can formulate the best meta-strategy for winning the contest. <br />
<br />
So, is it possible to fix the contest so that the results more accurately identify the best predictions?<br />
<br />
The <a href="https://www.kaggle.com/c/march-machine-learning-mania-2016/details/evaluation">log-loss scoring function</a> asks competitors to provide a confidence in their predictions, and then scores them based upon how confident their correct (or incorrect) prediction was. If you analyze this scoring approach, you find that the best strategy in the long run is for the competitors to set their confidence in each prediction to exactly their real confidence. And that's exactly what you want for a fair and accurate scoring system.<br />
<br />
The problem is that this contest is not a "long run". In fact, it's anything but a long run -- there are only 63 games being scored. That's a lot compared to predicting (say) just the Superbowl, but for a contest like this it's not nearly long enough to ensure that true predictions are the best strategy. <br />
<br />
So, how can we fix the scoring to better reward true predictions?<br />
<br />
The obvious fix of having the teams play a few thousand games is probably a non-starter. But it does point towards the necessary condition: We want the competitors to be making many choices instead of just 63. My suggestion is to have the competitors predict the Margin of Victory (MOV) for each game, and score them on how close they get to the actual MOVs. Now instead of making 63 binary predictions, the competitors are making 63 predictions with many more choices, and -- crucially -- they don't have control over how much they will win/lose on each prediction.<br />
<br />
It should be obvious that this makes it more difficult to "gamble" for an improved score. Consider last year, where Kentucky was viewed as an overwhelming favorite coming into the Tournament. Under the current scoring system there was an easy and obvious "gambling" strategy -- predict that Kentucky would win every game and set your confidence in each of those games very high. (And in fact, if Kentucky had won the championship game, a gambling strategy would probably have won the contest.) However, under the Margin of Victory scoring system, how would you "gamble" to improve your chances of winning the contest? It's hard to imagine any approach that would work better than submitting your actual best predictions.<br />
<br />
The Kaggle contest is a fun diversion and I think the results have provided some interesting insight into predicting college basketball games. But I think the contest would be improved by using a scoring system that more accurately identified the best predictor, and I'll continue my low-key lobbying efforts (*) for that change.<br />
<br />
(*) Which consist entirely of posting something like this every year :-)Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com5tag:blogger.com,1999:blog-5902620336509647050.post-22872273445445153442016-03-03T13:06:00.000-05:002016-03-03T13:06:03.741-05:00To Gamble or Not To Gamble, That is the QuestionOr at least that's "a" question -- one that comes up yearly in the Kaggle competition. <a href="https://www.kaggle.com/c/march-machine-learning-mania-2016/forums/t/19298/clipping-over-optimistic-predictions">Here's a version of it</a> that popped up this year. <br />
<br />
The Kaggle competition (for those who aren't aware) uses <a href="https://www.kaggle.com/c/march-machine-learning-mania-2016/details/evaluation">log-loss scoring</a>. Competitors predict which team will win as a confidence level (e.g., 95% certain of a win by Kentucky) and then are rewarded/punished accordingly. And since the scoring is logarithmic, you are punished a lot if you make a very confident wrong decision.<br />
<br />
The question that plagues competitors is whether forcing their predictions to be more conservative or less conservative will improve their chances of winning the contest. (Or at least finishing in the top five and receiving a cash prize.) Note that this is only concerned with winning the contest, not with improving the predictions. Presumably your predictions are already as accurate as you can make them, and artificially changing them would make them worse -- in the long run. But the Kaggle contest isn't concerned with the long run -- it's only concerned with how you perform during this particular March Madness.<br />
<br />
<br />
As a thought experiment, let's assume that you could change your entry right before the final game. You can see the current standings, but not any of the other entries. Would you change your entry? And if so, how?<br />
<br />
Well, if you see that you're in first place with a big lead, you might not change it at all. Or maybe you'd make your pick more conservative so that you could be sure you wouldn't lose much if your pick was wrong. But if you didn't have a big lead (and in general the farther away from first place you were) you'd probably want to gamble on getting that last game correct. At that point "average" performance cannot be expected to move you ahead of the team's ahead of you, and even "good" performance might be passed by someone behind you who was willing to gamble more than you.<br />
<br />
Since it's much more likely that you will be losing the contest going into the final game than in first place with a big lead, I think this argues that (if your goal is to maximize your expected profit) you should "gamble" on at least the last game. It's left to the reader to apply this reasoning recursively to games before the final game :-).<br />
<br />
As a concrete example of this, last year Juho Kokkala submitted entries based upon "Steal This Entry" but with Kentucky's probabilities turned up to 1.0. The non-gambling "Steal This Entry" finished in 42nd place, but if Kentucky had won out, Juho would have probably placed in the top two and collected some prize money. Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com4tag:blogger.com,1999:blog-5902620336509647050.post-16202884815858758782016-02-19T13:14:00.000-05:002016-02-19T13:14:13.491-05:00More Kaggle News, ESPN Irritates MeAs a follow-up to this <a href="http://netprophetblog.blogspot.com/2016/02/kaggle-competition-is-back-for-2016.html">previous post</a>, the Kaggle competition is <a href="https://www.kaggle.com/c/march-machine-learning-mania-2016">officially back</a>. A good deal of data is available, and the forums have been moderately active. The new Kaggle Notebooks feature is getting some exercise, too: there are 116 scripts for this competition at the moment, although I'm unclear on what they all are. There are at least a couple of scripts to calculate ELO ratings and similar things. Might be worth a look if you're just getting started in this area.<br />
<br />
Prizes this year are considerable -- $20K split 10/6/4/3/2. I suggested awarding prizes for the best performance on each round of the Tournament, but that might have been too hard to implement quickly. At any rate, spreading the prizes down to 5th place is a good improvement. The contest is basically random amongst about the top 100 or so contestants, so weighting all the money at the top makes it even more of "random number lottery."<br />
<br />
On a completely unrelated note, the NetProphet predictor broke on me last night. It turned out that ESPN has changed the format of its box scores. You can see the new format <a href="http://espn.go.com/mens-college-basketball/boxscore?gameId=400839358">here</a>. The change seems to have also broken all the past seasons. If you go to (say) November 2014 the scoreboard and schedule pages will claim that no games were played.<br />
<br />
ESPN has been modifying their page formats for a while now, and I was expecting a change at some point. The scoreboard page had earlier been modified to run from JSON data embedded in the page, and I was expecting to see something similar happen with the box scores and other game pages. But interestingly enough, although the page formats have changed, they haven't gone to using embedded JSON data on these pages. That's too bad, because pulling the JSON data out of the page, parsing it and then using it is more straightforward -- and probably a lot more robust -- than pulling data out of the HTML.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com0tag:blogger.com,1999:blog-5902620336509647050.post-47285294311573469982016-02-06T15:04:00.000-05:002016-02-06T15:04:09.852-05:00Kaggle Competition is Back for 2016I've been remiss about posting to the blog, but I thought I'd share that a little birdie hinted to me that the Kaggle Competition will be back again this year, with perhaps some new twists. So keep your predictors warmed up.<br />
<br />
I'm undecided whether I'm going to provide "Steal My Entry" again this year, but I might be interested in a private collaborative effort. In particular my thought is to merge an entry from my predictor -- which mostly focuses on regular-season games -- with a predictor that has specifically been trained on tournament games. I'll provide my model's game predictions for all the tournament games back to 2009, and then you train a tournament-specific model using my predictions along with any other information you think is valuable (e.g., team seedings, locations, etc.). Contact me if that sounds interesting -- and this isn't an exclusive offer, I'm happy to collaborate with multiple folks either individually or as part of a larger group.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com0tag:blogger.com,1999:blog-5902620336509647050.post-56697909107352110512015-12-09T20:24:00.001-05:002015-12-09T20:24:14.040-05:00Sports Information APITonight I stumbled across <a href="http://developer.sportradar.us/docs/NCAA_Mens_Basketball">Sportradar.us</a>, which seems to be the former SportsData. Interestingly, they have APIs to deliver all sorts of sports information, including comprehensive NCAA men's basketball coverage -- including play-by-play data and even location data, i.e., where on the court a shot was taken (!). <br />
<br />
The bad news is that the lowest pricing tier is $500/month. So not something I'll be buying for Christmas. But interesting.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com3tag:blogger.com,1999:blog-5902620336509647050.post-29160517092800943172015-12-09T17:00:00.000-05:002015-12-09T17:00:03.832-05:00Working Overtime<a href="http://bookweb.syr.edu/webitemimages/1/W49275.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="http://bookweb.syr.edu/webitemimages/1/W49275.jpg" height="320" title="" width="241" /></a>Overtime is one of the interesting quirks of basketball. In some sports -- particularly low-scoring sports like soccer and hockey -- a game may end in a tie. But in college basketball teams play additional periods -- as many as needed -- until a winner is determined.<br />
<br />
Overtime games skew team statistics. ESPN and other sites typically have <a href="http://espn.go.com/mens-college-basketball/statistics">pages of statistics</a> such as "Points Per Game". But if one game is 40 minutes long and another is <a href="http://espn.go.com/mens-college-basketball/boxscore?gameId=290710041">226 minutes long</a>, it's not really an apples-to-apples comparison. This is one reason analysts are fond of "per possession" statistics -- not only does it correct for pace of play, but it also corrects for overtime games.<br />
<br />
Clearly the statistics you feed into a predictor need to be corrected somehow for overtime games. But there's another interesting overtime issue to consider: What's the final score of an overtime game?<br />
<br />
One choice is to use the score at the end of the overtime(s). The other is to treat the game as a tie. There are intuitive arguments in favor of both choices. The fact that Syracuse beat Connecticut suggests that Syracuse is a better team, regardless of how many minutes that took, so we should treat the game as a win for Syracuse. On the other hand, the teams were deadlocked for six overtimes, which suggests that they're about as equal as it is possible to be, regardless of whether one team or the other managed to win the game in the wee hours of the morning.<br />
<br />
Or maybe the game should be treated as a tie for some statistics and not for others.<br />
<br />
As longtime readers of this blog are aware, I'm a believer in doing whatever works best. So in this case, I made two runs of my predictor, once treating overtime games as ties and once using the actual final scores. In my case, the predictor performed better treating overtime games as ties.<br />
<br />
Another possibility is to treat the final score of an overtime game as 1 or -1 (or 0.1 and -0.1 if your predictor can handle that), depending upon which team wins the overtime period(s). This retains the won/loss information, but otherwise treats the game as (nearly) a tie.<br />
<br />
For those of you who also have predictors, I encourage you to try the same experiment and report back which choice (if either) works better for you.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com4tag:blogger.com,1999:blog-5902620336509647050.post-18464627874284460602015-12-06T18:48:00.000-05:002015-12-06T18:48:12.679-05:00A Few Funny ThingsWhen I logged in to work on this post, I noticed that my blog had 100,000 page views. Since I have an audience of like six people, you guys must be checking my pages a lot. Good job! Anyway, I've been spending some time lately getting my data scraping working, and that always involves a few trips through the bowels of data validation.<br />
<br />
First stop is <a href="http://espn.go.com/ncb/boxscore?gameId=400844995">this game</a>. I was running the predictor when it warned me about an unusual event: a conference game in early November. Unusual, but it happens (often a <a href="http://www.philadelphiabig5.org/">Big5</a> game). What was more surprising was that it was a team playing itself. According to the predictor, UNC Greensboro had come up with the clever notion of scheduling a home game against itself. Or maybe it was on the road. <br />
<br />
One of the challenges of predicting NCAA basketball is that every data source uses different names for teams. To try to match them up I have lists of alternate team names:<br />
<br />
<blockquote class="tr_bq">
St. Francis (NY)<br />
1383<br />
St. Francis BRK<br />
St. Francis (N.Y.)<br />
St. Francis-NY<br />
St Francis NY<br />
St Francis(NY)<br />
St Francis (NY)<br />
St. Francis Brooklyn<br />
St. Francis NY<br />
St. Francis-NY Terriers<br />
St Francis (BKN)<br />
st.-francis-(NY)-terriers<br />
St. Francis (BKN) </blockquote>
(That weird-looking "1383" is the name for St. Francis (NY) in the <a href="https://www.kaggle.com/c/march-machine-learning-mania-2015">Kaggle</a> contest. Because it's run by data scientists, so why use a human-readable name when you can use an arbitrary and completely useless number?)<br />
<br />
In this case the predictor too aggressively (although reasonably) determined that Div III Greensboro College was a nickname for UNC Greensboro. (By the way, my list of nicknames and the Python code that goes with it is available for the asking. But you're on your own dealing with Greensboro vs. Greensboro.)<br />
<br />
Next up is <a href="http://espn.go.com/mens-college-basketball/boxscore?gameId=400499857">this game</a>. Looks like a perfectly reasonable WAC Conference game. Problem is, one of those teams was not in the WAC. Actually, one of those teams didn't even exist.<br />
<br />
You see, last year the University of Texas decided to merge two campuses -- the University of Texas Brownville and the University of Texas Pan American -- to form a brand new campus the University of Texas Rio Grande Valley. Brownsville didn't have sports, but UT-PA was a Division I team in the WAC, so the new campus stayed in the WAC and became the "Vaqueros."<br />
<br />
(Trivia Question: Name the other four NCAA Division I basketball nicknames that are Spanish words.)<br />
<br />
Well, ESPN decided the easiest way to deal with this whole business was to just go into their database and replace every instance of "University of Texas Pan American Broncs" with "UTRGV Vaqueros." Hence the mysterious 2013 game involving a university that wouldn't exist for several more years.<br />
<br />Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com1tag:blogger.com,1999:blog-5902620336509647050.post-28566337028175364512015-11-18T15:18:00.002-05:002015-11-18T15:18:48.646-05:00Why I Hate RainbowsThe <a href="http://espn.go.com/mens-college-basketball/team/_/id/62/hawaii-rainbow-warriors">University of Hawaii Rainbow Warriors</a> play their home games at a five hour offset from the East Coast. I don't begrudge them living in Paradise, but the peculiarity of the time zones means that ESPN often reports their games as happening the day after they were actually scheduled.<br />
<br />
This annoyance I don't need.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com2tag:blogger.com,1999:blog-5902620336509647050.post-17491852175774126242015-11-17T10:58:00.002-05:002015-11-17T10:58:34.409-05:00Kind of Amazing<iframe allowfullscreen="" frameborder="0" height="400" src="http://stats.nba.com/movement/#!/?GameID=0041400235&GameEventID=308" width="700"></iframe>
<br />
That's an animation of NBA movement data, which apparently you can get via a free API. Savvas Tjortjoglou goes into detail <a href="http://savvastjortjoglou.com/nba-play-by-play-movements.html">here</a>. Who knows what sort of predictive model you could build exploiting this data. Thankfully the NCAA doesn't have anything of the sort, or I'd have to quit my day job.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com4tag:blogger.com,1999:blog-5902620336509647050.post-73692013962146220092015-11-14T21:34:00.001-05:002015-11-14T21:34:38.203-05:00Really, ESPN?With the first day's games done I fired up my ESPN score scraper to gather up the data and get the season started.<br />
<br />
It crashed.<br />
<br />
It seems like ESPN chose the first day of the season to roll out a new format (and URL scheme) for their basketball scoreboard page.<br />
<br />
To be fair, I wrote my current (Python Scrapy-based) scraper with the help of Brandon Harris (*) and he warned me when I started down this road that ESPN was busy mucking up all their scoreboard pages. "Oh no," said I, "it looks fine, I'm sure they won't change it at the last second." So I have no one to blame but myself.<br />
<br />
(*) And by help I mean he basically gave me working code.<br />
<br />
ESPN went all Web 3.0 in their page redesign, which means that rather than send a web page, they send a bunch of Javascript and raw data and make your web browser build the page. (Which probably saves them millions of dollars a year in server costs, so who can blame them?) This breaks the whole scraper paradigm, which is to Xpath through the HTML to find the bits you need. There's no HTML left to parse. The good news in this case is that ESPN was nice enough to include the entire URL I need in the data portion of the new format, so it is very easy to do a regular expression search and pull out the good parts. Otherwise you get into some kludgy solutions like using a headless web browser to execute the Javascript and build the actual HTML page. Or trying to find the mobile version of the page and hope that's more parseable.<br />
<br />
I don't do anything much with the model until after a few weeks of games, so I have some time to fix my code. And I suppose that if you want to scrape data from the Web, you'd better be prepared to deal with change.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com7tag:blogger.com,1999:blog-5902620336509647050.post-52561913140420215412015-11-13T21:53:00.001-05:002015-11-13T21:59:55.536-05:00Improving Early Season Performance(I hope you're enjoying the first evening of college basketball!) <br />
<br />
In my <a href="http://netprophetblog.blogspot.com/2015/11/performance-by-week-of-season.html">previous posting</a>, I looked at the performance of my predictor on a week-by-week basis throughout the season. This showed that performance was poor at the beginning of the season (when our knowledge about teams is most uncertain) and improved throughout the season (at least until play shifts to tournaments). Is there any way we can use this insight to improve performance?<br />
<br />
One approach is to let the model try to correct for the week to week differences. To do this, the model needs to know the week of the season for each game. But it isn't sufficient to simply have a feature with the week value (e.g., WEEK_OF_SEASON=22) because most models (and linear regression in particular) will treat that feature as a continuous value and be unable to apply a specific correction for a specific week. The solution to this is to use "<a href="https://en.wikipedia.org/wiki/One-hot">one hot encoding</a>".<br />
<br />
One hot encoding is applicable to any sort of categorical feature -- a feature with distinct values that represent different categories, such as week in the season, day of the week, etc. One hot encoding splits the categorical feature up into a number of new features, one for each possible value of the categorical feature. For example, DAY_OF_THE_WEEK would get split into 7 new features. For each example in our data set, the appropriate feature is set to 1 while all the other features are set to 0. For a game that took place on Tuesday, the new feature DAY_OF_THE_WEEK_2 would get set to 1 (assuming Sunday is Day Zero), and DAY_OF_THE_WEEK_0, DAY_OF_THE_WEEK_1, etc. would get set to 0.<br />
<br />
Once we've hot encoded the WEEK_OF_SEASON the model can correct on a week-by-week basis. However, whatever correction the model applies will apply to every game that week, so this approach is only suitable to correct any overall bias for that week. If the error in the week is completely random, this approach won't help.<br />
<br />
So is there a weekly bias (at least in my model)?<br />
<br />
The following chart shows the Absolute Error for my model each week both before (red) and after (blue) applying this technique.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiy0CZurMg1vq5U5paYpqgvcMUYhU1-HtSNLPRTLfc_DQrrUvjlRnAtvzhRif44QqhYurW2J6ev9SWkyw1YzYUadU9DxsaV6sLro27a_Ep8z2lKuxUN7r383HHvbsjwBK2hL4fz7g_NNhox/s1600/Image4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="388" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiy0CZurMg1vq5U5paYpqgvcMUYhU1-HtSNLPRTLfc_DQrrUvjlRnAtvzhRif44QqhYurW2J6ev9SWkyw1YzYUadU9DxsaV6sLro27a_Ep8z2lKuxUN7r383HHvbsjwBK2hL4fz7g_NNhox/s640/Image4.png" title="" width="640" /></a></div>
<br />
As you can see, in many weeks before applying the correction there's significant bias. Afterwards the bias is reduced to near zero. That's not surprising -- essentially what a linear regressor will do in this situation is set the new feature to be worth the negative of the mean bias, thus it will exactly cancel the bias. <br />
<br />
Adding the hot-encoded WEEK_OF_SEASON eliminates weekly bias, but it doesn't eliminate more complex errors. A potentially better approach to reducing early season error is to add more reliable information about the strength of teams in the early season. But until teams play some games, how can we know how good they are? An obvious approach is to guess that they're about as good as they were the previous season. This isn't a perfect proxy -- after all, teams do get better or worse from season to season -- but there is a strong correlation between seasons, so it's generally a pretty good guess.<br />
<br />
But there's a problem with just throwing data from the previous season into our model. We really only want the model to use the old data until the new data is better than the old data. It's fairly straightforward to figure out when that happens -- we run the model once on the old data and once on the new data, and look for where the new data starts to outperform the old data. But what's not easy is to stop using the old data in the model. You can't change the number of features in your training data halfway through building a model!<br />
<br />
There are several ways you might address this problem. You could have two models, one that uses the old data up to the crossover point, and another only uses the new data after that point. You could use a weighted moving average, and start the year with the old data and gradually replace it with the new data. Or you could have both the old data and the new data in the model, but replace the old data with the new data once you hit the crossing point.<br />
<br />
I've tried all of these approaches. Having two models is very cumbersome and creates a lot of workflow problems. The second is the most intellectually appealing, but I've never been able to get it to perform well. The third approach is simple and flexible, but has the drawback that after the crossing point the new data is in the model twice. Despite that drawback, this approach has worked the best for me.<br />
<br />
The plot below shows the mean squared error for the model both before (red) and after (blue) adding in data from the previous season.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWWg13kcpCF-LL4_RTQp8Xp-0FP-0auE9Xu5ama1foIF6B-MbwLMXj_IIglyLzHR9RxpCidlGmiqSgpfVBaJ2C1aprpYKgTq4jfgguM4ulhoMMpPVp8rZlrhvN-lbc38pE0L943aMeD8Ip/s1600/Image2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="394" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWWg13kcpCF-LL4_RTQp8Xp-0FP-0auE9Xu5ama1foIF6B-MbwLMXj_IIglyLzHR9RxpCidlGmiqSgpfVBaJ2C1aprpYKgTq4jfgguM4ulhoMMpPVp8rZlrhvN-lbc38pE0L943aMeD8Ip/s640/Image2.png" title="" width="640" /></a></div>
<br />
As you would expect, this shows the most improvement early in the season -- quite dramatically in the first few weeks -- and tapers off after that. In this case, the cutoff is in the 12th week of the season. After that the impact of the old data is eliminated.<br />
<br />
It's also interesting to look at how the old data impacts performance against the spread:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzqJ_omVRsD79Txj0GcvNqeh5_SbH_BnKfOv5qqfm5WzARQz7IjrtaAgZ8vyXj6oQwshUV8vJQXFU-86W4vkubZuLyYXWFW22b5KJfnlu6UOcZ_5vn2va03YoVzMJYVlwhybRWTEmd62GU/s1600/Image1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="408" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzqJ_omVRsD79Txj0GcvNqeh5_SbH_BnKfOv5qqfm5WzARQz7IjrtaAgZ8vyXj6oQwshUV8vJQXFU-86W4vkubZuLyYXWFW22b5KJfnlu6UOcZ_5vn2va03YoVzMJYVlwhybRWTEmd62GU/s640/Image1.png" title="" width="640" /></a></div>
<br />
(In this graph, bigger is better.) In this case, the addition of the previous season's data helps our performance against the spread through about the first ten weeks (and nearly eliminates the anomalous performance in Week 5). Interestingly enough, this actually hurts performance slightly after Week 16.<br />
<br />
Overall, accounting for week-by-week bias and using the previous season's data to improve early season predictions is an effective approach. It should be noted, though, that the overall improvement from these changes are modest: about 0.10 point in RMSE and less than 1% in WATS.<br />
<br />
An interesting line of speculation is whether it is possible to easily improve the value of the previous season's data. For example, it's reasonable to expect that from year-to-year teams will tend to "regress to the mean". If that's true, regressing the previous year's data towards the mean might further improve performance.<br />
<br />
<br />Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com0tag:blogger.com,1999:blog-5902620336509647050.post-21084343649407836062015-11-06T15:20:00.002-05:002015-11-06T15:20:23.513-05:00Performance By Week of SeasonPython has some nice built-in plotting capabilities, so with a little work I was able to analyze the performance of my predictor by week of the season. This is what that looks like: <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFcgUmHxeP9u4lTF_GqdntLhh0WIos3agbGvPyuBTAztDPsB82abY2tz-9DiF7LQ8wyKsD3an99wZJs14y1sMRyaFidoevKMZEC0v7q50u6iFeUeZTEU9azhsDZao_k1Ffyml0YWsPYVCA/s1600/figure_1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="432" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhFcgUmHxeP9u4lTF_GqdntLhh0WIos3agbGvPyuBTAztDPsB82abY2tz-9DiF7LQ8wyKsD3an99wZJs14y1sMRyaFidoevKMZEC0v7q50u6iFeUeZTEU9azhsDZao_k1Ffyml0YWsPYVCA/s640/figure_1.png" title="" width="640" /></a></div>
The red dotted line is the predictor's average long-term performance. As you might expect, performance starts off very poorly but improves quite rapidly during the early part of the season. It gets to average performance after about 4 weeks, but I typically start predicting after 2 weeks (800 games), when the error is still around 12 RMSE. The more interesting observation is that prediction accuracy gets steadily worse at the end of the season. Most of the games in those last three weeks of the season are tournament games: the various conference tournaments followed by the NCAA Tournament. I assume my predictor is just worse at tournament games.<br />
<br />
It's also interesting to look at a different measure of performance: Wins Against the Spread (ATS). This is the percentage of games where the predictor would have placed a winning bet against the Las Vegas closing line. <br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCEE3jT62DKxoMYcq-1RnrGBxu3mrCAmcGQaKIlr9NqyIxBSExulRQwNjsHXrgBdlAGKgHA_XyWK-VmowW5ZVAVH79QL7JgdS5fcIGFsBIT_un21DYLxkRkKPpPWryKF64H0A1MGx_raIi/s1600/figure_2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;" title=""><img border="0" height="466" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCEE3jT62DKxoMYcq-1RnrGBxu3mrCAmcGQaKIlr9NqyIxBSExulRQwNjsHXrgBdlAGKgHA_XyWK-VmowW5ZVAVH79QL7JgdS5fcIGFsBIT_un21DYLxkRkKPpPWryKF64H0A1MGx_raIi/s640/figure_2.png" title="" width="640" /></a></div>
(Again, the red dotted line shows the average performance.) This shows a different shape than the RMSE plot. In the late part of the season, the predictor's performance ATS gets very good -- even though its RMSE performance is getting worse. There's a similar although less obvious trend in the early part of the season, too. (I'm not sure what's going on in Week 6.) Apparently my predictor performs poorly in the late season -- but the Vegas lines perform even worse.Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com0tag:blogger.com,1999:blog-5902620336509647050.post-90864006329153092262015-10-02T19:53:00.000-04:002015-10-02T19:53:00.853-04:00Calculating the Massey Rating, Part 3<script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>
<style type="text/css">
/*!
*
* Twitter Bootstrap
*
*//*! normalize.css v3.0.2 | MIT License | git.io/normalize */html{font-family:sans-serif;-ms-text-size-adjust:100%;-webkit-text-size-adjust:100%;font-size:10px;-webkit-tap-highlight-color:transparent}article,aside,details,figcaption,figure,footer,header,hgroup,main,menu,nav,section,summary{display:block}audio,canvas,progress,video{display:inline-block;vertical-align:baseline}audio:not([controls]){display:none;height:0}[hidden],template{display:none}a{background-color:transparent}a:active,a:hover{outline:0}abbr[title]{border-bottom:1px dotted}b,optgroup,strong{font-weight:700}dfn{font-style:italic}h1{font-size:2em;margin:.67em 0}mark{background:#ff0;color:#000}small{font-size:80%}sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline}sup{top:-.5em}sub{bottom:-.25em}img{border:0;vertical-align:middle}svg:not(:root){overflow:hidden}hr{-moz-box-sizing:content-box;box-sizing:content-box;height:0}pre,textarea{overflow:auto}code,kbd,pre,samp{font-size:1em}button,input,optgroup,select,textarea{color:inherit;font:inherit;margin:0}button{overflow:visible}button,select{text-transform:none}button,html input[type=button],input[type=reset],input[type=submit]{-webkit-appearance:button;cursor:pointer}button[disabled],html input[disabled]{cursor:default}button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0}input{line-height:normal}input[type=checkbox],input[type=radio]{box-sizing:border-box;padding:0}input[type=number]::-webkit-inner-spin-button,input[type=number]::-webkit-outer-spin-button{height:auto}input[type=search]::-webkit-search-cancel-button,input[type=search]::-webkit-search-decoration{-webkit-appearance:none}table{border-collapse:collapse;border-spacing:0}td,th{padding:0}/*! Source: https://github.com/h5bp/html5-boilerplate/blob/master/src/css/main.css */@media print{*,:after,:before{background:0 0!important;color:#000!important;box-shadow:none!important;text-shadow:none!important}a,a:visited{text-decoration:underline}a[href]:after{content:" (" attr(href)")"}abbr[title]:after{content:" (" attr(title)")"}a[href^="javascript:"]:after,a[href^="#"]:after{content:""}blockquote,pre{border:1px solid #999;page-break-inside:avoid}thead{display:table-header-group}img,tr{page-break-inside:avoid}img{max-width:100%!important}h2,h3,p{orphans:3;widows:3}h2,h3{page-break-after:avoid}select{background:#fff!important}.navbar{display:none}.btn>.caret,.dropup>.btn>.caret{border-top-color:#000!important}.label{border:1px solid #000}.table{border-collapse:collapse!important}.table td,.table th{background-color:#fff!important}.table-bordered td,.table-bordered th{border:1px solid #ddd!important}}@font-face{font-family:'Glyphicons Halflings';src:url(../components/bootstrap/fonts/glyphicons-halflings-regular.eot);src:url(../components/bootstrap/fonts/glyphicons-halflings-regular.eot?#iefix)format('embedded-opentype'),url(../components/bootstrap/fonts/glyphicons-halflings-regular.woff2)format('woff2'),url(../components/bootstrap/fonts/glyphicons-halflings-regular.woff)format('woff'),url(../components/bootstrap/fonts/glyphicons-halflings-regular.ttf)format('truetype'),url(../components/bootstrap/fonts/glyphicons-halflings-regular.svg#glyphicons_halflingsregular)format('svg')}.glyphicon{position:relative;top:1px;display:inline-block;font-family:'Glyphicons Halflings';font-style:normal;font-weight:400;line-height:1;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}.glyphicon-asterisk:before{content:"\2a"}.glyphicon-plus:before{content:"\2b"}.glyphicon-eur:before,.glyphicon-euro:before{content:"\20ac"}.glyphicon-minus:before{content:"\2212"}.glyphicon-cloud:before{content:"\2601"}.glyphicon-envelope:before{content:"\2709"}.glyphicon-pencil:before{content:"\270f"}.glyphicon-glass:before{content:"\e001"}.glyphicon-music:before{content:"\e002"}.glyphicon-search:before{content:"\e003"}.glyphicon-heart:before{content:"\e005"}.glyphicon-star:before{content:"\e006"}.glyphicon-star-empty:before{content:"\e007"}.glyphicon-user:before{content:"\e008"}.glyphicon-film:before{content:"\e009"}.glyphicon-th-large:before{content:"\e010"}.glyphicon-th:before{content:"\e011"}.glyphicon-th-list:before{content:"\e012"}.glyphicon-ok:before{content:"\e013"}.glyphicon-remove:before{content:"\e014"}.glyphicon-zoom-in:before{content:"\e015"}.glyphicon-zoom-out:before{content:"\e016"}.glyphicon-off:before{content:"\e017"}.glyphicon-signal:before{content:"\e018"}.glyphicon-cog:before{content:"\e019"}.glyphicon-trash:before{content:"\e020"}.glyphicon-home:before{content:"\e021"}.glyphicon-file:before{content:"\e022"}.glyphicon-time:before{content:"\e023"}.glyphicon-road:before{content:"\e024"}.glyphicon-download-alt:before{content:"\e025"}.glyphicon-download:before{content:"\e026"}.glyphicon-upload:before{content:"\e027"}.glyphicon-inbox:before{content:"\e028"}.glyphicon-play-circle:before{content:"\e029"}.glyphicon-repeat:before{content:"\e030"}.glyphicon-refresh:before{content:"\e031"}.glyphicon-list-alt:before{content:"\e032"}.glyphicon-lock:before{content:"\e033"}.glyphicon-flag:before{content:"\e034"}.glyphicon-headphones:before{content:"\e035"}.glyphicon-volume-off:before{content:"\e036"}.glyphicon-volume-down:before{content:"\e037"}.glyphicon-volume-up:before{content:"\e038"}.glyphicon-qrcode:before{content:"\e039"}.glyphicon-barcode:before{content:"\e040"}.glyphicon-tag:before{content:"\e041"}.glyphicon-tags:before{content:"\e042"}.glyphicon-book:before{content:"\e043"}.glyphicon-bookmark:before{content:"\e044"}.glyphicon-print:before{content:"\e045"}.glyphicon-camera:before{content:"\e046"}.glyphicon-font:before{content:"\e047"}.glyphicon-bold:before{content:"\e048"}.glyphicon-italic:before{content:"\e049"}.glyphicon-text-height:before{content:"\e050"}.glyphicon-text-width:before{content:"\e051"}.glyphicon-align-left:before{content:"\e052"}.glyphicon-align-center:before{content:"\e053"}.glyphicon-align-right:before{content:"\e054"}.glyphicon-align-justify:before{content:"\e055"}.glyphicon-list:before{content:"\e056"}.glyphicon-indent-left:before{content:"\e057"}.glyphicon-indent-right:before{content:"\e058"}.glyphicon-facetime-video:before{content:"\e059"}.glyphicon-picture:before{content:"\e060"}.glyphicon-map-marker:before{content:"\e062"}.glyphicon-adjust:before{content:"\e063"}.glyphicon-tint:before{content:"\e064"}.glyphicon-edit:before{content:"\e065"}.glyphicon-share:before{content:"\e066"}.glyphicon-check:before{content:"\e067"}.glyphicon-move:before{content:"\e068"}.glyphicon-step-backward:before{content:"\e069"}.glyphicon-fast-backward:before{content:"\e070"}.glyphicon-backward:before{content:"\e071"}.glyphicon-play:before{content:"\e072"}.glyphicon-pause:before{content:"\e073"}.glyphicon-stop:before{content:"\e074"}.glyphicon-forward:before{content:"\e075"}.glyphicon-fast-forward:before{content:"\e076"}.glyphicon-step-forward:before{content:"\e077"}.glyphicon-eject:before{content:"\e078"}.glyphicon-chevron-left:before{content:"\e079"}.glyphicon-chevron-right:before{content:"\e080"}.glyphicon-plus-sign:before{content:"\e081"}.glyphicon-minus-sign:before{content:"\e082"}.glyphicon-remove-sign:before{content:"\e083"}.glyphicon-ok-sign:before{content:"\e084"}.glyphicon-question-sign:before{content:"\e085"}.glyphicon-info-sign:before{content:"\e086"}.glyphicon-screenshot:before{content:"\e087"}.glyphicon-remove-circle:before{content:"\e088"}.glyphicon-ok-circle:before{content:"\e089"}.glyphicon-ban-circle:before{content:"\e090"}.glyphicon-arrow-left:before{content:"\e091"}.glyphicon-arrow-right:before{content:"\e092"}.glyphicon-arrow-up:before{content:"\e093"}.glyphicon-arrow-down:before{content:"\e094"}.glyphicon-share-alt:before{content:"\e095"}.glyphicon-resize-full:before{content:"\e096"}.glyphicon-resize-small:before{content:"\e097"}.glyphicon-exclamation-sign:before{content:"\e101"}.glyphicon-gift:before{content:"\e102"}.glyphicon-leaf:before{content:"\e103"}.glyphicon-fire:before{content:"\e104"}.glyphicon-eye-open:before{content:"\e105"}.glyphicon-eye-close:before{content:"\e106"}.glyphicon-warning-sign:before{content:"\e107"}.glyphicon-plane:before{content:"\e108"}.glyphicon-calendar:before{content:"\e109"}.glyphicon-random:before{content:"\e110"}.glyphicon-comment:before{content:"\e111"}.glyphicon-magnet:before{content:"\e112"}.glyphicon-chevron-up:before{content:"\e113"}.glyphicon-chevron-down:before{content:"\e114"}.glyphicon-retweet:before{content:"\e115"}.glyphicon-shopping-cart:before{content:"\e116"}.glyphicon-folder-close:before{content:"\e117"}.glyphicon-folder-open:before{content:"\e118"}.glyphicon-resize-vertical:before{content:"\e119"}.glyphicon-resize-horizontal:before{content:"\e120"}.glyphicon-hdd:before{content:"\e121"}.glyphicon-bullhorn:before{content:"\e122"}.glyphicon-bell:before{content:"\e123"}.glyphicon-certificate:before{content:"\e124"}.glyphicon-thumbs-up:before{content:"\e125"}.glyphicon-thumbs-down:before{content:"\e126"}.glyphicon-hand-right:before{content:"\e127"}.glyphicon-hand-left:before{content:"\e128"}.glyphicon-hand-up:before{content:"\e129"}.glyphicon-hand-down:before{content:"\e130"}.glyphicon-circle-arrow-right:before{content:"\e131"}.glyphicon-circle-arrow-left:before{content:"\e132"}.glyphicon-circle-arrow-up:before{content:"\e133"}.glyphicon-circle-arrow-down:before{content:"\e134"}.glyphicon-globe:before{content:"\e135"}.glyphicon-wrench:before{content:"\e136"}.glyphicon-tasks:before{content:"\e137"}.glyphicon-filter:before{content:"\e138"}.glyphicon-briefcase:before{content:"\e139"}.glyphicon-fullscreen:before{content:"\e140"}.glyphicon-dashboard:before{content:"\e141"}.glyphicon-paperclip:before{content:"\e142"}.glyphicon-heart-empty:before{content:"\e143"}.glyphicon-link:before{content:"\e144"}.glyphicon-phone:before{content:"\e145"}.glyphicon-pushpin:before{content:"\e146"}.glyphicon-usd:before{content:"\e148"}.glyphicon-gbp:before{content:"\e149"}.glyphicon-sort:before{content:"\e150"}.glyphicon-sort-by-alphabet:before{content:"\e151"}.glyphicon-sort-by-alphabet-alt:before{content:"\e152"}.glyphicon-sort-by-order:before{content:"\e153"}.glyphicon-sort-by-order-alt:before{content:"\e154"}.glyphicon-sort-by-attributes:before{content:"\e155"}.glyphicon-sort-by-attributes-alt:before{content:"\e156"}.glyphicon-unchecked:before{content:"\e157"}.glyphicon-expand:before{content:"\e158"}.glyphicon-collapse-down:before{content:"\e159"}.glyphicon-collapse-up:before{content:"\e160"}.glyphicon-log-in:before{content:"\e161"}.glyphicon-flash:before{content:"\e162"}.glyphicon-log-out:before{content:"\e163"}.glyphicon-new-window:before{content:"\e164"}.glyphicon-record:before{content:"\e165"}.glyphicon-save:before{content:"\e166"}.glyphicon-open:before{content:"\e167"}.glyphicon-saved:before{content:"\e168"}.glyphicon-import:before{content:"\e169"}.glyphicon-export:before{content:"\e170"}.glyphicon-send:before{content:"\e171"}.glyphicon-floppy-disk:before{content:"\e172"}.glyphicon-floppy-saved:before{content:"\e173"}.glyphicon-floppy-remove:before{content:"\e174"}.glyphicon-floppy-save:before{content:"\e175"}.glyphicon-floppy-open:before{content:"\e176"}.glyphicon-credit-card:before{content:"\e177"}.glyphicon-transfer:before{content:"\e178"}.glyphicon-cutlery:before{content:"\e179"}.glyphicon-header:before{content:"\e180"}.glyphicon-compressed:before{content:"\e181"}.glyphicon-earphone:before{content:"\e182"}.glyphicon-phone-alt:before{content:"\e183"}.glyphicon-tower:before{content:"\e184"}.glyphicon-stats:before{content:"\e185"}.glyphicon-sd-video:before{content:"\e186"}.glyphicon-hd-video:before{content:"\e187"}.glyphicon-subtitles:before{content:"\e188"}.glyphicon-sound-stereo:before{content:"\e189"}.glyphicon-sound-dolby:before{content:"\e190"}.glyphicon-sound-5-1:before{content:"\e191"}.glyphicon-sound-6-1:before{content:"\e192"}.glyphicon-sound-7-1:before{content:"\e193"}.glyphicon-copyright-mark:before{content:"\e194"}.glyphicon-registration-mark:before{content:"\e195"}.glyphicon-cloud-download:before{content:"\e197"}.glyphicon-cloud-upload:before{content:"\e198"}.glyphicon-tree-conifer:before{content:"\e199"}.glyphicon-tree-deciduous:before{content:"\e200"}.glyphicon-cd:before{content:"\e201"}.glyphicon-save-file:before{content:"\e202"}.glyphicon-open-file:before{content:"\e203"}.glyphicon-level-up:before{content:"\e204"}.glyphicon-copy:before{content:"\e205"}.glyphicon-paste:before{content:"\e206"}.glyphicon-alert:before{content:"\e209"}.glyphicon-equalizer:before{content:"\e210"}.glyphicon-king:before{content:"\e211"}.glyphicon-queen:before{content:"\e212"}.glyphicon-pawn:before{content:"\e213"}.glyphicon-bishop:before{content:"\e214"}.glyphicon-knight:before{content:"\e215"}.glyphicon-baby-formula:before{content:"\e216"}.glyphicon-tent:before{content:"\26fa"}.glyphicon-blackboard:before{content:"\e218"}.glyphicon-bed:before{content:"\e219"}.glyphicon-apple:before{content:"\f8ff"}.glyphicon-erase:before{content:"\e221"}.glyphicon-hourglass:before{content:"\231b"}.glyphicon-lamp:before{content:"\e223"}.glyphicon-duplicate:before{content:"\e224"}.glyphicon-piggy-bank:before{content:"\e225"}.glyphicon-scissors:before{content:"\e226"}.glyphicon-bitcoin:before,.glyphicon-btc:before,.glyphicon-xbt:before{content:"\e227"}.glyphicon-jpy:before,.glyphicon-yen:before{content:"\00a5"}.glyphicon-rub:before,.glyphicon-ruble:before{content:"\20bd"}.glyphicon-scale:before{content:"\e230"}.glyphicon-ice-lolly:before{content:"\e231"}.glyphicon-ice-lolly-tasted:before{content:"\e232"}.glyphicon-education:before{content:"\e233"}.glyphicon-option-horizontal:before{content:"\e234"}.glyphicon-option-vertical:before{content:"\e235"}.glyphicon-menu-hamburger:before{content:"\e236"}.glyphicon-modal-window:before{content:"\e237"}.glyphicon-oil:before{content:"\e238"}.glyphicon-grain:before{content:"\e239"}.glyphicon-sunglasses:before{content:"\e240"}.glyphicon-text-size:before{content:"\e241"}.glyphicon-text-color:before{content:"\e242"}.glyphicon-text-background:before{content:"\e243"}.glyphicon-object-align-top:before{content:"\e244"}.glyphicon-object-align-bottom:before{content:"\e245"}.glyphicon-object-align-horizontal:before{content:"\e246"}.glyphicon-object-align-left:before{content:"\e247"}.glyphicon-object-align-vertical:before{content:"\e248"}.glyphicon-object-align-right:before{content:"\e249"}.glyphicon-triangle-right:before{content:"\e250"}.glyphicon-triangle-left:before{content:"\e251"}.glyphicon-triangle-bottom:before{content:"\e252"}.glyphicon-triangle-top:before{content:"\e253"}.glyphicon-console:before{content:"\e254"}.glyphicon-superscript:before{content:"\e255"}.glyphicon-subscript:before{content:"\e256"}.glyphicon-menu-left:before{content:"\e257"}.glyphicon-menu-right:before{content:"\e258"}.glyphicon-menu-down:before{content:"\e259"}.glyphicon-menu-up:before{content:"\e260"}*,:after,:before{-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}body{margin:0;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:13px;line-height:1.42857143;color:#000;background-color:#fff}button,input,select,textarea{font-family:inherit;font-size:inherit;line-height:inherit}a{color:#337ab7;text-decoration:none}a:focus,a:hover{color:#23527c;text-decoration:underline}a:focus{outline:dotted thin;outline:-webkit-focus-ring-color auto 5px;outline-offset:-2px}figure{margin:0}.carousel-inner>.item>a>img,.carousel-inner>.item>img,.img-responsive,.thumbnail a>img,.thumbnail>img{display:block;max-width:100%;height:auto}.img-rounded{border-radius:3px}.img-thumbnail{padding:4px;line-height:1.42857143;background-color:#fff;border:1px solid #ddd;border-radius:2px;-webkit-transition:all .2s ease-in-out;-o-transition:all .2s ease-in-out;transition:all .2s ease-in-out;display:inline-block;max-width:100%;height:auto}.img-circle{border-radius:50%}hr{margin-top:18px;margin-bottom:18px;border:0;border-top:1px solid #eee}.sr-only{position:absolute;width:1px;height:1px;margin:-1px;padding:0;overflow:hidden;clip:rect(0,0,0,0);border:0}.sr-only-focusable:active,.sr-only-focusable:focus{position:static;width:auto;height:auto;margin:0;overflow:visible;clip:auto}[role=button]{cursor:pointer}.h1,.h2,.h3,.h4,.h5,.h6,h1,h2,h3,h4,h5,h6{font-family:inherit;font-weight:500;line-height:1.1;color:inherit}.h1 .small,.h1 small,.h2 .small,.h2 small,.h3 .small,.h3 small,.h4 .small,.h4 small,.h5 .small,.h5 small,.h6 .small,.h6 small,h1 .small,h1 small,h2 .small,h2 small,h3 .small,h3 small,h4 .small,h4 small,h5 .small,h5 small,h6 .small,h6 small{font-weight:400;line-height:1;color:#777}.h1,.h2,.h3,h1,h2,h3{margin-top:18px;margin-bottom:9px}.h1 .small,.h1 small,.h2 .small,.h2 small,.h3 .small,.h3 small,h1 .small,h1 small,h2 .small,h2 small,h3 .small,h3 small{font-size:65%}.h4,.h5,.h6,h4,h5,h6{margin-top:9px;margin-bottom:9px}.h4 .small,.h4 small,.h5 .small,.h5 small,.h6 .small,.h6 small,h4 .small,h4 small,h5 .small,h5 small,h6 .small,h6 small{font-size:75%}.h1,h1{font-size:33px}.h2,h2{font-size:27px}.h3,h3{font-size:23px}.h4,h4{font-size:17px}.h5,h5{font-size:13px}.h6,h6{font-size:12px}p{margin:0 0 9px}.lead{margin-bottom:18px;font-size:14px;font-weight:300;line-height:1.4}@media (min-width:768px){.lead{font-size:19.5px}}.small,small{font-size:92%}.mark,mark{background-color:#fcf8e3;padding:.2em}.text-left{text-align:left}.text-right{text-align:right}.text-center{text-align:center}.text-justify{text-align:justify}.text-nowrap{white-space:nowrap}.text-lowercase{text-transform:lowercase}.text-uppercase{text-transform:uppercase}.text-capitalize{text-transform:capitalize}.text-muted{color:#777}.text-primary{color:#337ab7}a.text-primary:hover{color:#286090}.text-success{color:#3c763d}a.text-success:hover{color:#2b542c}.text-info{color:#31708f}a.text-info:hover{color:#245269}.text-warning{color:#8a6d3b}a.text-warning:hover{color:#66512c}.text-danger{color:#a94442}a.text-danger:hover{color:#843534}.bg-primary{color:#fff;background-color:#337ab7}a.bg-primary:hover{background-color:#286090}.bg-success{background-color:#dff0d8}a.bg-success:hover{background-color:#c1e2b3}.bg-info{background-color:#d9edf7}a.bg-info:hover{background-color:#afd9ee}.bg-warning{background-color:#fcf8e3}a.bg-warning:hover{background-color:#f7ecb5}.bg-danger{background-color:#f2dede}a.bg-danger:hover{background-color:#e4b9b9}.page-header{padding-bottom:8px;margin:36px 0 18px;border-bottom:1px solid #eee}ol,ul{margin-top:0;margin-bottom:9px}ol ol,ol ul,ul ol,ul ul{margin-bottom:0}.list-unstyled{padding-left:0;list-style:none}.list-inline{padding-left:0;list-style:none;margin-left:-5px}.list-inline>li{display:inline-block;padding-left:5px;padding-right:5px}dl{margin-top:0;margin-bottom:18px}dd,dt{line-height:1.42857143}dt{font-weight:700}dd{margin-left:0}@media (min-width:541px){.dl-horizontal dt{float:left;width:160px;clear:left;text-align:right;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}.dl-horizontal dd{margin-left:180px}}abbr[data-original-title],abbr[title]{cursor:help;border-bottom:1px dotted #777}.initialism{font-size:90%;text-transform:uppercase}blockquote{padding:9px 18px;margin:0 0 18px;font-size:inherit;border-left:5px solid #eee}blockquote ol:last-child,blockquote p:last-child,blockquote ul:last-child{margin-bottom:0}blockquote .small,blockquote footer,blockquote small{display:block;font-size:80%;line-height:1.42857143;color:#777}blockquote .small:before,blockquote footer:before,blockquote small:before{content:'\2014 \00A0'}.blockquote-reverse,blockquote.pull-right{padding-right:15px;padding-left:0;border-right:5px solid #eee;border-left:0;text-align:right}.blockquote-reverse .small:before,.blockquote-reverse footer:before,.blockquote-reverse small:before,blockquote.pull-right .small:before,blockquote.pull-right footer:before,blockquote.pull-right small:before{content:''}.blockquote-reverse .small:after,.blockquote-reverse footer:after,.blockquote-reverse small:after,blockquote.pull-right .small:after,blockquote.pull-right footer:after,blockquote.pull-right small:after{content:'\00A0 \2014'}address{margin-bottom:18px;font-style:normal;line-height:1.42857143}code,kbd,pre,samp{font-family:monospace}code{padding:2px 4px;font-size:90%;background-color:#f9f2f4;border-radius:2px}kbd{padding:2px 4px;font-size:90%;color:#fff;background-color:#333;border-radius:1px;box-shadow:inset 0 -1px 0 rgba(0,0,0,.25)}kbd kbd{padding:0;font-size:100%;font-weight:700;box-shadow:none}pre{display:block;padding:8.5px;margin:0 0 9px;word-break:break-all;word-wrap:break-word;color:#333;background-color:#f5f5f5;border:1px solid #ccc;border-radius:2px}pre code{padding:0;font-size:inherit;color:inherit;white-space:pre-wrap;background-color:transparent;border-radius:0}.pre-scrollable{max-height:340px;overflow-y:scroll}.container{margin-right:auto;margin-left:auto;padding-left:0;padding-right:0}@media (min-width:768px){.container{width:768px}}@media (min-width:992px){.container{width:940px}}@media (min-width:auto){.container{width:auto}}.container-fluid{margin-right:auto;margin-left:auto;padding-left:0;padding-right:0}.row{margin-left:0;margin-right:0}.col-lg-1,.col-lg-10,.col-lg-11,.col-lg-12,.col-lg-2,.col-lg-3,.col-lg-4,.col-lg-5,.col-lg-6,.col-lg-7,.col-lg-8,.col-lg-9,.col-md-1,.col-md-10,.col-md-11,.col-md-12,.col-md-2,.col-md-3,.col-md-4,.col-md-5,.col-md-6,.col-md-7,.col-md-8,.col-md-9,.col-sm-1,.col-sm-10,.col-sm-11,.col-sm-12,.col-sm-2,.col-sm-3,.col-sm-4,.col-sm-5,.col-sm-6,.col-sm-7,.col-sm-8,.col-sm-9,.col-xs-1,.col-xs-10,.col-xs-11,.col-xs-12,.col-xs-2,.col-xs-3,.col-xs-4,.col-xs-5,.col-xs-6,.col-xs-7,.col-xs-8,.col-xs-9{position:relative;min-height:1px;padding-left:0;padding-right:0}.col-xs-1,.col-xs-10,.col-xs-11,.col-xs-12,.col-xs-2,.col-xs-3,.col-xs-4,.col-xs-5,.col-xs-6,.col-xs-7,.col-xs-8,.col-xs-9{float:left}.col-xs-12{width:100%}.col-xs-11{width:91.66666667%}.col-xs-10{width:83.33333333%}.col-xs-9{width:75%}.col-xs-8{width:66.66666667%}.col-xs-7{width:58.33333333%}.col-xs-6{width:50%}.col-xs-5{width:41.66666667%}.col-xs-4{width:33.33333333%}.col-xs-3{width:25%}.col-xs-2{width:16.66666667%}.col-xs-1{width:8.33333333%}.col-xs-pull-12{right:100%}.col-xs-pull-11{right:91.66666667%}.col-xs-pull-10{right:83.33333333%}.col-xs-pull-9{right:75%}.col-xs-pull-8{right:66.66666667%}.col-xs-pull-7{right:58.33333333%}.col-xs-pull-6{right:50%}.col-xs-pull-5{right:41.66666667%}.col-xs-pull-4{right:33.33333333%}.col-xs-pull-3{right:25%}.col-xs-pull-2{right:16.66666667%}.col-xs-pull-1{right:8.33333333%}.col-xs-pull-0{right:auto}.col-xs-push-12{left:100%}.col-xs-push-11{left:91.66666667%}.col-xs-push-10{left:83.33333333%}.col-xs-push-9{left:75%}.col-xs-push-8{left:66.66666667%}.col-xs-push-7{left:58.33333333%}.col-xs-push-6{left:50%}.col-xs-push-5{left:41.66666667%}.col-xs-push-4{left:33.33333333%}.col-xs-push-3{left:25%}.col-xs-push-2{left:16.66666667%}.col-xs-push-1{left:8.33333333%}.col-xs-push-0{left:auto}.col-xs-offset-12{margin-left:100%}.col-xs-offset-11{margin-left:91.66666667%}.col-xs-offset-10{margin-left:83.33333333%}.col-xs-offset-9{margin-left:75%}.col-xs-offset-8{margin-left:66.66666667%}.col-xs-offset-7{margin-left:58.33333333%}.col-xs-offset-6{margin-left:50%}.col-xs-offset-5{margin-left:41.66666667%}.col-xs-offset-4{margin-left:33.33333333%}.col-xs-offset-3{margin-left:25%}.col-xs-offset-2{margin-left:16.66666667%}.col-xs-offset-1{margin-left:8.33333333%}.col-xs-offset-0{margin-left:0}@media (min-width:768px){.col-sm-1,.col-sm-10,.col-sm-11,.col-sm-12,.col-sm-2,.col-sm-3,.col-sm-4,.col-sm-5,.col-sm-6,.col-sm-7,.col-sm-8,.col-sm-9{float:left}.col-sm-12{width:100%}.col-sm-11{width:91.66666667%}.col-sm-10{width:83.33333333%}.col-sm-9{width:75%}.col-sm-8{width:66.66666667%}.col-sm-7{width:58.33333333%}.col-sm-6{width:50%}.col-sm-5{width:41.66666667%}.col-sm-4{width:33.33333333%}.col-sm-3{width:25%}.col-sm-2{width:16.66666667%}.col-sm-1{width:8.33333333%}.col-sm-pull-12{right:100%}.col-sm-pull-11{right:91.66666667%}.col-sm-pull-10{right:83.33333333%}.col-sm-pull-9{right:75%}.col-sm-pull-8{right:66.66666667%}.col-sm-pull-7{right:58.33333333%}.col-sm-pull-6{right:50%}.col-sm-pull-5{right:41.66666667%}.col-sm-pull-4{right:33.33333333%}.col-sm-pull-3{right:25%}.col-sm-pull-2{right:16.66666667%}.col-sm-pull-1{right:8.33333333%}.col-sm-pull-0{right:auto}.col-sm-push-12{left:100%}.col-sm-push-11{left:91.66666667%}.col-sm-push-10{left:83.33333333%}.col-sm-push-9{left:75%}.col-sm-push-8{left:66.66666667%}.col-sm-push-7{left:58.33333333%}.col-sm-push-6{left:50%}.col-sm-push-5{left:41.66666667%}.col-sm-push-4{left:33.33333333%}.col-sm-push-3{left:25%}.col-sm-push-2{left:16.66666667%}.col-sm-push-1{left:8.33333333%}.col-sm-push-0{left:auto}.col-sm-offset-12{margin-left:100%}.col-sm-offset-11{margin-left:91.66666667%}.col-sm-offset-10{margin-left:83.33333333%}.col-sm-offset-9{margin-left:75%}.col-sm-offset-8{margin-left:66.66666667%}.col-sm-offset-7{margin-left:58.33333333%}.col-sm-offset-6{margin-left:50%}.col-sm-offset-5{margin-left:41.66666667%}.col-sm-offset-4{margin-left:33.33333333%}.col-sm-offset-3{margin-left:25%}.col-sm-offset-2{margin-left:16.66666667%}.col-sm-offset-1{margin-left:8.33333333%}.col-sm-offset-0{margin-left:0}}@media (min-width:auto){.col-md-1,.col-md-10,.col-md-11,.col-md-12,.col-md-2,.col-md-3,.col-md-4,.col-md-5,.col-md-6,.col-md-7,.col-md-8,.col-md-9{float:left}.col-md-12{width:100%}.col-md-11{width:91.66666667%}.col-md-10{width:83.33333333%}.col-md-9{width:75%}.col-md-8{width:66.66666667%}.col-md-7{width:58.33333333%}.col-md-6{width:50%}.col-md-5{width:41.66666667%}.col-md-4{width:33.33333333%}.col-md-3{width:25%}.col-md-2{width:16.66666667%}.col-md-1{width:8.33333333%}.col-md-pull-12{right:100%}.col-md-pull-11{right:91.66666667%}.col-md-pull-10{right:83.33333333%}.col-md-pull-9{right:75%}.col-md-pull-8{right:66.66666667%}.col-md-pull-7{right:58.33333333%}.col-md-pull-6{right:50%}.col-md-pull-5{right:41.66666667%}.col-md-pull-4{right:33.33333333%}.col-md-pull-3{right:25%}.col-md-pull-2{right:16.66666667%}.col-md-pull-1{right:8.33333333%}.col-md-pull-0{right:auto}.col-md-push-12{left:100%}.col-md-push-11{left:91.66666667%}.col-md-push-10{left:83.33333333%}.col-md-push-9{left:75%}.col-md-push-8{left:66.66666667%}.col-md-push-7{left:58.33333333%}.col-md-push-6{left:50%}.col-md-push-5{left:41.66666667%}.col-md-push-4{left:33.33333333%}.col-md-push-3{left:25%}.col-md-push-2{left:16.66666667%}.col-md-push-1{left:8.33333333%}.col-md-push-0{left:auto}.col-md-offset-12{margin-left:100%}.col-md-offset-11{margin-left:91.66666667%}.col-md-offset-10{margin-left:83.33333333%}.col-md-offset-9{margin-left:75%}.col-md-offset-8{margin-left:66.66666667%}.col-md-offset-7{margin-left:58.33333333%}.col-md-offset-6{margin-left:50%}.col-md-offset-5{margin-left:41.66666667%}.col-md-offset-4{margin-left:33.33333333%}.col-md-offset-3{margin-left:25%}.col-md-offset-2{margin-left:16.66666667%}.col-md-offset-1{margin-left:8.33333333%}.col-md-offset-0{margin-left:0}}@media (min-width:auto){.col-lg-1,.col-lg-10,.col-lg-11,.col-lg-12,.col-lg-2,.col-lg-3,.col-lg-4,.col-lg-5,.col-lg-6,.col-lg-7,.col-lg-8,.col-lg-9{float:left}.col-lg-12{width:100%}.col-lg-11{width:91.66666667%}.col-lg-10{width:83.33333333%}.col-lg-9{width:75%}.col-lg-8{width:66.66666667%}.col-lg-7{width:58.33333333%}.col-lg-6{width:50%}.col-lg-5{width:41.66666667%}.col-lg-4{width:33.33333333%}.col-lg-3{width:25%}.col-lg-2{width:16.66666667%}.col-lg-1{width:8.33333333%}.col-lg-pull-12{right:100%}.col-lg-pull-11{right:91.66666667%}.col-lg-pull-10{right:83.33333333%}.col-lg-pull-9{right:75%}.col-lg-pull-8{right:66.66666667%}.col-lg-pull-7{right:58.33333333%}.col-lg-pull-6{right:50%}.col-lg-pull-5{right:41.66666667%}.col-lg-pull-4{right:33.33333333%}.col-lg-pull-3{right:25%}.col-lg-pull-2{right:16.66666667%}.col-lg-pull-1{right:8.33333333%}.col-lg-pull-0{right:auto}.col-lg-push-12{left:100%}.col-lg-push-11{left:91.66666667%}.col-lg-push-10{left:83.33333333%}.col-lg-push-9{left:75%}.col-lg-push-8{left:66.66666667%}.col-lg-push-7{left:58.33333333%}.col-lg-push-6{left:50%}.col-lg-push-5{left:41.66666667%}.col-lg-push-4{left:33.33333333%}.col-lg-push-3{left:25%}.col-lg-push-2{left:16.66666667%}.col-lg-push-1{left:8.33333333%}.col-lg-push-0{left:auto}.col-lg-offset-12{margin-left:100%}.col-lg-offset-11{margin-left:91.66666667%}.col-lg-offset-10{margin-left:83.33333333%}.col-lg-offset-9{margin-left:75%}.col-lg-offset-8{margin-left:66.66666667%}.col-lg-offset-7{margin-left:58.33333333%}.col-lg-offset-6{margin-left:50%}.col-lg-offset-5{margin-left:41.66666667%}.col-lg-offset-4{margin-left:33.33333333%}.col-lg-offset-3{margin-left:25%}.col-lg-offset-2{margin-left:16.66666667%}.col-lg-offset-1{margin-left:8.33333333%}.col-lg-offset-0{margin-left:0}}table{background-color:transparent}caption{padding-top:8px;padding-bottom:8px;color:#777;text-align:left}th{text-align:left}.table{width:100%;max-width:100%;margin-bottom:18px}.table>tbody>tr>td,.table>tbody>tr>th,.table>tfoot>tr>td,.table>tfoot>tr>th,.table>thead>tr>td,.table>thead>tr>th{padding:8px;line-height:1.42857143;vertical-align:top;border-top:1px solid #ddd}.table>thead>tr>th{vertical-align:bottom;border-bottom:2px solid #ddd}.table>caption+thead>tr:first-child>td,.table>caption+thead>tr:first-child>th,.table>colgroup+thead>tr:first-child>td,.table>colgroup+thead>tr:first-child>th,.table>thead:first-child>tr:first-child>td,.table>thead:first-child>tr:first-child>th{border-top:0}.table>tbody+tbody{border-top:2px solid #ddd}.table .table{background-color:#fff}.table-condensed>tbody>tr>td,.table-condensed>tbody>tr>th,.table-condensed>tfoot>tr>td,.table-condensed>tfoot>tr>th,.table-condensed>thead>tr>td,.table-condensed>thead>tr>th{padding:5px}.table-bordered,.table-bordered>tbody>tr>td,.table-bordered>tbody>tr>th,.table-bordered>tfoot>tr>td,.table-bordered>tfoot>tr>th,.table-bordered>thead>tr>td,.table-bordered>thead>tr>th{border:1px solid #ddd}.table-bordered>thead>tr>td,.table-bordered>thead>tr>th{border-bottom-width:2px}.table-striped>tbody>tr:nth-of-type(odd){background-color:#f9f9f9}.table-hover>tbody>tr:hover{background-color:#f5f5f5}table col[class*=col-]{position:static;float:none;display:table-column}table td[class*=col-],table th[class*=col-]{position:static;float:none;display:table-cell}.table>tbody>tr.active>td,.table>tbody>tr.active>th,.table>tbody>tr>td.active,.table>tbody>tr>th.active,.table>tfoot>tr.active>td,.table>tfoot>tr.active>th,.table>tfoot>tr>td.active,.table>tfoot>tr>th.active,.table>thead>tr.active>td,.table>thead>tr.active>th,.table>thead>tr>td.active,.table>thead>tr>th.active{background-color:#f5f5f5}.table-hover>tbody>tr.active:hover>td,.table-hover>tbody>tr.active:hover>th,.table-hover>tbody>tr:hover>.active,.table-hover>tbody>tr>td.active:hover,.table-hover>tbody>tr>th.active:hover{background-color:#e8e8e8}.table>tbody>tr.success>td,.table>tbody>tr.success>th,.table>tbody>tr>td.success,.table>tbody>tr>th.success,.table>tfoot>tr.success>td,.table>tfoot>tr.success>th,.table>tfoot>tr>td.success,.table>tfoot>tr>th.success,.table>thead>tr.success>td,.table>thead>tr.success>th,.table>thead>tr>td.success,.table>thead>tr>th.success{background-color:#dff0d8}.table-hover>tbody>tr.success:hover>td,.table-hover>tbody>tr.success:hover>th,.table-hover>tbody>tr:hover>.success,.table-hover>tbody>tr>td.success:hover,.table-hover>tbody>tr>th.success:hover{background-color:#d0e9c6}.table>tbody>tr.info>td,.table>tbody>tr.info>th,.table>tbody>tr>td.info,.table>tbody>tr>th.info,.table>tfoot>tr.info>td,.table>tfoot>tr.info>th,.table>tfoot>tr>td.info,.table>tfoot>tr>th.info,.table>thead>tr.info>td,.table>thead>tr.info>th,.table>thead>tr>td.info,.table>thead>tr>th.info{background-color:#d9edf7}.table-hover>tbody>tr.info:hover>td,.table-hover>tbody>tr.info:hover>th,.table-hover>tbody>tr:hover>.info,.table-hover>tbody>tr>td.info:hover,.table-hover>tbody>tr>th.info:hover{background-color:#c4e3f3}.table>tbody>tr.warning>td,.table>tbody>tr.warning>th,.table>tbody>tr>td.warning,.table>tbody>tr>th.warning,.table>tfoot>tr.warning>td,.table>tfoot>tr.warning>th,.table>tfoot>tr>td.warning,.table>tfoot>tr>th.warning,.table>thead>tr.warning>td,.table>thead>tr.warning>th,.table>thead>tr>td.warning,.table>thead>tr>th.warning{background-color:#fcf8e3}.table-hover>tbody>tr.warning:hover>td,.table-hover>tbody>tr.warning:hover>th,.table-hover>tbody>tr:hover>.warning,.table-hover>tbody>tr>td.warning:hover,.table-hover>tbody>tr>th.warning:hover{background-color:#faf2cc}.table>tbody>tr.danger>td,.table>tbody>tr.danger>th,.table>tbody>tr>td.danger,.table>tbody>tr>th.danger,.table>tfoot>tr.danger>td,.table>tfoot>tr.danger>th,.table>tfoot>tr>td.danger,.table>tfoot>tr>th.danger,.table>thead>tr.danger>td,.table>thead>tr.danger>th,.table>thead>tr>td.danger,.table>thead>tr>th.danger{background-color:#f2dede}.table-hover>tbody>tr.danger:hover>td,.table-hover>tbody>tr.danger:hover>th,.table-hover>tbody>tr:hover>.danger,.table-hover>tbody>tr>td.danger:hover,.table-hover>tbody>tr>th.danger:hover{background-color:#ebcccc}.table-responsive{overflow-x:auto;min-height:.01%}@media screen and (max-width:767px){.table-responsive{width:100%;margin-bottom:13.5px;overflow-y:hidden;-ms-overflow-style:-ms-autohiding-scrollbar;border:1px solid #ddd}.table-responsive>.table{margin-bottom:0}.table-responsive>.table>tbody>tr>td,.table-responsive>.table>tbody>tr>th,.table-responsive>.table>tfoot>tr>td,.table-responsive>.table>tfoot>tr>th,.table-responsive>.table>thead>tr>td,.table-responsive>.table>thead>tr>th{white-space:nowrap}.table-responsive>.table-bordered{border:0}.table-responsive>.table-bordered>tbody>tr>td:first-child,.table-responsive>.table-bordered>tbody>tr>th:first-child,.table-responsive>.table-bordered>tfoot>tr>td:first-child,.table-responsive>.table-bordered>tfoot>tr>th:first-child,.table-responsive>.table-bordered>thead>tr>td:first-child,.table-responsive>.table-bordered>thead>tr>th:first-child{border-left:0}.table-responsive>.table-bordered>tbody>tr>td:last-child,.table-responsive>.table-bordered>tbody>tr>th:last-child,.table-responsive>.table-bordered>tfoot>tr>td:last-child,.table-responsive>.table-bordered>tfoot>tr>th:last-child,.table-responsive>.table-bordered>thead>tr>td:last-child,.table-responsive>.table-bordered>thead>tr>th:last-child{border-right:0}.table-responsive>.table-bordered>tbody>tr:last-child>td,.table-responsive>.table-bordered>tbody>tr:last-child>th,.table-responsive>.table-bordered>tfoot>tr:last-child>td,.table-responsive>.table-bordered>tfoot>tr:last-child>th{border-bottom:0}}fieldset{padding:0;margin:0;border:0;min-width:0}legend{display:block;width:100%;padding:0;margin-bottom:18px;font-size:19.5px;line-height:inherit;color:#333;border:0;border-bottom:1px solid #e5e5e5}label{display:inline-block;max-width:100%;margin-bottom:5px}input[type=search]{-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box;-webkit-appearance:none}input[type=checkbox],input[type=radio]{margin:4px 0 0;margin-top:1px \9;line-height:normal}input[type=file]{display:block}input[type=range]{display:block;width:100%}select[multiple],select[size]{height:auto}input[type=file]:focus,input[type=checkbox]:focus,input[type=radio]:focus{outline:dotted thin;outline:-webkit-focus-ring-color auto 5px;outline-offset:-2px}output{display:block;padding-top:7px;font-size:13px;line-height:1.42857143;color:#555}.form-control{display:block;width:100%;height:32px;padding:6px 12px;font-size:13px;line-height:1.42857143;color:#555;background-color:#fff;background-image:none;border:1px solid #ccc;border-radius:2px;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075);box-shadow:inset 0 1px 1px rgba(0,0,0,.075);-webkit-transition:border-color ease-in-out .15s,box-shadow ease-in-out .15s;-o-transition:border-color ease-in-out .15s,box-shadow ease-in-out .15s;transition:border-color ease-in-out .15s,box-shadow ease-in-out .15s}.form-control:focus{border-color:#66afe9;outline:0;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 8px rgba(102,175,233,.6);box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 8px rgba(102,175,233,.6)}.form-control::-moz-placeholder{color:#999;opacity:1}.form-control:-ms-input-placeholder{color:#999}.form-control::-webkit-input-placeholder{color:#999}.form-control[disabled],.form-control[readonly],fieldset[disabled] .form-control{background-color:#eee;opacity:1}.form-control[disabled],fieldset[disabled] .form-control{cursor:not-allowed}textarea.form-control{height:auto}@media screen and (-webkit-min-device-pixel-ratio:0){input[type=date],input[type=time],input[type=datetime-local],input[type=month]{line-height:32px}.input-group-sm input[type=date],.input-group-sm input[type=time],.input-group-sm input[type=datetime-local],.input-group-sm input[type=month],input[type=date].input-sm,input[type=time].input-sm,input[type=datetime-local].input-sm,input[type=month].input-sm{line-height:30px}.input-group-lg input[type=date],.input-group-lg input[type=time],.input-group-lg input[type=datetime-local],.input-group-lg input[type=month],input[type=date].input-lg,input[type=time].input-lg,input[type=datetime-local].input-lg,input[type=month].input-lg{line-height:45px}}.form-group{margin-bottom:15px}.checkbox,.radio{position:relative;display:block;margin-top:10px;margin-bottom:10px}.checkbox label,.radio label{min-height:18px;padding-left:20px;margin-bottom:0;font-weight:400;cursor:pointer}.checkbox input[type=checkbox],.checkbox-inline input[type=checkbox],.radio input[type=radio],.radio-inline input[type=radio]{position:absolute;margin-left:-20px;margin-top:4px \9}.checkbox+.checkbox,.radio+.radio{margin-top:-5px}.checkbox-inline,.radio-inline{position:relative;display:inline-block;padding-left:20px;margin-bottom:0;vertical-align:middle;font-weight:400;cursor:pointer}.checkbox-inline+.checkbox-inline,.radio-inline+.radio-inline{margin-top:0;margin-left:10px}.checkbox-inline.disabled,.checkbox.disabled label,.radio-inline.disabled,.radio.disabled label,fieldset[disabled] .checkbox label,fieldset[disabled] .checkbox-inline,fieldset[disabled] .radio label,fieldset[disabled] .radio-inline,fieldset[disabled] input[type=checkbox],fieldset[disabled] input[type=radio],input[type=checkbox].disabled,input[type=checkbox][disabled],input[type=radio].disabled,input[type=radio][disabled]{cursor:not-allowed}.form-control-static{padding-top:7px;padding-bottom:7px;margin-bottom:0;min-height:31px}.form-control-static.input-lg,.form-control-static.input-sm{padding-left:0;padding-right:0}.input-sm{height:30px;padding:5px 10px;font-size:12px;line-height:1.5;border-radius:1px}select.input-sm{height:30px;line-height:30px}select[multiple].input-sm,textarea.input-sm{height:auto}.form-group-sm .form-control{height:30px;padding:5px 10px;font-size:12px;line-height:1.5;border-radius:1px}select.form-group-sm .form-control{height:30px;line-height:30px}select[multiple].form-group-sm .form-control,textarea.form-group-sm .form-control{height:auto}.form-group-sm .form-control-static{height:30px;padding:5px 10px;font-size:12px;line-height:1.5;min-height:30px}.input-lg{height:45px;padding:10px 16px;font-size:17px;line-height:1.3333333;border-radius:3px}select.input-lg{height:45px;line-height:45px}select[multiple].input-lg,textarea.input-lg{height:auto}.form-group-lg .form-control{height:45px;padding:10px 16px;font-size:17px;line-height:1.3333333;border-radius:3px}select.form-group-lg .form-control{height:45px;line-height:45px}select[multiple].form-group-lg .form-control,textarea.form-group-lg .form-control{height:auto}.form-group-lg .form-control-static{height:45px;padding:10px 16px;font-size:17px;line-height:1.3333333;min-height:35px}.has-feedback{position:relative}.has-feedback .form-control{padding-right:40px}.form-control-feedback{position:absolute;top:0;right:0;z-index:2;display:block;width:32px;height:32px;line-height:32px;text-align:center;pointer-events:none}.input-lg+.form-control-feedback{width:45px;height:45px;line-height:45px}.input-sm+.form-control-feedback{width:30px;height:30px;line-height:30px}.has-success .checkbox,.has-success .checkbox-inline,.has-success .control-label,.has-success .help-block,.has-success .radio,.has-success .radio-inline,.has-success.checkbox label,.has-success.checkbox-inline label,.has-success.radio label,.has-success.radio-inline label{color:#3c763d}.has-success .form-control{border-color:#3c763d;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075);box-shadow:inset 0 1px 1px rgba(0,0,0,.075)}.has-success .form-control:focus{border-color:#2b542c;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #67b168;box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #67b168}.has-success .input-group-addon{color:#3c763d;border-color:#3c763d;background-color:#dff0d8}.has-success .form-control-feedback{color:#3c763d}.has-warning .checkbox,.has-warning .checkbox-inline,.has-warning .control-label,.has-warning .help-block,.has-warning .radio,.has-warning .radio-inline,.has-warning.checkbox label,.has-warning.checkbox-inline label,.has-warning.radio label,.has-warning.radio-inline label{color:#8a6d3b}.has-warning .form-control{border-color:#8a6d3b;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075);box-shadow:inset 0 1px 1px rgba(0,0,0,.075)}.has-warning .form-control:focus{border-color:#66512c;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #c0a16b;box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #c0a16b}.has-warning .input-group-addon{color:#8a6d3b;border-color:#8a6d3b;background-color:#fcf8e3}.has-warning .form-control-feedback{color:#8a6d3b}.has-error .checkbox,.has-error .checkbox-inline,.has-error .control-label,.has-error .help-block,.has-error .radio,.has-error .radio-inline,.has-error.checkbox label,.has-error.checkbox-inline label,.has-error.radio label,.has-error.radio-inline label{color:#a94442}.has-error .form-control{border-color:#a94442;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075);box-shadow:inset 0 1px 1px rgba(0,0,0,.075)}.has-error .form-control:focus{border-color:#843534;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #ce8483;box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 6px #ce8483}.has-error .input-group-addon{color:#a94442;border-color:#a94442;background-color:#f2dede}.has-error .form-control-feedback{color:#a94442}.has-feedback label~.form-control-feedback{top:23px}.has-feedback label.sr-only~.form-control-feedback{top:0}.help-block{display:block;margin-top:5px;margin-bottom:10px;color:#404040}.form-horizontal .checkbox,.form-horizontal .checkbox-inline,.form-horizontal .radio,.form-horizontal .radio-inline{margin-top:0;margin-bottom:0;padding-top:7px}.form-horizontal .checkbox,.form-horizontal .radio{min-height:25px}.form-horizontal .form-group{margin-left:0;margin-right:0}.form-horizontal .has-feedback .form-control-feedback{right:0}@media (min-width:768px){.form-inline .form-group{display:inline-block;margin-bottom:0;vertical-align:middle}.form-inline .form-control{display:inline-block;width:auto;vertical-align:middle}.form-inline .form-control-static{display:inline-block}.form-inline .input-group{display:inline-table;vertical-align:middle}.form-inline .input-group .form-control,.form-inline .input-group .input-group-addon,.form-inline .input-group .input-group-btn{width:auto}.form-inline .input-group>.form-control{width:100%}.form-inline .control-label{margin-bottom:0;vertical-align:middle}.form-inline .checkbox,.form-inline .radio{display:inline-block;margin-top:0;margin-bottom:0;vertical-align:middle}.form-inline .checkbox label,.form-inline .radio label{padding-left:0}.form-inline .checkbox input[type=checkbox],.form-inline .radio input[type=radio]{position:relative;margin-left:0}.form-inline .has-feedback .form-control-feedback{top:0}.form-horizontal .control-label{text-align:right;margin-bottom:0;padding-top:7px}.form-horizontal .form-group-lg .control-label{padding-top:14.33px}.form-horizontal .form-group-sm .control-label{padding-top:6px}}.btn{display:inline-block;margin-bottom:0;font-weight:400;text-align:center;vertical-align:middle;touch-action:manipulation;cursor:pointer;background-image:none;border:1px solid transparent;white-space:nowrap;padding:6px 12px;font-size:13px;line-height:1.42857143;border-radius:2px;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}.btn.active.focus,.btn.active:focus,.btn.focus,.btn:active.focus,.btn:active:focus,.btn:focus{outline:dotted thin;outline:-webkit-focus-ring-color auto 5px;outline-offset:-2px}.btn.focus,.btn:focus,.btn:hover{color:#333;text-decoration:none}.btn.active,.btn:active{outline:0;background-image:none;-webkit-box-shadow:inset 0 3px 5px rgba(0,0,0,.125);box-shadow:inset 0 3px 5px rgba(0,0,0,.125)}.btn.disabled,.btn[disabled],fieldset[disabled] .btn{cursor:not-allowed;pointer-events:none;opacity:.65;filter:alpha(opacity=65);-webkit-box-shadow:none;box-shadow:none}.btn-default{color:#333;background-color:#fff;border-color:#ccc}.btn-default.active,.btn-default.focus,.btn-default:active,.btn-default:focus,.btn-default:hover,.open>.dropdown-toggle.btn-default{color:#333;background-color:#e6e6e6;border-color:#adadad}.btn-default.active,.btn-default:active,.open>.dropdown-toggle.btn-default{background-image:none}.btn-default.disabled,.btn-default.disabled.active,.btn-default.disabled.focus,.btn-default.disabled:active,.btn-default.disabled:focus,.btn-default.disabled:hover,.btn-default[disabled],.btn-default[disabled].active,.btn-default[disabled].focus,.btn-default[disabled]:active,.btn-default[disabled]:focus,.btn-default[disabled]:hover,fieldset[disabled] .btn-default,fieldset[disabled] .btn-default.active,fieldset[disabled] .btn-default.focus,fieldset[disabled] .btn-default:active,fieldset[disabled] .btn-default:focus,fieldset[disabled] .btn-default:hover{background-color:#fff;border-color:#ccc}.btn-default .badge{color:#fff;background-color:#333}.btn-primary{color:#fff;background-color:#337ab7;border-color:#2e6da4}.btn-primary.active,.btn-primary.focus,.btn-primary:active,.btn-primary:focus,.btn-primary:hover,.open>.dropdown-toggle.btn-primary{color:#fff;background-color:#286090;border-color:#204d74}.btn-primary.active,.btn-primary:active,.open>.dropdown-toggle.btn-primary{background-image:none}.btn-primary.disabled,.btn-primary.disabled.active,.btn-primary.disabled.focus,.btn-primary.disabled:active,.btn-primary.disabled:focus,.btn-primary.disabled:hover,.btn-primary[disabled],.btn-primary[disabled].active,.btn-primary[disabled].focus,.btn-primary[disabled]:active,.btn-primary[disabled]:focus,.btn-primary[disabled]:hover,fieldset[disabled] .btn-primary,fieldset[disabled] .btn-primary.active,fieldset[disabled] .btn-primary.focus,fieldset[disabled] .btn-primary:active,fieldset[disabled] .btn-primary:focus,fieldset[disabled] .btn-primary:hover{background-color:#337ab7;border-color:#2e6da4}.btn-primary .badge{color:#337ab7;background-color:#fff}.btn-success{color:#fff;background-color:#5cb85c;border-color:#4cae4c}.btn-success.active,.btn-success.focus,.btn-success:active,.btn-success:focus,.btn-success:hover,.open>.dropdown-toggle.btn-success{color:#fff;background-color:#449d44;border-color:#398439}.btn-success.active,.btn-success:active,.open>.dropdown-toggle.btn-success{background-image:none}.btn-success.disabled,.btn-success.disabled.active,.btn-success.disabled.focus,.btn-success.disabled:active,.btn-success.disabled:focus,.btn-success.disabled:hover,.btn-success[disabled],.btn-success[disabled].active,.btn-success[disabled].focus,.btn-success[disabled]:active,.btn-success[disabled]:focus,.btn-success[disabled]:hover,fieldset[disabled] .btn-success,fieldset[disabled] .btn-success.active,fieldset[disabled] .btn-success.focus,fieldset[disabled] .btn-success:active,fieldset[disabled] .btn-success:focus,fieldset[disabled] .btn-success:hover{background-color:#5cb85c;border-color:#4cae4c}.btn-success .badge{color:#5cb85c;background-color:#fff}.btn-info{color:#fff;background-color:#5bc0de;border-color:#46b8da}.btn-info.active,.btn-info.focus,.btn-info:active,.btn-info:focus,.btn-info:hover,.open>.dropdown-toggle.btn-info{color:#fff;background-color:#31b0d5;border-color:#269abc}.btn-info.active,.btn-info:active,.open>.dropdown-toggle.btn-info{background-image:none}.btn-info.disabled,.btn-info.disabled.active,.btn-info.disabled.focus,.btn-info.disabled:active,.btn-info.disabled:focus,.btn-info.disabled:hover,.btn-info[disabled],.btn-info[disabled].active,.btn-info[disabled].focus,.btn-info[disabled]:active,.btn-info[disabled]:focus,.btn-info[disabled]:hover,fieldset[disabled] .btn-info,fieldset[disabled] .btn-info.active,fieldset[disabled] .btn-info.focus,fieldset[disabled] .btn-info:active,fieldset[disabled] .btn-info:focus,fieldset[disabled] .btn-info:hover{background-color:#5bc0de;border-color:#46b8da}.btn-info .badge{color:#5bc0de;background-color:#fff}.btn-warning{color:#fff;background-color:#f0ad4e;border-color:#eea236}.btn-warning.active,.btn-warning.focus,.btn-warning:active,.btn-warning:focus,.btn-warning:hover,.open>.dropdown-toggle.btn-warning{color:#fff;background-color:#ec971f;border-color:#d58512}.btn-warning.active,.btn-warning:active,.open>.dropdown-toggle.btn-warning{background-image:none}.btn-warning.disabled,.btn-warning.disabled.active,.btn-warning.disabled.focus,.btn-warning.disabled:active,.btn-warning.disabled:focus,.btn-warning.disabled:hover,.btn-warning[disabled],.btn-warning[disabled].active,.btn-warning[disabled].focus,.btn-warning[disabled]:active,.btn-warning[disabled]:focus,.btn-warning[disabled]:hover,fieldset[disabled] .btn-warning,fieldset[disabled] .btn-warning.active,fieldset[disabled] .btn-warning.focus,fieldset[disabled] .btn-warning:active,fieldset[disabled] .btn-warning:focus,fieldset[disabled] .btn-warning:hover{background-color:#f0ad4e;border-color:#eea236}.btn-warning .badge{color:#f0ad4e;background-color:#fff}.btn-danger{color:#fff;background-color:#d9534f;border-color:#d43f3a}.btn-danger.active,.btn-danger.focus,.btn-danger:active,.btn-danger:focus,.btn-danger:hover,.open>.dropdown-toggle.btn-danger{color:#fff;background-color:#c9302c;border-color:#ac2925}.btn-danger.active,.btn-danger:active,.open>.dropdown-toggle.btn-danger{background-image:none}.btn-danger.disabled,.btn-danger.disabled.active,.btn-danger.disabled.focus,.btn-danger.disabled:active,.btn-danger.disabled:focus,.btn-danger.disabled:hover,.btn-danger[disabled],.btn-danger[disabled].active,.btn-danger[disabled].focus,.btn-danger[disabled]:active,.btn-danger[disabled]:focus,.btn-danger[disabled]:hover,fieldset[disabled] .btn-danger,fieldset[disabled] .btn-danger.active,fieldset[disabled] .btn-danger.focus,fieldset[disabled] .btn-danger:active,fieldset[disabled] .btn-danger:focus,fieldset[disabled] .btn-danger:hover{background-color:#d9534f;border-color:#d43f3a}.btn-danger .badge{color:#d9534f;background-color:#fff}.btn-link{color:#337ab7;font-weight:400;border-radius:0}.btn-link,.btn-link.active,.btn-link:active,.btn-link[disabled],fieldset[disabled] .btn-link{background-color:transparent;-webkit-box-shadow:none;box-shadow:none}.btn-link,.btn-link:active,.btn-link:focus,.btn-link:hover{border-color:transparent}.btn-link:focus,.btn-link:hover{color:#23527c;text-decoration:underline;background-color:transparent}.btn-link[disabled]:focus,.btn-link[disabled]:hover,fieldset[disabled] .btn-link:focus,fieldset[disabled] .btn-link:hover{color:#777;text-decoration:none}.btn-group-lg>.btn,.btn-lg{padding:10px 16px;font-size:17px;line-height:1.3333333;border-radius:3px}.btn-group-sm>.btn,.btn-sm{padding:5px 10px;font-size:12px;line-height:1.5;border-radius:1px}.btn-group-xs>.btn,.btn-xs{padding:1px 5px;font-size:12px;line-height:1.5;border-radius:1px}.btn-block{display:block;width:100%}.btn-block+.btn-block{margin-top:5px}input[type=button].btn-block,input[type=reset].btn-block,input[type=submit].btn-block{width:100%}.fade{opacity:0;-webkit-transition:opacity .15s linear;-o-transition:opacity .15s linear;transition:opacity .15s linear}.fade.in{opacity:1}.collapse{display:none}.collapse.in{display:block}tr.collapse.in{display:table-row}tbody.collapse.in{display:table-row-group}.collapsing{position:relative;height:0;overflow:hidden;-webkit-transition-property:height,visibility;transition-property:height,visibility;-webkit-transition-duration:.35s;transition-duration:.35s;-webkit-transition-timing-function:ease;transition-timing-function:ease}.caret{display:inline-block;width:0;height:0;margin-left:2px;vertical-align:middle;border-top:4px dashed;border-right:4px solid transparent;border-left:4px solid transparent}.dropdown,.dropup{position:relative}.dropdown-toggle:focus{outline:0}.dropdown-menu{position:absolute;top:100%;left:0;z-index:1000;display:none;float:left;min-width:160px;padding:5px 0;margin:2px 0 0;list-style:none;font-size:13px;text-align:left;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,.15);border-radius:2px;-webkit-box-shadow:0 6px 12px rgba(0,0,0,.175);box-shadow:0 6px 12px rgba(0,0,0,.175);background-clip:padding-box}.dropdown-menu.pull-right{right:0;left:auto}.dropdown-menu .divider{height:1px;margin:8px 0;overflow:hidden;background-color:#e5e5e5}.dropdown-menu>li>a{display:block;padding:3px 20px;clear:both;font-weight:400;line-height:1.42857143;color:#333;white-space:nowrap}.dropdown-menu>li>a:focus,.dropdown-menu>li>a:hover{text-decoration:none;color:#262626;background-color:#f5f5f5}.dropdown-menu>.active>a,.dropdown-menu>.active>a:focus,.dropdown-menu>.active>a:hover{color:#fff;text-decoration:none;outline:0;background-color:#337ab7}.dropdown-menu>.disabled>a,.dropdown-menu>.disabled>a:focus,.dropdown-menu>.disabled>a:hover{color:#777}.dropdown-menu>.disabled>a:focus,.dropdown-menu>.disabled>a:hover{text-decoration:none;background-color:transparent;background-image:none;filter:progid:DXImageTransform.Microsoft.gradient(enabled=false);cursor:not-allowed}.open>.dropdown-menu{display:block}.open>a{outline:0}.dropdown-menu-right{left:auto;right:0}.dropdown-menu-left{left:0;right:auto}.dropdown-header{display:block;padding:3px 20px;font-size:12px;line-height:1.42857143;color:#777;white-space:nowrap}.dropdown-backdrop{position:fixed;left:0;right:0;bottom:0;top:0;z-index:990}.pull-right>.dropdown-menu{right:0;left:auto}.dropup .caret,.navbar-fixed-bottom .dropdown .caret{border-top:0;border-bottom:4px solid;content:""}.dropup .dropdown-menu,.navbar-fixed-bottom .dropdown .dropdown-menu{top:auto;bottom:100%;margin-bottom:2px}@media (min-width:541px){.navbar-right .dropdown-menu{left:auto;right:0}.navbar-right .dropdown-menu-left{left:0;right:auto}}.btn-group,.btn-group-vertical{position:relative;display:inline-block;vertical-align:middle}.btn-group-vertical>.btn,.btn-group>.btn{position:relative;float:left}.btn-group-vertical>.btn.active,.btn-group-vertical>.btn:active,.btn-group-vertical>.btn:focus,.btn-group-vertical>.btn:hover,.btn-group>.btn.active,.btn-group>.btn:active,.btn-group>.btn:focus,.btn-group>.btn:hover{z-index:2}.btn-group .btn+.btn,.btn-group .btn+.btn-group,.btn-group .btn-group+.btn,.btn-group .btn-group+.btn-group{margin-left:-1px}.btn-toolbar{margin-left:-5px}.btn-toolbar .btn-group,.btn-toolbar .input-group{float:left}.btn-toolbar>.btn,.btn-toolbar>.btn-group,.btn-toolbar>.input-group{margin-left:5px}.btn-group>.btn:not(:first-child):not(:last-child):not(.dropdown-toggle){border-radius:0}.btn-group>.btn:first-child{margin-left:0}.btn-group>.btn:first-child:not(:last-child):not(.dropdown-toggle){border-bottom-right-radius:0;border-top-right-radius:0}.btn-group>.btn:last-child:not(:first-child),.btn-group>.dropdown-toggle:not(:first-child){border-bottom-left-radius:0;border-top-left-radius:0}.btn-group>.btn-group{float:left}.btn-group>.btn-group:not(:first-child):not(:last-child)>.btn{border-radius:0}.btn-group>.btn-group:first-child:not(:last-child)>.btn:last-child,.btn-group>.btn-group:first-child:not(:last-child)>.dropdown-toggle{border-bottom-right-radius:0;border-top-right-radius:0}.btn-group>.btn-group:last-child:not(:first-child)>.btn:first-child{border-bottom-left-radius:0;border-top-left-radius:0}.btn-group .dropdown-toggle:active,.btn-group.open .dropdown-toggle{outline:0}.btn-group>.btn+.dropdown-toggle{padding-left:8px;padding-right:8px}.btn-group>.btn-lg+.dropdown-toggle{padding-left:12px;padding-right:12px}.btn-group.open .dropdown-toggle{-webkit-box-shadow:inset 0 3px 5px rgba(0,0,0,.125);box-shadow:inset 0 3px 5px rgba(0,0,0,.125)}.btn-group.open .dropdown-toggle.btn-link{-webkit-box-shadow:none;box-shadow:none}.btn .caret{margin-left:0}.btn-lg .caret{border-width:5px 5px 0}.dropup .btn-lg .caret{border-width:0 5px 5px}.btn-group-vertical>.btn,.btn-group-vertical>.btn-group,.btn-group-vertical>.btn-group>.btn{display:block;float:none;width:100%;max-width:100%}.btn-group-vertical>.btn-group>.btn{float:none}.btn-group-vertical>.btn+.btn,.btn-group-vertical>.btn+.btn-group,.btn-group-vertical>.btn-group+.btn,.btn-group-vertical>.btn-group+.btn-group{margin-top:-1px;margin-left:0}.btn-group-vertical>.btn:not(:first-child):not(:last-child){border-radius:0}.btn-group-vertical>.btn:first-child:not(:last-child){border-top-right-radius:2px;border-bottom-right-radius:0;border-bottom-left-radius:0}.btn-group-vertical>.btn:last-child:not(:first-child){border-bottom-left-radius:2px;border-top-right-radius:0;border-top-left-radius:0}.btn-group-vertical>.btn-group:not(:first-child):not(:last-child)>.btn{border-radius:0}.btn-group-vertical>.btn-group:first-child:not(:last-child)>.btn:last-child,.btn-group-vertical>.btn-group:first-child:not(:last-child)>.dropdown-toggle{border-bottom-right-radius:0;border-bottom-left-radius:0}.btn-group-vertical>.btn-group:last-child:not(:first-child)>.btn:first-child{border-top-right-radius:0;border-top-left-radius:0}.btn-group-justified{display:table;width:100%;table-layout:fixed;border-collapse:separate}.btn-group-justified>.btn,.btn-group-justified>.btn-group{float:none;display:table-cell;width:1%}.btn-group-justified>.btn-group .btn{width:100%}.btn-group-justified>.btn-group .dropdown-menu{left:auto}[data-toggle=buttons]>.btn input[type=checkbox],[data-toggle=buttons]>.btn input[type=radio],[data-toggle=buttons]>.btn-group>.btn input[type=checkbox],[data-toggle=buttons]>.btn-group>.btn input[type=radio]{position:absolute;clip:rect(0,0,0,0);pointer-events:none}.input-group{position:relative;display:table;border-collapse:separate}.input-group[class*=col-]{float:none;padding-left:0;padding-right:0}.input-group .form-control{position:relative;z-index:2;float:left;width:100%;margin-bottom:0}.input-group-lg>.form-control,.input-group-lg>.input-group-addon,.input-group-lg>.input-group-btn>.btn{height:45px;padding:10px 16px;font-size:17px;line-height:1.3333333;border-radius:3px}select.input-group-lg>.form-control,select.input-group-lg>.input-group-addon,select.input-group-lg>.input-group-btn>.btn{height:45px;line-height:45px}select[multiple].input-group-lg>.form-control,select[multiple].input-group-lg>.input-group-addon,select[multiple].input-group-lg>.input-group-btn>.btn,textarea.input-group-lg>.form-control,textarea.input-group-lg>.input-group-addon,textarea.input-group-lg>.input-group-btn>.btn{height:auto}.input-group-sm>.form-control,.input-group-sm>.input-group-addon,.input-group-sm>.input-group-btn>.btn{height:30px;padding:5px 10px;font-size:12px;line-height:1.5;border-radius:1px}select.input-group-sm>.form-control,select.input-group-sm>.input-group-addon,select.input-group-sm>.input-group-btn>.btn{height:30px;line-height:30px}select[multiple].input-group-sm>.form-control,select[multiple].input-group-sm>.input-group-addon,select[multiple].input-group-sm>.input-group-btn>.btn,textarea.input-group-sm>.form-control,textarea.input-group-sm>.input-group-addon,textarea.input-group-sm>.input-group-btn>.btn{height:auto}.input-group .form-control,.input-group-addon,.input-group-btn{display:table-cell}.input-group .form-control:not(:first-child):not(:last-child),.input-group-addon:not(:first-child):not(:last-child),.input-group-btn:not(:first-child):not(:last-child){border-radius:0}.input-group-addon,.input-group-btn{width:1%;white-space:nowrap;vertical-align:middle}.input-group-addon{padding:6px 12px;font-size:13px;font-weight:400;line-height:1;color:#555;text-align:center;background-color:#eee;border:1px solid #ccc;border-radius:2px}.input-group-addon.input-sm{padding:5px 10px;font-size:12px;border-radius:1px}.input-group-addon.input-lg{padding:10px 16px;font-size:17px;border-radius:3px}.input-group-addon input[type=checkbox],.input-group-addon input[type=radio]{margin-top:0}.input-group .form-control:first-child,.input-group-addon:first-child,.input-group-btn:first-child>.btn,.input-group-btn:first-child>.btn-group>.btn,.input-group-btn:first-child>.dropdown-toggle,.input-group-btn:last-child>.btn-group:not(:last-child)>.btn,.input-group-btn:last-child>.btn:not(:last-child):not(.dropdown-toggle){border-bottom-right-radius:0;border-top-right-radius:0}.input-group-addon:first-child{border-right:0}.input-group .form-control:last-child,.input-group-addon:last-child,.input-group-btn:first-child>.btn-group:not(:first-child)>.btn,.input-group-btn:first-child>.btn:not(:first-child),.input-group-btn:last-child>.btn,.input-group-btn:last-child>.btn-group>.btn,.input-group-btn:last-child>.dropdown-toggle{border-bottom-left-radius:0;border-top-left-radius:0}.input-group-addon:last-child{border-left:0}.input-group-btn{position:relative;font-size:0;white-space:nowrap}.input-group-btn>.btn{position:relative}.input-group-btn>.btn+.btn{margin-left:-1px}.input-group-btn>.btn:active,.input-group-btn>.btn:focus,.input-group-btn>.btn:hover{z-index:2}.input-group-btn:first-child>.btn,.input-group-btn:first-child>.btn-group{margin-right:-1px}.input-group-btn:last-child>.btn,.input-group-btn:last-child>.btn-group{margin-left:-1px}.nav{margin-bottom:0;padding-left:0;list-style:none}.nav>li{position:relative;display:block}.nav>li>a{position:relative;display:block;padding:10px 15px}.nav>li>a:focus,.nav>li>a:hover{text-decoration:none;background-color:#eee}.nav>li.disabled>a{color:#777}.nav>li.disabled>a:focus,.nav>li.disabled>a:hover{color:#777;text-decoration:none;background-color:transparent;cursor:not-allowed}.nav .open>a,.nav .open>a:focus,.nav .open>a:hover{background-color:#eee;border-color:#337ab7}.nav .nav-divider{height:1px;margin:8px 0;overflow:hidden;background-color:#e5e5e5}.nav>li>a>img{max-width:none}.nav-tabs{border-bottom:1px solid #ddd}.nav-tabs>li{float:left;margin-bottom:-1px}.nav-tabs>li>a{margin-right:2px;line-height:1.42857143;border:1px solid transparent;border-radius:2px 2px 0 0}.nav-tabs>li>a:hover{border-color:#eee #eee #ddd}.nav-tabs>li.active>a,.nav-tabs>li.active>a:focus,.nav-tabs>li.active>a:hover{color:#555;background-color:#fff;border:1px solid #ddd;border-bottom-color:transparent;cursor:default}.nav-tabs.nav-justified{width:100%;border-bottom:0}.nav-tabs.nav-justified>li{float:none}.nav-tabs.nav-justified>li>a{text-align:center;margin-bottom:5px;margin-right:0;border-radius:2px}.nav-tabs.nav-justified>.dropdown .dropdown-menu{top:auto;left:auto}.nav-tabs.nav-justified>.active>a,.nav-tabs.nav-justified>.active>a:focus,.nav-tabs.nav-justified>.active>a:hover{border:1px solid #ddd}@media (min-width:768px){.nav-tabs.nav-justified>li{display:table-cell;width:1%}.nav-tabs.nav-justified>li>a{margin-bottom:0;border-bottom:1px solid #ddd;border-radius:2px 2px 0 0}.nav-tabs.nav-justified>.active>a,.nav-tabs.nav-justified>.active>a:focus,.nav-tabs.nav-justified>.active>a:hover{border-bottom-color:#fff}}.nav-pills>li{float:left}.nav-pills>li>a{border-radius:2px}.nav-pills>li+li{margin-left:2px}.nav-pills>li.active>a,.nav-pills>li.active>a:focus,.nav-pills>li.active>a:hover{color:#fff;background-color:#337ab7}.nav-stacked>li{float:none}.nav-stacked>li+li{margin-top:2px;margin-left:0}.nav-justified{width:100%}.nav-justified>li{float:none}.nav-justified>li>a{text-align:center;margin-bottom:5px}.nav-justified>.dropdown .dropdown-menu{top:auto;left:auto}.nav-tabs-justified{border-bottom:0}.nav-tabs-justified>li>a{margin-right:0;border-radius:2px}.nav-tabs-justified>.active>a,.nav-tabs-justified>.active>a:focus,.nav-tabs-justified>.active>a:hover{border:1px solid #ddd}@media (min-width:768px){.nav-justified>li{display:table-cell;width:1%}.nav-justified>li>a{margin-bottom:0}.nav-tabs-justified>li>a{border-bottom:1px solid #ddd;border-radius:2px 2px 0 0}.nav-tabs-justified>.active>a,.nav-tabs-justified>.active>a:focus,.nav-tabs-justified>.active>a:hover{border-bottom-color:#fff}}.tab-content>.tab-pane{display:none}.tab-content>.active{display:block}.nav-tabs .dropdown-menu{margin-top:-1px;border-top-right-radius:0;border-top-left-radius:0}.navbar{position:relative;min-height:30px;margin-bottom:18px;border:1px solid transparent}.navbar-collapse{overflow-x:visible;padding-right:0;padding-left:0;border-top:1px solid transparent;box-shadow:inset 0 1px 0 rgba(255,255,255,.1);-webkit-overflow-scrolling:touch}.navbar-collapse.in{overflow-y:auto}.navbar-fixed-bottom .navbar-collapse,.navbar-fixed-top .navbar-collapse{max-height:340px}@media (max-device-width:540px)and (orientation:landscape){.navbar-fixed-bottom .navbar-collapse,.navbar-fixed-top .navbar-collapse{max-height:200px}}.container-fluid>.navbar-collapse,.container-fluid>.navbar-header,.container>.navbar-collapse,.container>.navbar-header{margin-right:0;margin-left:0}.navbar-static-top{z-index:1000;border-width:0 0 1px}.navbar-fixed-bottom,.navbar-fixed-top{position:fixed;right:0;left:0;z-index:1030}@media (min-width:541px){.navbar{border-radius:2px}.navbar-header{float:left}.navbar-collapse{width:auto;border-top:0;box-shadow:none}.navbar-collapse.collapse{display:block!important;height:auto!important;padding-bottom:0;overflow:visible!important}.navbar-collapse.in{overflow-y:visible}.navbar-fixed-bottom .navbar-collapse,.navbar-fixed-top .navbar-collapse,.navbar-static-top .navbar-collapse{padding-left:0;padding-right:0}.container-fluid>.navbar-collapse,.container-fluid>.navbar-header,.container>.navbar-collapse,.container>.navbar-header{margin-right:0;margin-left:0}.navbar-fixed-bottom,.navbar-fixed-top,.navbar-static-top{border-radius:0}}.navbar-fixed-top{top:0;border-width:0 0 1px}.navbar-fixed-bottom{bottom:0;margin-bottom:0;border-width:1px 0 0}.navbar-brand{float:left;padding:6px 0;font-size:17px;line-height:18px;height:30px}.navbar-brand:focus,.navbar-brand:hover{text-decoration:none}.navbar-brand>img{display:block}.navbar-toggle{position:relative;float:right;margin-right:0;padding:9px 10px;margin-top:-2px;margin-bottom:-2px;background-color:transparent;background-image:none;border:1px solid transparent;border-radius:2px}.navbar-toggle:focus{outline:0}.navbar-toggle .icon-bar{display:block;width:22px;height:2px;border-radius:1px}.navbar-toggle .icon-bar+.icon-bar{margin-top:4px}@media (min-width:541px){.navbar>.container .navbar-brand,.navbar>.container-fluid .navbar-brand{margin-left:0}.navbar-toggle{display:none}}.navbar-nav{margin:3px 0}.navbar-nav>li>a{padding-top:10px;padding-bottom:10px;line-height:18px}@media (max-width:540px){.navbar-nav .open .dropdown-menu{position:static;float:none;width:auto;margin-top:0;background-color:transparent;border:0;box-shadow:none}.navbar-nav .open .dropdown-menu .dropdown-header,.navbar-nav .open .dropdown-menu>li>a{padding:5px 15px 5px 25px}.navbar-nav .open .dropdown-menu>li>a{line-height:18px}.navbar-nav .open .dropdown-menu>li>a:focus,.navbar-nav .open .dropdown-menu>li>a:hover{background-image:none}}@media (min-width:541px){.navbar-nav{float:left;margin:0}.navbar-nav>li{float:left}.navbar-nav>li>a{padding-top:6px;padding-bottom:6px}}.navbar-form{padding:10px 0;border-top:1px solid transparent;border-bottom:1px solid transparent;-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,.1),0 1px 0 rgba(255,255,255,.1);box-shadow:inset 0 1px 0 rgba(255,255,255,.1),0 1px 0 rgba(255,255,255,.1);margin:-1px 0}@media (min-width:768px){.navbar-form .form-group{display:inline-block;margin-bottom:0;vertical-align:middle}.navbar-form .form-control{display:inline-block;width:auto;vertical-align:middle}.navbar-form .form-control-static{display:inline-block}.navbar-form .input-group{display:inline-table;vertical-align:middle}.navbar-form .input-group .form-control,.navbar-form .input-group .input-group-addon,.navbar-form .input-group .input-group-btn{width:auto}.navbar-form .input-group>.form-control{width:100%}.navbar-form .control-label{margin-bottom:0;vertical-align:middle}.navbar-form .checkbox,.navbar-form .radio{display:inline-block;margin-top:0;margin-bottom:0;vertical-align:middle}.navbar-form .checkbox label,.navbar-form .radio label{padding-left:0}.navbar-form .checkbox input[type=checkbox],.navbar-form .radio input[type=radio]{position:relative;margin-left:0}.navbar-form .has-feedback .form-control-feedback{top:0}}@media (max-width:540px){.navbar-form .form-group{margin-bottom:5px}.navbar-form .form-group:last-child{margin-bottom:0}}.navbar-nav>li>.dropdown-menu{margin-top:0;border-top-right-radius:0;border-top-left-radius:0}.navbar-fixed-bottom .navbar-nav>li>.dropdown-menu{margin-bottom:0;border-radius:2px 2px 0 0}.navbar-btn{margin-top:-1px;margin-bottom:-1px}.navbar-btn.btn-sm{margin-top:0;margin-bottom:0}.navbar-btn.btn-xs{margin-top:4px;margin-bottom:4px}.navbar-text{margin-top:6px;margin-bottom:6px}@media (min-width:541px){.navbar-form{width:auto;border:0;margin-left:0;margin-right:0;padding-top:0;padding-bottom:0;-webkit-box-shadow:none;box-shadow:none}.navbar-text{float:left;margin-left:0;margin-right:0}.navbar-left{float:left!important;float:left}.navbar-right{float:right!important;float:right;margin-right:0}.navbar-right~.navbar-right{margin-right:0}}.navbar-default{background-color:#f8f8f8;border-color:#e7e7e7}.navbar-default .navbar-brand{color:#777}.navbar-default .navbar-brand:focus,.navbar-default .navbar-brand:hover{color:#5e5e5e;background-color:transparent}.navbar-default .navbar-nav>li>a,.navbar-default .navbar-text{color:#777}.navbar-default .navbar-nav>li>a:focus,.navbar-default .navbar-nav>li>a:hover{color:#333;background-color:transparent}.navbar-default .navbar-nav>.active>a,.navbar-default .navbar-nav>.active>a:focus,.navbar-default .navbar-nav>.active>a:hover{color:#555;background-color:#e7e7e7}.navbar-default .navbar-nav>.disabled>a,.navbar-default .navbar-nav>.disabled>a:focus,.navbar-default .navbar-nav>.disabled>a:hover{color:#ccc;background-color:transparent}.navbar-default .navbar-toggle{border-color:#ddd}.navbar-default .navbar-toggle:focus,.navbar-default .navbar-toggle:hover{background-color:#ddd}.navbar-default .navbar-toggle .icon-bar{background-color:#888}.navbar-default .navbar-collapse,.navbar-default .navbar-form{border-color:#e7e7e7}.navbar-default .navbar-nav>.open>a,.navbar-default .navbar-nav>.open>a:focus,.navbar-default .navbar-nav>.open>a:hover{background-color:#e7e7e7;color:#555}@media (max-width:540px){.navbar-default .navbar-nav .open .dropdown-menu>li>a{color:#777}.navbar-default .navbar-nav .open .dropdown-menu>li>a:focus,.navbar-default .navbar-nav .open .dropdown-menu>li>a:hover{color:#333;background-color:transparent}.navbar-default .navbar-nav .open .dropdown-menu>.active>a,.navbar-default .navbar-nav .open .dropdown-menu>.active>a:focus,.navbar-default .navbar-nav .open .dropdown-menu>.active>a:hover{color:#555;background-color:#e7e7e7}.navbar-default .navbar-nav .open .dropdown-menu>.disabled>a,.navbar-default .navbar-nav .open .dropdown-menu>.disabled>a:focus,.navbar-default .navbar-nav .open .dropdown-menu>.disabled>a:hover{color:#ccc;background-color:transparent}}.navbar-default .navbar-link{color:#777}.navbar-default .navbar-link:hover{color:#333}.navbar-default .btn-link{color:#777}.navbar-default .btn-link:focus,.navbar-default .btn-link:hover{color:#333}.navbar-default .btn-link[disabled]:focus,.navbar-default .btn-link[disabled]:hover,fieldset[disabled] .navbar-default .btn-link:focus,fieldset[disabled] .navbar-default .btn-link:hover{color:#ccc}.navbar-inverse{background-color:#222;border-color:#080808}.navbar-inverse .navbar-brand{color:#9d9d9d}.navbar-inverse .navbar-brand:focus,.navbar-inverse .navbar-brand:hover{color:#fff;background-color:transparent}.navbar-inverse .navbar-nav>li>a,.navbar-inverse .navbar-text{color:#9d9d9d}.navbar-inverse .navbar-nav>li>a:focus,.navbar-inverse .navbar-nav>li>a:hover{color:#fff;background-color:transparent}.navbar-inverse .navbar-nav>.active>a,.navbar-inverse .navbar-nav>.active>a:focus,.navbar-inverse .navbar-nav>.active>a:hover{color:#fff;background-color:#080808}.navbar-inverse .navbar-nav>.disabled>a,.navbar-inverse .navbar-nav>.disabled>a:focus,.navbar-inverse .navbar-nav>.disabled>a:hover{color:#444;background-color:transparent}.navbar-inverse .navbar-toggle{border-color:#333}.navbar-inverse .navbar-toggle:focus,.navbar-inverse .navbar-toggle:hover{background-color:#333}.navbar-inverse .navbar-toggle .icon-bar{background-color:#fff}.navbar-inverse .navbar-collapse,.navbar-inverse .navbar-form{border-color:#101010}.navbar-inverse .navbar-nav>.open>a,.navbar-inverse .navbar-nav>.open>a:focus,.navbar-inverse .navbar-nav>.open>a:hover{background-color:#080808;color:#fff}@media (max-width:540px){.navbar-inverse .navbar-nav .open .dropdown-menu>.dropdown-header{border-color:#080808}.navbar-inverse .navbar-nav .open .dropdown-menu .divider{background-color:#080808}.navbar-inverse .navbar-nav .open .dropdown-menu>li>a{color:#9d9d9d}.navbar-inverse .navbar-nav .open .dropdown-menu>li>a:focus,.navbar-inverse .navbar-nav .open .dropdown-menu>li>a:hover{color:#fff;background-color:transparent}.navbar-inverse .navbar-nav .open .dropdown-menu>.active>a,.navbar-inverse .navbar-nav .open .dropdown-menu>.active>a:focus,.navbar-inverse .navbar-nav .open .dropdown-menu>.active>a:hover{color:#fff;background-color:#080808}.navbar-inverse .navbar-nav .open .dropdown-menu>.disabled>a,.navbar-inverse .navbar-nav .open .dropdown-menu>.disabled>a:focus,.navbar-inverse .navbar-nav .open .dropdown-menu>.disabled>a:hover{color:#444;background-color:transparent}}.navbar-inverse .navbar-link{color:#9d9d9d}.navbar-inverse .navbar-link:hover{color:#fff}.navbar-inverse .btn-link{color:#9d9d9d}.navbar-inverse .btn-link:focus,.navbar-inverse .btn-link:hover{color:#fff}.navbar-inverse .btn-link[disabled]:focus,.navbar-inverse .btn-link[disabled]:hover,fieldset[disabled] .navbar-inverse .btn-link:focus,fieldset[disabled] .navbar-inverse .btn-link:hover{color:#444}.breadcrumb{padding:8px 15px;margin-bottom:18px;list-style:none;background-color:#f5f5f5;border-radius:2px}.breadcrumb>li{display:inline-block}.breadcrumb>li+li:before{content:"/\00a0";padding:0 5px;color:#5e5e5e}.breadcrumb>.active{color:#777}.pagination{display:inline-block;padding-left:0;margin:18px 0;border-radius:2px}.pagination>li{display:inline}.pagination>li>a,.pagination>li>span{position:relative;float:left;padding:6px 12px;line-height:1.42857143;text-decoration:none;color:#337ab7;background-color:#fff;border:1px solid #ddd;margin-left:-1px}.pagination>li:first-child>a,.pagination>li:first-child>span{margin-left:0;border-bottom-left-radius:2px;border-top-left-radius:2px}.pagination>li:last-child>a,.pagination>li:last-child>span{border-bottom-right-radius:2px;border-top-right-radius:2px}.pagination>li>a:focus,.pagination>li>a:hover,.pagination>li>span:focus,.pagination>li>span:hover{color:#23527c;background-color:#eee;border-color:#ddd}.pagination>.active>a,.pagination>.active>a:focus,.pagination>.active>a:hover,.pagination>.active>span,.pagination>.active>span:focus,.pagination>.active>span:hover{z-index:2;color:#fff;background-color:#337ab7;border-color:#337ab7;cursor:default}.pagination>.disabled>a,.pagination>.disabled>a:focus,.pagination>.disabled>a:hover,.pagination>.disabled>span,.pagination>.disabled>span:focus,.pagination>.disabled>span:hover{color:#777;background-color:#fff;border-color:#ddd;cursor:not-allowed}.pagination-lg>li>a,.pagination-lg>li>span{padding:10px 16px;font-size:17px}.pagination-lg>li:first-child>a,.pagination-lg>li:first-child>span{border-bottom-left-radius:3px;border-top-left-radius:3px}.pagination-lg>li:last-child>a,.pagination-lg>li:last-child>span{border-bottom-right-radius:3px;border-top-right-radius:3px}.pagination-sm>li>a,.pagination-sm>li>span{padding:5px 10px;font-size:12px}.pagination-sm>li:first-child>a,.pagination-sm>li:first-child>span{border-bottom-left-radius:1px;border-top-left-radius:1px}.pagination-sm>li:last-child>a,.pagination-sm>li:last-child>span{border-bottom-right-radius:1px;border-top-right-radius:1px}.pager{padding-left:0;margin:18px 0;list-style:none;text-align:center}.pager li{display:inline}.pager li>a,.pager li>span{display:inline-block;padding:5px 14px;background-color:#fff;border:1px solid #ddd;border-radius:15px}.pager li>a:focus,.pager li>a:hover{text-decoration:none;background-color:#eee}.pager .next>a,.pager .next>span{float:right}.pager .previous>a,.pager .previous>span{float:left}.pager .disabled>a,.pager .disabled>a:focus,.pager .disabled>a:hover,.pager .disabled>span{color:#777;background-color:#fff;cursor:not-allowed}.label{display:inline;padding:.2em .6em .3em;font-size:75%;font-weight:700;line-height:1;color:#fff;text-align:center;white-space:nowrap;vertical-align:baseline;border-radius:.25em}a.label:focus,a.label:hover{color:#fff;text-decoration:none;cursor:pointer}.label:empty{display:none}.btn .label{position:relative;top:-1px}.label-default{background-color:#777}.label-default[href]:focus,.label-default[href]:hover{background-color:#5e5e5e}.label-primary{background-color:#337ab7}.label-primary[href]:focus,.label-primary[href]:hover{background-color:#286090}.label-success{background-color:#5cb85c}.label-success[href]:focus,.label-success[href]:hover{background-color:#449d44}.label-info{background-color:#5bc0de}.label-info[href]:focus,.label-info[href]:hover{background-color:#31b0d5}.label-warning{background-color:#f0ad4e}.label-warning[href]:focus,.label-warning[href]:hover{background-color:#ec971f}.label-danger{background-color:#d9534f}.label-danger[href]:focus,.label-danger[href]:hover{background-color:#c9302c}.badge{display:inline-block;min-width:10px;padding:3px 7px;font-size:12px;font-weight:700;color:#fff;line-height:1;vertical-align:baseline;white-space:nowrap;text-align:center;background-color:#777;border-radius:10px}.badge:empty{display:none}.btn .badge{position:relative;top:-1px}.btn-group-xs>.btn .badge,.btn-xs .badge{top:0;padding:1px 5px}a.badge:focus,a.badge:hover{color:#fff;text-decoration:none;cursor:pointer}.list-group-item.active>.badge,.nav-pills>.active>a>.badge{color:#337ab7;background-color:#fff}.list-group-item>.badge{float:right}.list-group-item>.badge+.badge{margin-right:5px}.nav-pills>li>a>.badge{margin-left:3px}.jumbotron{padding:30px 15px;margin-bottom:30px;color:inherit;background-color:#eee}.jumbotron .h1,.jumbotron h1{color:inherit}.jumbotron p{margin-bottom:15px;font-size:20px;font-weight:200}.jumbotron>hr{border-top-color:#d5d5d5}.container .jumbotron,.container-fluid .jumbotron{border-radius:3px}.jumbotron .container{max-width:100%}@media screen and (min-width:768px){.jumbotron{padding:48px 0}.container .jumbotron,.container-fluid .jumbotron{padding-left:60px;padding-right:60px}.jumbotron .h1,.jumbotron h1{font-size:58.5px}}.thumbnail{display:block;padding:4px;margin-bottom:18px;line-height:1.42857143;background-color:#fff;border:1px solid #ddd;border-radius:2px;-webkit-transition:border .2s ease-in-out;-o-transition:border .2s ease-in-out;transition:border .2s ease-in-out}.thumbnail a>img,.thumbnail>img{margin-left:auto;margin-right:auto}a.thumbnail.active,a.thumbnail:focus,a.thumbnail:hover{border-color:#337ab7}.thumbnail .caption{padding:9px;color:#000}.alert{padding:15px;margin-bottom:18px;border:1px solid transparent;border-radius:2px}.alert h4{margin-top:0;color:inherit}.alert .alert-link{font-weight:700}.alert>p,.alert>ul{margin-bottom:0}.alert>p+p{margin-top:5px}.alert-dismissable,.alert-dismissible{padding-right:35px}.alert-dismissable .close,.alert-dismissible .close{position:relative;top:-2px;right:-21px;color:inherit}.alert-success{background-color:#dff0d8;border-color:#d6e9c6;color:#3c763d}.alert-success hr{border-top-color:#c9e2b3}.alert-success .alert-link{color:#2b542c}.alert-info{background-color:#d9edf7;border-color:#bce8f1;color:#31708f}.alert-info hr{border-top-color:#a6e1ec}.alert-info .alert-link{color:#245269}.alert-warning{background-color:#fcf8e3;border-color:#faebcc;color:#8a6d3b}.alert-warning hr{border-top-color:#f7e1b5}.alert-warning .alert-link{color:#66512c}.alert-danger{background-color:#f2dede;border-color:#ebccd1;color:#a94442}.alert-danger hr{border-top-color:#e4b9c0}.alert-danger .alert-link{color:#843534}@-webkit-keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}@keyframes progress-bar-stripes{from{background-position:40px 0}to{background-position:0 0}}.progress{overflow:hidden;height:18px;margin-bottom:18px;background-color:#f5f5f5;border-radius:2px;-webkit-box-shadow:inset 0 1px 2px rgba(0,0,0,.1);box-shadow:inset 0 1px 2px rgba(0,0,0,.1)}.progress-bar{float:left;width:0;height:100%;font-size:12px;line-height:18px;color:#fff;text-align:center;background-color:#337ab7;-webkit-box-shadow:inset 0 -1px 0 rgba(0,0,0,.15);box-shadow:inset 0 -1px 0 rgba(0,0,0,.15);-webkit-transition:width .6s ease;-o-transition:width .6s ease;transition:width .6s ease}.progress-bar-striped,.progress-striped .progress-bar{background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-size:40px 40px}.progress-bar.active,.progress.active .progress-bar{-webkit-animation:progress-bar-stripes 2s linear infinite;-o-animation:progress-bar-stripes 2s linear infinite;animation:progress-bar-stripes 2s linear infinite}.progress-bar-success{background-color:#5cb85c}.progress-striped .progress-bar-success{background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent)}.progress-bar-info{background-color:#5bc0de}.progress-striped .progress-bar-info{background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent)}.progress-bar-warning{background-color:#f0ad4e}.progress-striped .progress-bar-warning{background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent)}.progress-bar-danger{background-color:#d9534f}.progress-striped .progress-bar-danger{background-image:-webkit-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:-o-linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent);background-image:linear-gradient(45deg,rgba(255,255,255,.15) 25%,transparent 25%,transparent 50%,rgba(255,255,255,.15) 50%,rgba(255,255,255,.15) 75%,transparent 75%,transparent)}.media{margin-top:15px}.media:first-child{margin-top:0}.media,.media-body{zoom:1;overflow:hidden}.media-body{width:10000px}.media-object{display:block}.media-right,.media>.pull-right{padding-left:10px}.media-left,.media>.pull-left{padding-right:10px}.media-body,.media-left,.media-right{display:table-cell;vertical-align:top}.media-middle{vertical-align:middle}.media-bottom{vertical-align:bottom}.media-heading{margin-top:0;margin-bottom:5px}.media-list{padding-left:0;list-style:none}.list-group{margin-bottom:20px;padding-left:0}.list-group-item{position:relative;display:block;padding:10px 15px;margin-bottom:-1px;background-color:#fff;border:1px solid #ddd}.list-group-item:first-child{border-top-right-radius:2px;border-top-left-radius:2px}.list-group-item:last-child{margin-bottom:0;border-bottom-right-radius:2px;border-bottom-left-radius:2px}a.list-group-item{color:#555}a.list-group-item .list-group-item-heading{color:#333}a.list-group-item:focus,a.list-group-item:hover{text-decoration:none;color:#555;background-color:#f5f5f5}.list-group-item.disabled,.list-group-item.disabled:focus,.list-group-item.disabled:hover{background-color:#eee;color:#777;cursor:not-allowed}.list-group-item.disabled .list-group-item-heading,.list-group-item.disabled:focus .list-group-item-heading,.list-group-item.disabled:hover .list-group-item-heading{color:inherit}.list-group-item.disabled .list-group-item-text,.list-group-item.disabled:focus .list-group-item-text,.list-group-item.disabled:hover .list-group-item-text{color:#777}.list-group-item.active,.list-group-item.active:focus,.list-group-item.active:hover{z-index:2;color:#fff;background-color:#337ab7;border-color:#337ab7}.list-group-item.active .list-group-item-heading,.list-group-item.active .list-group-item-heading>.small,.list-group-item.active .list-group-item-heading>small,.list-group-item.active:focus .list-group-item-heading,.list-group-item.active:focus .list-group-item-heading>.small,.list-group-item.active:focus .list-group-item-heading>small,.list-group-item.active:hover .list-group-item-heading,.list-group-item.active:hover .list-group-item-heading>.small,.list-group-item.active:hover .list-group-item-heading>small{color:inherit}.list-group-item.active .list-group-item-text,.list-group-item.active:focus .list-group-item-text,.list-group-item.active:hover .list-group-item-text{color:#c7ddef}.list-group-item-success{color:#3c763d;background-color:#dff0d8}a.list-group-item-success{color:#3c763d}a.list-group-item-success .list-group-item-heading{color:inherit}a.list-group-item-success:focus,a.list-group-item-success:hover{color:#3c763d;background-color:#d0e9c6}a.list-group-item-success.active,a.list-group-item-success.active:focus,a.list-group-item-success.active:hover{color:#fff;background-color:#3c763d;border-color:#3c763d}.list-group-item-info{color:#31708f;background-color:#d9edf7}a.list-group-item-info{color:#31708f}a.list-group-item-info .list-group-item-heading{color:inherit}a.list-group-item-info:focus,a.list-group-item-info:hover{color:#31708f;background-color:#c4e3f3}a.list-group-item-info.active,a.list-group-item-info.active:focus,a.list-group-item-info.active:hover{color:#fff;background-color:#31708f;border-color:#31708f}.list-group-item-warning{color:#8a6d3b;background-color:#fcf8e3}a.list-group-item-warning{color:#8a6d3b}a.list-group-item-warning .list-group-item-heading{color:inherit}a.list-group-item-warning:focus,a.list-group-item-warning:hover{color:#8a6d3b;background-color:#faf2cc}a.list-group-item-warning.active,a.list-group-item-warning.active:focus,a.list-group-item-warning.active:hover{color:#fff;background-color:#8a6d3b;border-color:#8a6d3b}.list-group-item-danger{color:#a94442;background-color:#f2dede}a.list-group-item-danger{color:#a94442}a.list-group-item-danger .list-group-item-heading{color:inherit}a.list-group-item-danger:focus,a.list-group-item-danger:hover{color:#a94442;background-color:#ebcccc}a.list-group-item-danger.active,a.list-group-item-danger.active:focus,a.list-group-item-danger.active:hover{color:#fff;background-color:#a94442;border-color:#a94442}.list-group-item-heading{margin-top:0;margin-bottom:5px}.list-group-item-text{margin-bottom:0;line-height:1.3}.panel{margin-bottom:18px;background-color:#fff;border:1px solid transparent;border-radius:2px;-webkit-box-shadow:0 1px 1px rgba(0,0,0,.05);box-shadow:0 1px 1px rgba(0,0,0,.05)}.panel-body{padding:15px}.panel-heading{padding:10px 15px;border-bottom:1px solid transparent;border-top-right-radius:1px;border-top-left-radius:1px}.panel-heading>.dropdown .dropdown-toggle{color:inherit}.panel-title{margin-top:0;margin-bottom:0;font-size:15px;color:inherit}.panel-title>.small,.panel-title>.small>a,.panel-title>a,.panel-title>small,.panel-title>small>a{color:inherit}.panel-footer{padding:10px 15px;background-color:#f5f5f5;border-top:1px solid #ddd;border-bottom-right-radius:1px;border-bottom-left-radius:1px}.panel>.list-group,.panel>.panel-collapse>.list-group{margin-bottom:0}.panel>.list-group .list-group-item,.panel>.panel-collapse>.list-group .list-group-item{border-width:1px 0;border-radius:0}.panel>.list-group:first-child .list-group-item:first-child,.panel>.panel-collapse>.list-group:first-child .list-group-item:first-child{border-top:0;border-top-right-radius:1px;border-top-left-radius:1px}.panel>.list-group:last-child .list-group-item:last-child,.panel>.panel-collapse>.list-group:last-child .list-group-item:last-child{border-bottom:0;border-bottom-right-radius:1px;border-bottom-left-radius:1px}.list-group+.panel-footer,.panel-heading+.list-group .list-group-item:first-child{border-top-width:0}.panel>.panel-collapse>.table,.panel>.table,.panel>.table-responsive>.table{margin-bottom:0}.panel>.panel-collapse>.table caption,.panel>.table caption,.panel>.table-responsive>.table caption{padding-left:15px;padding-right:15px}.panel>.table-responsive:first-child>.table:first-child,.panel>.table:first-child{border-top-right-radius:1px;border-top-left-radius:1px}.panel>.table-responsive:first-child>.table:first-child>tbody:first-child>tr:first-child,.panel>.table-responsive:first-child>.table:first-child>thead:first-child>tr:first-child,.panel>.table:first-child>tbody:first-child>tr:first-child,.panel>.table:first-child>thead:first-child>tr:first-child{border-top-left-radius:1px;border-top-right-radius:1px}.panel>.table-responsive:first-child>.table:first-child>tbody:first-child>tr:first-child td:first-child,.panel>.table-responsive:first-child>.table:first-child>tbody:first-child>tr:first-child th:first-child,.panel>.table-responsive:first-child>.table:first-child>thead:first-child>tr:first-child td:first-child,.panel>.table-responsive:first-child>.table:first-child>thead:first-child>tr:first-child th:first-child,.panel>.table:first-child>tbody:first-child>tr:first-child td:first-child,.panel>.table:first-child>tbody:first-child>tr:first-child th:first-child,.panel>.table:first-child>thead:first-child>tr:first-child td:first-child,.panel>.table:first-child>thead:first-child>tr:first-child th:first-child{border-top-left-radius:1px}.panel>.table-responsive:first-child>.table:first-child>tbody:first-child>tr:first-child td:last-child,.panel>.table-responsive:first-child>.table:first-child>tbody:first-child>tr:first-child th:last-child,.panel>.table-responsive:first-child>.table:first-child>thead:first-child>tr:first-child td:last-child,.panel>.table-responsive:first-child>.table:first-child>thead:first-child>tr:first-child th:last-child,.panel>.table:first-child>tbody:first-child>tr:first-child td:last-child,.panel>.table:first-child>tbody:first-child>tr:first-child th:last-child,.panel>.table:first-child>thead:first-child>tr:first-child td:last-child,.panel>.table:first-child>thead:first-child>tr:first-child th:last-child{border-top-right-radius:1px}.panel>.table-responsive:last-child>.table:last-child,.panel>.table:last-child{border-bottom-right-radius:1px;border-bottom-left-radius:1px}.panel>.table-responsive:last-child>.table:last-child>tbody:last-child>tr:last-child,.panel>.table-responsive:last-child>.table:last-child>tfoot:last-child>tr:last-child,.panel>.table:last-child>tbody:last-child>tr:last-child,.panel>.table:last-child>tfoot:last-child>tr:last-child{border-bottom-left-radius:1px;border-bottom-right-radius:1px}.panel>.table-responsive:last-child>.table:last-child>tbody:last-child>tr:last-child td:first-child,.panel>.table-responsive:last-child>.table:last-child>tbody:last-child>tr:last-child th:first-child,.panel>.table-responsive:last-child>.table:last-child>tfoot:last-child>tr:last-child td:first-child,.panel>.table-responsive:last-child>.table:last-child>tfoot:last-child>tr:last-child th:first-child,.panel>.table:last-child>tbody:last-child>tr:last-child td:first-child,.panel>.table:last-child>tbody:last-child>tr:last-child th:first-child,.panel>.table:last-child>tfoot:last-child>tr:last-child td:first-child,.panel>.table:last-child>tfoot:last-child>tr:last-child th:first-child{border-bottom-left-radius:1px}.panel>.table-responsive:last-child>.table:last-child>tbody:last-child>tr:last-child td:last-child,.panel>.table-responsive:last-child>.table:last-child>tbody:last-child>tr:last-child th:last-child,.panel>.table-responsive:last-child>.table:last-child>tfoot:last-child>tr:last-child td:last-child,.panel>.table-responsive:last-child>.table:last-child>tfoot:last-child>tr:last-child th:last-child,.panel>.table:last-child>tbody:last-child>tr:last-child td:last-child,.panel>.table:last-child>tbody:last-child>tr:last-child th:last-child,.panel>.table:last-child>tfoot:last-child>tr:last-child td:last-child,.panel>.table:last-child>tfoot:last-child>tr:last-child th:last-child{border-bottom-right-radius:1px}.panel>.panel-body+.table,.panel>.panel-body+.table-responsive,.panel>.table+.panel-body,.panel>.table-responsive+.panel-body{border-top:1px solid #ddd}.panel>.table>tbody:first-child>tr:first-child td,.panel>.table>tbody:first-child>tr:first-child th{border-top:0}.panel>.table-bordered,.panel>.table-responsive>.table-bordered{border:0}.panel>.table-bordered>tbody>tr>td:first-child,.panel>.table-bordered>tbody>tr>th:first-child,.panel>.table-bordered>tfoot>tr>td:first-child,.panel>.table-bordered>tfoot>tr>th:first-child,.panel>.table-bordered>thead>tr>td:first-child,.panel>.table-bordered>thead>tr>th:first-child,.panel>.table-responsive>.table-bordered>tbody>tr>td:first-child,.panel>.table-responsive>.table-bordered>tbody>tr>th:first-child,.panel>.table-responsive>.table-bordered>tfoot>tr>td:first-child,.panel>.table-responsive>.table-bordered>tfoot>tr>th:first-child,.panel>.table-responsive>.table-bordered>thead>tr>td:first-child,.panel>.table-responsive>.table-bordered>thead>tr>th:first-child{border-left:0}.panel>.table-bordered>tbody>tr>td:last-child,.panel>.table-bordered>tbody>tr>th:last-child,.panel>.table-bordered>tfoot>tr>td:last-child,.panel>.table-bordered>tfoot>tr>th:last-child,.panel>.table-bordered>thead>tr>td:last-child,.panel>.table-bordered>thead>tr>th:last-child,.panel>.table-responsive>.table-bordered>tbody>tr>td:last-child,.panel>.table-responsive>.table-bordered>tbody>tr>th:last-child,.panel>.table-responsive>.table-bordered>tfoot>tr>td:last-child,.panel>.table-responsive>.table-bordered>tfoot>tr>th:last-child,.panel>.table-responsive>.table-bordered>thead>tr>td:last-child,.panel>.table-responsive>.table-bordered>thead>tr>th:last-child{border-right:0}.panel>.table-bordered>tbody>tr:first-child>td,.panel>.table-bordered>tbody>tr:first-child>th,.panel>.table-bordered>tbody>tr:last-child>td,.panel>.table-bordered>tbody>tr:last-child>th,.panel>.table-bordered>tfoot>tr:last-child>td,.panel>.table-bordered>tfoot>tr:last-child>th,.panel>.table-bordered>thead>tr:first-child>td,.panel>.table-bordered>thead>tr:first-child>th,.panel>.table-responsive>.table-bordered>tbody>tr:first-child>td,.panel>.table-responsive>.table-bordered>tbody>tr:first-child>th,.panel>.table-responsive>.table-bordered>tbody>tr:last-child>td,.panel>.table-responsive>.table-bordered>tbody>tr:last-child>th,.panel>.table-responsive>.table-bordered>tfoot>tr:last-child>td,.panel>.table-responsive>.table-bordered>tfoot>tr:last-child>th,.panel>.table-responsive>.table-bordered>thead>tr:first-child>td,.panel>.table-responsive>.table-bordered>thead>tr:first-child>th{border-bottom:0}.panel>.table-responsive{border:0;margin-bottom:0}.panel-group{margin-bottom:18px}.panel-group .panel{margin-bottom:0;border-radius:2px}.panel-group .panel+.panel{margin-top:5px}.panel-group .panel-heading{border-bottom:0}.panel-group .panel-heading+.panel-collapse>.list-group,.panel-group .panel-heading+.panel-collapse>.panel-body{border-top:1px solid #ddd}.panel-group .panel-footer{border-top:0}.panel-group .panel-footer+.panel-collapse .panel-body{border-bottom:1px solid #ddd}.panel-default{border-color:#ddd}.panel-default>.panel-heading{color:#333;background-color:#f5f5f5;border-color:#ddd}.panel-default>.panel-heading+.panel-collapse>.panel-body{border-top-color:#ddd}.panel-default>.panel-heading .badge{color:#f5f5f5;background-color:#333}.panel-default>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#ddd}.panel-primary{border-color:#337ab7}.panel-primary>.panel-heading{color:#fff;background-color:#337ab7;border-color:#337ab7}.panel-primary>.panel-heading+.panel-collapse>.panel-body{border-top-color:#337ab7}.panel-primary>.panel-heading .badge{color:#337ab7;background-color:#fff}.panel-primary>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#337ab7}.panel-success{border-color:#d6e9c6}.panel-success>.panel-heading{color:#3c763d;background-color:#dff0d8;border-color:#d6e9c6}.panel-success>.panel-heading+.panel-collapse>.panel-body{border-top-color:#d6e9c6}.panel-success>.panel-heading .badge{color:#dff0d8;background-color:#3c763d}.panel-success>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#d6e9c6}.panel-info{border-color:#bce8f1}.panel-info>.panel-heading{color:#31708f;background-color:#d9edf7;border-color:#bce8f1}.panel-info>.panel-heading+.panel-collapse>.panel-body{border-top-color:#bce8f1}.panel-info>.panel-heading .badge{color:#d9edf7;background-color:#31708f}.panel-info>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#bce8f1}.panel-warning{border-color:#faebcc}.panel-warning>.panel-heading{color:#8a6d3b;background-color:#fcf8e3;border-color:#faebcc}.panel-warning>.panel-heading+.panel-collapse>.panel-body{border-top-color:#faebcc}.panel-warning>.panel-heading .badge{color:#fcf8e3;background-color:#8a6d3b}.panel-warning>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#faebcc}.panel-danger{border-color:#ebccd1}.panel-danger>.panel-heading{color:#a94442;background-color:#f2dede;border-color:#ebccd1}.panel-danger>.panel-heading+.panel-collapse>.panel-body{border-top-color:#ebccd1}.panel-danger>.panel-heading .badge{color:#f2dede;background-color:#a94442}.panel-danger>.panel-footer+.panel-collapse>.panel-body{border-bottom-color:#ebccd1}.embed-responsive{position:relative;display:block;height:0;padding:0;overflow:hidden}.embed-responsive .embed-responsive-item,.embed-responsive embed,.embed-responsive iframe,.embed-responsive object,.embed-responsive video{position:absolute;top:0;left:0;bottom:0;height:100%;width:100%;border:0}.embed-responsive-16by9{padding-bottom:56.25%}.embed-responsive-4by3{padding-bottom:75%}.well{min-height:20px;padding:19px;margin-bottom:20px;background-color:#f5f5f5;border:1px solid #e3e3e3;border-radius:2px;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.05);box-shadow:inset 0 1px 1px rgba(0,0,0,.05)}.well blockquote{border-color:#ddd;border-color:rgba(0,0,0,.15)}.well-lg{padding:24px;border-radius:3px}.well-sm{padding:9px;border-radius:1px}.close{float:right;font-size:19.5px;font-weight:700;line-height:1;color:#000;text-shadow:0 1px 0 #fff;opacity:.2;filter:alpha(opacity=20)}.close:focus,.close:hover{color:#000;text-decoration:none;cursor:pointer;opacity:.5;filter:alpha(opacity=50)}button.close{padding:0;cursor:pointer;background:0 0;border:0;-webkit-appearance:none}.modal-open{overflow:hidden}.modal{display:none;overflow:hidden;position:fixed;top:0;right:0;bottom:0;left:0;z-index:1050;-webkit-overflow-scrolling:touch;outline:0}.modal.fade .modal-dialog{-webkit-transition:-webkit-transform .3s ease-out;-moz-transition:-moz-transform .3s ease-out;-o-transition:-o-transform .3s ease-out;transition:transform .3s ease-out}.modal.in .modal-dialog{-webkit-transform:translate(0,0);-ms-transform:translate(0,0);-o-transform:translate(0,0);transform:translate(0,0)}.modal-open .modal{overflow-x:hidden;overflow-y:auto}.modal-dialog{position:relative;width:auto;margin:10px}.modal-content{position:relative;background-color:#fff;border:1px solid #999;border:1px solid rgba(0,0,0,.2);border-radius:3px;-webkit-box-shadow:0 3px 9px rgba(0,0,0,.5);box-shadow:0 3px 9px rgba(0,0,0,.5);background-clip:padding-box;outline:0}.modal-backdrop{position:fixed;top:0;right:0;bottom:0;left:0;z-index:1040;background-color:#000}.modal-backdrop.fade{opacity:0;filter:alpha(opacity=0)}.modal-backdrop.in{opacity:.5;filter:alpha(opacity=50)}.modal-header{padding:15px;border-bottom:1px solid #e5e5e5;min-height:16.43px}.modal-header .close{margin-top:-2px}.modal-title{margin:0;line-height:1.42857143}.modal-body{position:relative;padding:15px}.modal-footer{padding:15px;text-align:right;border-top:1px solid #e5e5e5}.modal-footer .btn+.btn{margin-left:5px;margin-bottom:0}.modal-footer .btn-group .btn+.btn{margin-left:-1px}.modal-footer .btn-block+.btn-block{margin-left:0}.modal-scrollbar-measure{position:absolute;top:-9999px;width:50px;height:50px;overflow:scroll}@media (min-width:768px){.modal-dialog{width:600px;margin:30px auto}.modal-content{-webkit-box-shadow:0 5px 15px rgba(0,0,0,.5);box-shadow:0 5px 15px rgba(0,0,0,.5)}.modal-sm{width:300px}}@media (min-width:992px){.modal-lg{width:900px}}.tooltip{position:absolute;z-index:1070;display:block;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:12px;font-weight:400;line-height:1.4;opacity:0;filter:alpha(opacity=0)}.tooltip.in{opacity:.9;filter:alpha(opacity=90)}.tooltip.top{margin-top:-3px;padding:5px 0}.tooltip.right{margin-left:3px;padding:0 5px}.tooltip.bottom{margin-top:3px;padding:5px 0}.tooltip.left{margin-left:-3px;padding:0 5px}.tooltip-inner{max-width:200px;padding:3px 8px;color:#fff;text-align:center;text-decoration:none;background-color:#000;border-radius:2px}.tooltip-arrow{position:absolute;width:0;height:0;border-color:transparent;border-style:solid}.tooltip.top .tooltip-arrow{bottom:0;left:50%;margin-left:-5px;border-width:5px 5px 0;border-top-color:#000}.tooltip.top-left .tooltip-arrow{bottom:0;right:5px;margin-bottom:-5px;border-width:5px 5px 0;border-top-color:#000}.tooltip.top-right .tooltip-arrow{bottom:0;left:5px;margin-bottom:-5px;border-width:5px 5px 0;border-top-color:#000}.tooltip.right .tooltip-arrow{top:50%;left:0;margin-top:-5px;border-width:5px 5px 5px 0;border-right-color:#000}.tooltip.left .tooltip-arrow{top:50%;right:0;margin-top:-5px;border-width:5px 0 5px 5px;border-left-color:#000}.tooltip.bottom .tooltip-arrow{top:0;left:50%;margin-left:-5px;border-width:0 5px 5px;border-bottom-color:#000}.tooltip.bottom-left .tooltip-arrow{top:0;right:5px;margin-top:-5px;border-width:0 5px 5px;border-bottom-color:#000}.tooltip.bottom-right .tooltip-arrow{top:0;left:5px;margin-top:-5px;border-width:0 5px 5px;border-bottom-color:#000}.popover{position:absolute;top:0;left:0;z-index:1060;display:none;max-width:276px;padding:1px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:13px;font-weight:400;line-height:1.42857143;text-align:left;background-color:#fff;background-clip:padding-box;border:1px solid #ccc;border:1px solid rgba(0,0,0,.2);border-radius:3px;-webkit-box-shadow:0 5px 10px rgba(0,0,0,.2);box-shadow:0 5px 10px rgba(0,0,0,.2);white-space:normal}.popover.top{margin-top:-10px}.popover.right{margin-left:10px}.popover.bottom{margin-top:10px}.popover.left{margin-left:-10px}.popover-title{margin:0;padding:8px 14px;font-size:13px;background-color:#f7f7f7;border-bottom:1px solid #ebebeb;border-radius:2px 2px 0 0}.popover-content{padding:9px 14px}.popover>.arrow,.popover>.arrow:after{position:absolute;display:block;width:0;height:0;border-color:transparent;border-style:solid}.popover>.arrow{border-width:11px}.popover>.arrow:after{border-width:10px;content:""}.popover.top>.arrow{left:50%;margin-left:-11px;border-bottom-width:0;border-top-color:#999;border-top-color:rgba(0,0,0,.25);bottom:-11px}.popover.top>.arrow:after{content:" ";bottom:1px;margin-left:-10px;border-bottom-width:0;border-top-color:#fff}.popover.right>.arrow{top:50%;left:-11px;margin-top:-11px;border-left-width:0;border-right-color:#999;border-right-color:rgba(0,0,0,.25)}.popover.right>.arrow:after{content:" ";left:1px;bottom:-10px;border-left-width:0;border-right-color:#fff}.popover.bottom>.arrow{left:50%;margin-left:-11px;border-top-width:0;border-bottom-color:#999;border-bottom-color:rgba(0,0,0,.25);top:-11px}.popover.bottom>.arrow:after{content:" ";top:1px;margin-left:-10px;border-top-width:0;border-bottom-color:#fff}.popover.left>.arrow{top:50%;right:-11px;margin-top:-11px;border-right-width:0;border-left-color:#999;border-left-color:rgba(0,0,0,.25)}.popover.left>.arrow:after{content:" ";right:1px;border-right-width:0;border-left-color:#fff;bottom:-10px}.carousel{position:relative}.carousel-inner{position:relative;overflow:hidden;width:100%}.carousel-inner>.item{display:none;position:relative;-webkit-transition:.6s ease-in-out left;-o-transition:.6s ease-in-out left;transition:.6s ease-in-out left}.carousel-inner>.item>a>img,.carousel-inner>.item>img{line-height:1}@media all and (transform-3d),(-webkit-transform-3d){.carousel-inner>.item{-webkit-transition:-webkit-transform .6s ease-in-out;-moz-transition:-moz-transform .6s ease-in-out;-o-transition:-o-transform .6s ease-in-out;transition:transform .6s ease-in-out;-webkit-backface-visibility:hidden;-moz-backface-visibility:hidden;backface-visibility:hidden;-webkit-perspective:1000;-moz-perspective:1000;perspective:1000}.carousel-inner>.item.active.right,.carousel-inner>.item.next{-webkit-transform:translate3d(100%,0,0);transform:translate3d(100%,0,0);left:0}.carousel-inner>.item.active.left,.carousel-inner>.item.prev{-webkit-transform:translate3d(-100%,0,0);transform:translate3d(-100%,0,0);left:0}.carousel-inner>.item.active,.carousel-inner>.item.next.left,.carousel-inner>.item.prev.right{-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0);left:0}}.carousel-inner>.active,.carousel-inner>.next,.carousel-inner>.prev{display:block}.carousel-inner>.active{left:0}.carousel-inner>.next,.carousel-inner>.prev{position:absolute;top:0;width:100%}.carousel-inner>.next{left:100%}.carousel-inner>.prev{left:-100%}.carousel-inner>.next.left,.carousel-inner>.prev.right{left:0}.carousel-inner>.active.left{left:-100%}.carousel-inner>.active.right{left:100%}.carousel-control{position:absolute;top:0;left:0;bottom:0;width:15%;opacity:.5;filter:alpha(opacity=50);font-size:20px;color:#fff;text-align:center;text-shadow:0 1px 2px rgba(0,0,0,.6)}.carousel-control.left{background-image:-webkit-linear-gradient(left,rgba(0,0,0,.5) 0,rgba(0,0,0,.0001) 100%);background-image:-o-linear-gradient(left,rgba(0,0,0,.5) 0,rgba(0,0,0,.0001) 100%);background-image:linear-gradient(to right,rgba(0,0,0,.5) 0,rgba(0,0,0,.0001) 100%);background-repeat:repeat-x;filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#80000000', endColorstr='#00000000', GradientType=1)}.carousel-control.right{left:auto;right:0;background-image:-webkit-linear-gradient(left,rgba(0,0,0,.0001) 0,rgba(0,0,0,.5) 100%);background-image:-o-linear-gradient(left,rgba(0,0,0,.0001) 0,rgba(0,0,0,.5) 100%);background-image:linear-gradient(to right,rgba(0,0,0,.0001) 0,rgba(0,0,0,.5) 100%);background-repeat:repeat-x;filter:progid:DXImageTransform.Microsoft.gradient(startColorstr='#00000000', endColorstr='#80000000', GradientType=1)}.carousel-control:focus,.carousel-control:hover{outline:0;color:#fff;text-decoration:none;opacity:.9;filter:alpha(opacity=90)}.carousel-control .glyphicon-chevron-left,.carousel-control .glyphicon-chevron-right,.carousel-control .icon-next,.carousel-control .icon-prev{position:absolute;top:50%;z-index:5;display:inline-block}.carousel-control .glyphicon-chevron-left,.carousel-control .icon-prev{left:50%;margin-left:-10px}.carousel-control .glyphicon-chevron-right,.carousel-control .icon-next{right:50%;margin-right:-10px}.carousel-control .icon-next,.carousel-control .icon-prev{width:20px;height:20px;margin-top:-10px;line-height:1;font-family:serif}.carousel-control .icon-prev:before{content:'\2039'}.carousel-control .icon-next:before{content:'\203a'}.carousel-indicators{position:absolute;bottom:10px;left:50%;z-index:15;width:60%;margin-left:-30%;padding-left:0;list-style:none;text-align:center}.carousel-indicators li{display:inline-block;width:10px;height:10px;margin:1px;text-indent:-999px;border:1px solid #fff;border-radius:10px;cursor:pointer;background-color:transparent}.carousel-indicators .active{margin:0;width:12px;height:12px;background-color:#fff}.carousel-caption{position:absolute;left:15%;right:15%;bottom:20px;z-index:10;padding-top:20px;padding-bottom:20px;color:#fff;text-align:center;text-shadow:0 1px 2px rgba(0,0,0,.6)}.carousel-caption .btn{text-shadow:none}@media screen and (min-width:768px){.carousel-control .glyphicon-chevron-left,.carousel-control .glyphicon-chevron-right,.carousel-control .icon-next,.carousel-control .icon-prev{width:30px;height:30px;margin-top:-15px;font-size:30px}.carousel-control .glyphicon-chevron-left,.carousel-control .icon-prev{margin-left:-15px}.carousel-control .glyphicon-chevron-right,.carousel-control .icon-next{margin-right:-15px}.carousel-caption{left:20%;right:20%;padding-bottom:30px}.carousel-indicators{bottom:20px}}.btn-group-vertical>.btn-group:after,.btn-group-vertical>.btn-group:before,.btn-toolbar:after,.btn-toolbar:before,.clearfix:after,.clearfix:before,.container-fluid:after,.container-fluid:before,.container:after,.container:before,.dl-horizontal dd:after,.dl-horizontal dd:before,.form-horizontal .form-group:after,.form-horizontal .form-group:before,.item_buttons:after,.item_buttons:before,.modal-footer:after,.modal-footer:before,.nav:after,.nav:before,.navbar-collapse:after,.navbar-collapse:before,.navbar-header:after,.navbar-header:before,.navbar:after,.navbar:before,.pager:after,.pager:before,.panel-body:after,.panel-body:before,.row:after,.row:before{content:" ";display:table}.btn-group-vertical>.btn-group:after,.btn-toolbar:after,.clearfix:after,.container-fluid:after,.container:after,.dl-horizontal dd:after,.form-horizontal .form-group:after,.item_buttons:after,.modal-footer:after,.nav:after,.navbar-collapse:after,.navbar-header:after,.navbar:after,.pager:after,.panel-body:after,.row:after{clear:both}.center-block{display:block;margin-left:auto;margin-right:auto}.pull-right{float:right!important}.pull-left{float:left!important}.hide{display:none!important}.show{display:block!important}.invisible{visibility:hidden}.text-hide{font:0/0 a;color:transparent;text-shadow:none;background-color:transparent;border:0}.hidden{display:none!important}.affix{position:fixed}@-ms-viewport{width:device-width}.visible-lg,.visible-lg-block,.visible-lg-inline,.visible-lg-inline-block,.visible-md,.visible-md-block,.visible-md-inline,.visible-md-inline-block,.visible-sm,.visible-sm-block,.visible-sm-inline,.visible-sm-inline-block,.visible-xs,.visible-xs-block,.visible-xs-inline,.visible-xs-inline-block{display:none!important}@media (max-width:767px){.visible-xs{display:block!important}table.visible-xs{display:table}tr.visible-xs{display:table-row!important}td.visible-xs,th.visible-xs{display:table-cell!important}.visible-xs-block{display:block!important}.visible-xs-inline{display:inline!important}.visible-xs-inline-block{display:inline-block!important}}@media (min-width:768px)and (max-width:991px){.visible-sm{display:block!important}table.visible-sm{display:table}tr.visible-sm{display:table-row!important}td.visible-sm,th.visible-sm{display:table-cell!important}.visible-sm-block{display:block!important}.visible-sm-inline{display:inline!important}.visible-sm-inline-block{display:inline-block!important}}@media (min-width:992px)and (max-width:1199px){.visible-md{display:block!important}table.visible-md{display:table}tr.visible-md{display:table-row!important}td.visible-md,th.visible-md{display:table-cell!important}.visible-md-block{display:block!important}.visible-md-inline{display:inline!important}.visible-md-inline-block{display:inline-block!important}}@media (min-width:auto){.visible-lg{display:block!important}table.visible-lg{display:table}tr.visible-lg{display:table-row!important}td.visible-lg,th.visible-lg{display:table-cell!important}.visible-lg-block{display:block!important}.visible-lg-inline{display:inline!important}.visible-lg-inline-block{display:inline-block!important}}@media (max-width:767px){.hidden-xs{display:none!important}}@media (min-width:768px)and (max-width:991px){.hidden-sm{display:none!important}}@media (min-width:992px)and (max-width:1199px){.hidden-md{display:none!important}}@media (min-width:auto){.hidden-lg{display:none!important}}.visible-print{display:none!important}@media print{.visible-print{display:block!important}table.visible-print{display:table}tr.visible-print{display:table-row!important}td.visible-print,th.visible-print{display:table-cell!important}}.visible-print-block{display:none!important}@media print{.visible-print-block{display:block!important}}.visible-print-inline{display:none!important}@media print{.visible-print-inline{display:inline!important}}.visible-print-inline-block{display:none!important}@media print{.visible-print-inline-block{display:inline-block!important}.hidden-print{display:none!important}}/*!
*
* Font Awesome
*
*//*!
* Font Awesome 4.2.0 by @davegandy - http://fontawesome.io - @fontawesome
* License - http://fontawesome.io/license (Font: SIL OFL 1.1, CSS: MIT License)
*/@font-face{font-family:'FontAwesome';src:url(../components/font-awesome/fonts/fontawesome-webfont.eot?v=4.2.0);src:url(../components/font-awesome/fonts/fontawesome-webfont.eot?#iefix&v=4.2.0)format('embedded-opentype'),url(../components/font-awesome/fonts/fontawesome-webfont.woff?v=4.2.0)format('woff'),url(../components/font-awesome/fonts/fontawesome-webfont.ttf?v=4.2.0)format('truetype'),url(../components/font-awesome/fonts/fontawesome-webfont.svg?v=4.2.0#fontawesomeregular)format('svg');font-weight:400;font-style:normal}.fa{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale}.fa-lg{font-size:1.33333333em;line-height:.75em;vertical-align:-15%}.fa-2x{font-size:2em}.fa-3x{font-size:3em}.fa-4x{font-size:4em}.fa-5x{font-size:5em}.fa-fw{width:1.28571429em;text-align:center}.fa-ul{padding-left:0;margin-left:2.14285714em;list-style-type:none}.fa-ul>li{position:relative}.fa-li{position:absolute;left:-2.14285714em;width:2.14285714em;top:.14285714em;text-align:center}.fa-li.fa-lg{left:-1.85714286em}.fa-border{padding:.2em .25em .15em;border:.08em solid #eee;border-radius:.1em}.fa.pull-left{margin-right:.3em}.fa.pull-right{margin-left:.3em}.fa-spin{-webkit-animation:fa-spin 2s infinite linear;animation:fa-spin 2s infinite linear}@-webkit-keyframes fa-spin{0%{-webkit-transform:rotate(0);transform:rotate(0)}100%{-webkit-transform:rotate(359deg);transform:rotate(359deg)}}@keyframes fa-spin{0%{-webkit-transform:rotate(0);transform:rotate(0)}100%{-webkit-transform:rotate(359deg);transform:rotate(359deg)}}.fa-rotate-90{filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=1);-webkit-transform:rotate(90deg);-ms-transform:rotate(90deg);transform:rotate(90deg)}.fa-rotate-180{filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=2);-webkit-transform:rotate(180deg);-ms-transform:rotate(180deg);transform:rotate(180deg)}.fa-rotate-270{filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=3);-webkit-transform:rotate(270deg);-ms-transform:rotate(270deg);transform:rotate(270deg)}.fa-flip-horizontal{filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=0, mirror=1);-webkit-transform:scale(-1,1);-ms-transform:scale(-1,1);transform:scale(-1,1)}.fa-flip-vertical{filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=2, mirror=1);-webkit-transform:scale(1,-1);-ms-transform:scale(1,-1);transform:scale(1,-1)}:root .fa-flip-horizontal,:root .fa-flip-vertical,:root .fa-rotate-180,:root .fa-rotate-270,:root .fa-rotate-90{filter:none}.fa-stack{position:relative;display:inline-block;width:2em;height:2em;line-height:2em;vertical-align:middle}.fa-stack-1x,.fa-stack-2x{position:absolute;left:0;width:100%;text-align:center}.fa-stack-1x{line-height:inherit}.fa-stack-2x{font-size:2em}.fa-inverse{color:#fff}.fa-glass:before{content:"\f000"}.fa-music:before{content:"\f001"}.fa-search:before{content:"\f002"}.fa-envelope-o:before{content:"\f003"}.fa-heart:before{content:"\f004"}.fa-star:before{content:"\f005"}.fa-star-o:before{content:"\f006"}.fa-user:before{content:"\f007"}.fa-film:before{content:"\f008"}.fa-th-large:before{content:"\f009"}.fa-th:before{content:"\f00a"}.fa-th-list:before{content:"\f00b"}.fa-check:before{content:"\f00c"}.fa-close:before,.fa-remove:before,.fa-times:before{content:"\f00d"}.fa-search-plus:before{content:"\f00e"}.fa-search-minus:before{content:"\f010"}.fa-power-off:before{content:"\f011"}.fa-signal:before{content:"\f012"}.fa-cog:before,.fa-gear:before{content:"\f013"}.fa-trash-o:before{content:"\f014"}.fa-home:before{content:"\f015"}.fa-file-o:before{content:"\f016"}.fa-clock-o:before{content:"\f017"}.fa-road:before{content:"\f018"}.fa-download:before{content:"\f019"}.fa-arrow-circle-o-down:before{content:"\f01a"}.fa-arrow-circle-o-up:before{content:"\f01b"}.fa-inbox:before{content:"\f01c"}.fa-play-circle-o:before{content:"\f01d"}.fa-repeat:before,.fa-rotate-right:before{content:"\f01e"}.fa-refresh:before{content:"\f021"}.fa-list-alt:before{content:"\f022"}.fa-lock:before{content:"\f023"}.fa-flag:before{content:"\f024"}.fa-headphones:before{content:"\f025"}.fa-volume-off:before{content:"\f026"}.fa-volume-down:before{content:"\f027"}.fa-volume-up:before{content:"\f028"}.fa-qrcode:before{content:"\f029"}.fa-barcode:before{content:"\f02a"}.fa-tag:before{content:"\f02b"}.fa-tags:before{content:"\f02c"}.fa-book:before{content:"\f02d"}.fa-bookmark:before{content:"\f02e"}.fa-print:before{content:"\f02f"}.fa-camera:before{content:"\f030"}.fa-font:before{content:"\f031"}.fa-bold:before{content:"\f032"}.fa-italic:before{content:"\f033"}.fa-text-height:before{content:"\f034"}.fa-text-width:before{content:"\f035"}.fa-align-left:before{content:"\f036"}.fa-align-center:before{content:"\f037"}.fa-align-right:before{content:"\f038"}.fa-align-justify:before{content:"\f039"}.fa-list:before{content:"\f03a"}.fa-dedent:before,.fa-outdent:before{content:"\f03b"}.fa-indent:before{content:"\f03c"}.fa-video-camera:before{content:"\f03d"}.fa-image:before,.fa-photo:before,.fa-picture-o:before{content:"\f03e"}.fa-pencil:before{content:"\f040"}.fa-map-marker:before{content:"\f041"}.fa-adjust:before{content:"\f042"}.fa-tint:before{content:"\f043"}.fa-edit:before,.fa-pencil-square-o:before{content:"\f044"}.fa-share-square-o:before{content:"\f045"}.fa-check-square-o:before{content:"\f046"}.fa-arrows:before{content:"\f047"}.fa-step-backward:before{content:"\f048"}.fa-fast-backward:before{content:"\f049"}.fa-backward:before{content:"\f04a"}.fa-play:before{content:"\f04b"}.fa-pause:before{content:"\f04c"}.fa-stop:before{content:"\f04d"}.fa-forward:before{content:"\f04e"}.fa-fast-forward:before{content:"\f050"}.fa-step-forward:before{content:"\f051"}.fa-eject:before{content:"\f052"}.fa-chevron-left:before{content:"\f053"}.fa-chevron-right:before{content:"\f054"}.fa-plus-circle:before{content:"\f055"}.fa-minus-circle:before{content:"\f056"}.fa-times-circle:before{content:"\f057"}.fa-check-circle:before{content:"\f058"}.fa-question-circle:before{content:"\f059"}.fa-info-circle:before{content:"\f05a"}.fa-crosshairs:before{content:"\f05b"}.fa-times-circle-o:before{content:"\f05c"}.fa-check-circle-o:before{content:"\f05d"}.fa-ban:before{content:"\f05e"}.fa-arrow-left:before{content:"\f060"}.fa-arrow-right:before{content:"\f061"}.fa-arrow-up:before{content:"\f062"}.fa-arrow-down:before{content:"\f063"}.fa-mail-forward:before,.fa-share:before{content:"\f064"}.fa-expand:before{content:"\f065"}.fa-compress:before{content:"\f066"}.fa-plus:before{content:"\f067"}.fa-minus:before{content:"\f068"}.fa-asterisk:before{content:"\f069"}.fa-exclamation-circle:before{content:"\f06a"}.fa-gift:before{content:"\f06b"}.fa-leaf:before{content:"\f06c"}.fa-fire:before{content:"\f06d"}.fa-eye:before{content:"\f06e"}.fa-eye-slash:before{content:"\f070"}.fa-exclamation-triangle:before,.fa-warning:before{content:"\f071"}.fa-plane:before{content:"\f072"}.fa-calendar:before{content:"\f073"}.fa-random:before{content:"\f074"}.fa-comment:before{content:"\f075"}.fa-magnet:before{content:"\f076"}.fa-chevron-up:before{content:"\f077"}.fa-chevron-down:before{content:"\f078"}.fa-retweet:before{content:"\f079"}.fa-shopping-cart:before{content:"\f07a"}.fa-folder:before{content:"\f07b"}.fa-folder-open:before{content:"\f07c"}.fa-arrows-v:before{content:"\f07d"}.fa-arrows-h:before{content:"\f07e"}.fa-bar-chart-o:before,.fa-bar-chart:before{content:"\f080"}.fa-twitter-square:before{content:"\f081"}.fa-facebook-square:before{content:"\f082"}.fa-camera-retro:before{content:"\f083"}.fa-key:before{content:"\f084"}.fa-cogs:before,.fa-gears:before{content:"\f085"}.fa-comments:before{content:"\f086"}.fa-thumbs-o-up:before{content:"\f087"}.fa-thumbs-o-down:before{content:"\f088"}.fa-star-half:before{content:"\f089"}.fa-heart-o:before{content:"\f08a"}.fa-sign-out:before{content:"\f08b"}.fa-linkedin-square:before{content:"\f08c"}.fa-thumb-tack:before{content:"\f08d"}.fa-external-link:before{content:"\f08e"}.fa-sign-in:before{content:"\f090"}.fa-trophy:before{content:"\f091"}.fa-github-square:before{content:"\f092"}.fa-upload:before{content:"\f093"}.fa-lemon-o:before{content:"\f094"}.fa-phone:before{content:"\f095"}.fa-square-o:before{content:"\f096"}.fa-bookmark-o:before{content:"\f097"}.fa-phone-square:before{content:"\f098"}.fa-twitter:before{content:"\f099"}.fa-facebook:before{content:"\f09a"}.fa-github:before{content:"\f09b"}.fa-unlock:before{content:"\f09c"}.fa-credit-card:before{content:"\f09d"}.fa-rss:before{content:"\f09e"}.fa-hdd-o:before{content:"\f0a0"}.fa-bullhorn:before{content:"\f0a1"}.fa-bell:before{content:"\f0f3"}.fa-certificate:before{content:"\f0a3"}.fa-hand-o-right:before{content:"\f0a4"}.fa-hand-o-left:before{content:"\f0a5"}.fa-hand-o-up:before{content:"\f0a6"}.fa-hand-o-down:before{content:"\f0a7"}.fa-arrow-circle-left:before{content:"\f0a8"}.fa-arrow-circle-right:before{content:"\f0a9"}.fa-arrow-circle-up:before{content:"\f0aa"}.fa-arrow-circle-down:before{content:"\f0ab"}.fa-globe:before{content:"\f0ac"}.fa-wrench:before{content:"\f0ad"}.fa-tasks:before{content:"\f0ae"}.fa-filter:before{content:"\f0b0"}.fa-briefcase:before{content:"\f0b1"}.fa-arrows-alt:before{content:"\f0b2"}.fa-group:before,.fa-users:before{content:"\f0c0"}.fa-chain:before,.fa-link:before{content:"\f0c1"}.fa-cloud:before{content:"\f0c2"}.fa-flask:before{content:"\f0c3"}.fa-cut:before,.fa-scissors:before{content:"\f0c4"}.fa-copy:before,.fa-files-o:before{content:"\f0c5"}.fa-paperclip:before{content:"\f0c6"}.fa-floppy-o:before,.fa-save:before{content:"\f0c7"}.fa-square:before{content:"\f0c8"}.fa-bars:before,.fa-navicon:before,.fa-reorder:before{content:"\f0c9"}.fa-list-ul:before{content:"\f0ca"}.fa-list-ol:before{content:"\f0cb"}.fa-strikethrough:before{content:"\f0cc"}.fa-underline:before{content:"\f0cd"}.fa-table:before{content:"\f0ce"}.fa-magic:before{content:"\f0d0"}.fa-truck:before{content:"\f0d1"}.fa-pinterest:before{content:"\f0d2"}.fa-pinterest-square:before{content:"\f0d3"}.fa-google-plus-square:before{content:"\f0d4"}.fa-google-plus:before{content:"\f0d5"}.fa-money:before{content:"\f0d6"}.fa-caret-down:before{content:"\f0d7"}.fa-caret-up:before{content:"\f0d8"}.fa-caret-left:before{content:"\f0d9"}.fa-caret-right:before{content:"\f0da"}.fa-columns:before{content:"\f0db"}.fa-sort:before,.fa-unsorted:before{content:"\f0dc"}.fa-sort-desc:before,.fa-sort-down:before{content:"\f0dd"}.fa-sort-asc:before,.fa-sort-up:before{content:"\f0de"}.fa-envelope:before{content:"\f0e0"}.fa-linkedin:before{content:"\f0e1"}.fa-rotate-left:before,.fa-undo:before{content:"\f0e2"}.fa-gavel:before,.fa-legal:before{content:"\f0e3"}.fa-dashboard:before,.fa-tachometer:before{content:"\f0e4"}.fa-comment-o:before{content:"\f0e5"}.fa-comments-o:before{content:"\f0e6"}.fa-bolt:before,.fa-flash:before{content:"\f0e7"}.fa-sitemap:before{content:"\f0e8"}.fa-umbrella:before{content:"\f0e9"}.fa-clipboard:before,.fa-paste:before{content:"\f0ea"}.fa-lightbulb-o:before{content:"\f0eb"}.fa-exchange:before{content:"\f0ec"}.fa-cloud-download:before{content:"\f0ed"}.fa-cloud-upload:before{content:"\f0ee"}.fa-user-md:before{content:"\f0f0"}.fa-stethoscope:before{content:"\f0f1"}.fa-suitcase:before{content:"\f0f2"}.fa-bell-o:before{content:"\f0a2"}.fa-coffee:before{content:"\f0f4"}.fa-cutlery:before{content:"\f0f5"}.fa-file-text-o:before{content:"\f0f6"}.fa-building-o:before{content:"\f0f7"}.fa-hospital-o:before{content:"\f0f8"}.fa-ambulance:before{content:"\f0f9"}.fa-medkit:before{content:"\f0fa"}.fa-fighter-jet:before{content:"\f0fb"}.fa-beer:before{content:"\f0fc"}.fa-h-square:before{content:"\f0fd"}.fa-plus-square:before{content:"\f0fe"}.fa-angle-double-left:before{content:"\f100"}.fa-angle-double-right:before{content:"\f101"}.fa-angle-double-up:before{content:"\f102"}.fa-angle-double-down:before{content:"\f103"}.fa-angle-left:before{content:"\f104"}.fa-angle-right:before{content:"\f105"}.fa-angle-up:before{content:"\f106"}.fa-angle-down:before{content:"\f107"}.fa-desktop:before{content:"\f108"}.fa-laptop:before{content:"\f109"}.fa-tablet:before{content:"\f10a"}.fa-mobile-phone:before,.fa-mobile:before{content:"\f10b"}.fa-circle-o:before{content:"\f10c"}.fa-quote-left:before{content:"\f10d"}.fa-quote-right:before{content:"\f10e"}.fa-spinner:before{content:"\f110"}.fa-circle:before{content:"\f111"}.fa-mail-reply:before,.fa-reply:before{content:"\f112"}.fa-github-alt:before{content:"\f113"}.fa-folder-o:before{content:"\f114"}.fa-folder-open-o:before{content:"\f115"}.fa-smile-o:before{content:"\f118"}.fa-frown-o:before{content:"\f119"}.fa-meh-o:before{content:"\f11a"}.fa-gamepad:before{content:"\f11b"}.fa-keyboard-o:before{content:"\f11c"}.fa-flag-o:before{content:"\f11d"}.fa-flag-checkered:before{content:"\f11e"}.fa-terminal:before{content:"\f120"}.fa-code:before{content:"\f121"}.fa-mail-reply-all:before,.fa-reply-all:before{content:"\f122"}.fa-star-half-empty:before,.fa-star-half-full:before,.fa-star-half-o:before{content:"\f123"}.fa-location-arrow:before{content:"\f124"}.fa-crop:before{content:"\f125"}.fa-code-fork:before{content:"\f126"}.fa-chain-broken:before,.fa-unlink:before{content:"\f127"}.fa-question:before{content:"\f128"}.fa-info:before{content:"\f129"}.fa-exclamation:before{content:"\f12a"}.fa-superscript:before{content:"\f12b"}.fa-subscript:before{content:"\f12c"}.fa-eraser:before{content:"\f12d"}.fa-puzzle-piece:before{content:"\f12e"}.fa-microphone:before{content:"\f130"}.fa-microphone-slash:before{content:"\f131"}.fa-shield:before{content:"\f132"}.fa-calendar-o:before{content:"\f133"}.fa-fire-extinguisher:before{content:"\f134"}.fa-rocket:before{content:"\f135"}.fa-maxcdn:before{content:"\f136"}.fa-chevron-circle-left:before{content:"\f137"}.fa-chevron-circle-right:before{content:"\f138"}.fa-chevron-circle-up:before{content:"\f139"}.fa-chevron-circle-down:before{content:"\f13a"}.fa-html5:before{content:"\f13b"}.fa-css3:before{content:"\f13c"}.fa-anchor:before{content:"\f13d"}.fa-unlock-alt:before{content:"\f13e"}.fa-bullseye:before{content:"\f140"}.fa-ellipsis-h:before{content:"\f141"}.fa-ellipsis-v:before{content:"\f142"}.fa-rss-square:before{content:"\f143"}.fa-play-circle:before{content:"\f144"}.fa-ticket:before{content:"\f145"}.fa-minus-square:before{content:"\f146"}.fa-minus-square-o:before{content:"\f147"}.fa-level-up:before{content:"\f148"}.fa-level-down:before{content:"\f149"}.fa-check-square:before{content:"\f14a"}.fa-pencil-square:before{content:"\f14b"}.fa-external-link-square:before{content:"\f14c"}.fa-share-square:before{content:"\f14d"}.fa-compass:before{content:"\f14e"}.fa-caret-square-o-down:before,.fa-toggle-down:before{content:"\f150"}.fa-caret-square-o-up:before,.fa-toggle-up:before{content:"\f151"}.fa-caret-square-o-right:before,.fa-toggle-right:before{content:"\f152"}.fa-eur:before,.fa-euro:before{content:"\f153"}.fa-gbp:before{content:"\f154"}.fa-dollar:before,.fa-usd:before{content:"\f155"}.fa-inr:before,.fa-rupee:before{content:"\f156"}.fa-cny:before,.fa-jpy:before,.fa-rmb:before,.fa-yen:before{content:"\f157"}.fa-rouble:before,.fa-rub:before,.fa-ruble:before{content:"\f158"}.fa-krw:before,.fa-won:before{content:"\f159"}.fa-bitcoin:before,.fa-btc:before{content:"\f15a"}.fa-file:before{content:"\f15b"}.fa-file-text:before{content:"\f15c"}.fa-sort-alpha-asc:before{content:"\f15d"}.fa-sort-alpha-desc:before{content:"\f15e"}.fa-sort-amount-asc:before{content:"\f160"}.fa-sort-amount-desc:before{content:"\f161"}.fa-sort-numeric-asc:before{content:"\f162"}.fa-sort-numeric-desc:before{content:"\f163"}.fa-thumbs-up:before{content:"\f164"}.fa-thumbs-down:before{content:"\f165"}.fa-youtube-square:before{content:"\f166"}.fa-youtube:before{content:"\f167"}.fa-xing:before{content:"\f168"}.fa-xing-square:before{content:"\f169"}.fa-youtube-play:before{content:"\f16a"}.fa-dropbox:before{content:"\f16b"}.fa-stack-overflow:before{content:"\f16c"}.fa-instagram:before{content:"\f16d"}.fa-flickr:before{content:"\f16e"}.fa-adn:before{content:"\f170"}.fa-bitbucket:before{content:"\f171"}.fa-bitbucket-square:before{content:"\f172"}.fa-tumblr:before{content:"\f173"}.fa-tumblr-square:before{content:"\f174"}.fa-long-arrow-down:before{content:"\f175"}.fa-long-arrow-up:before{content:"\f176"}.fa-long-arrow-left:before{content:"\f177"}.fa-long-arrow-right:before{content:"\f178"}.fa-apple:before{content:"\f179"}.fa-windows:before{content:"\f17a"}.fa-android:before{content:"\f17b"}.fa-linux:before{content:"\f17c"}.fa-dribbble:before{content:"\f17d"}.fa-skype:before{content:"\f17e"}.fa-foursquare:before{content:"\f180"}.fa-trello:before{content:"\f181"}.fa-female:before{content:"\f182"}.fa-male:before{content:"\f183"}.fa-gittip:before{content:"\f184"}.fa-sun-o:before{content:"\f185"}.fa-moon-o:before{content:"\f186"}.fa-archive:before{content:"\f187"}.fa-bug:before{content:"\f188"}.fa-vk:before{content:"\f189"}.fa-weibo:before{content:"\f18a"}.fa-renren:before{content:"\f18b"}.fa-pagelines:before{content:"\f18c"}.fa-stack-exchange:before{content:"\f18d"}.fa-arrow-circle-o-right:before{content:"\f18e"}.fa-arrow-circle-o-left:before{content:"\f190"}.fa-caret-square-o-left:before,.fa-toggle-left:before{content:"\f191"}.fa-dot-circle-o:before{content:"\f192"}.fa-wheelchair:before{content:"\f193"}.fa-vimeo-square:before{content:"\f194"}.fa-try:before,.fa-turkish-lira:before{content:"\f195"}.fa-plus-square-o:before{content:"\f196"}.fa-space-shuttle:before{content:"\f197"}.fa-slack:before{content:"\f198"}.fa-envelope-square:before{content:"\f199"}.fa-wordpress:before{content:"\f19a"}.fa-openid:before{content:"\f19b"}.fa-bank:before,.fa-institution:before,.fa-university:before{content:"\f19c"}.fa-graduation-cap:before,.fa-mortar-board:before{content:"\f19d"}.fa-yahoo:before{content:"\f19e"}.fa-google:before{content:"\f1a0"}.fa-reddit:before{content:"\f1a1"}.fa-reddit-square:before{content:"\f1a2"}.fa-stumbleupon-circle:before{content:"\f1a3"}.fa-stumbleupon:before{content:"\f1a4"}.fa-delicious:before{content:"\f1a5"}.fa-digg:before{content:"\f1a6"}.fa-pied-piper:before{content:"\f1a7"}.fa-pied-piper-alt:before{content:"\f1a8"}.fa-drupal:before{content:"\f1a9"}.fa-joomla:before{content:"\f1aa"}.fa-language:before{content:"\f1ab"}.fa-fax:before{content:"\f1ac"}.fa-building:before{content:"\f1ad"}.fa-child:before{content:"\f1ae"}.fa-paw:before{content:"\f1b0"}.fa-spoon:before{content:"\f1b1"}.fa-cube:before{content:"\f1b2"}.fa-cubes:before{content:"\f1b3"}.fa-behance:before{content:"\f1b4"}.fa-behance-square:before{content:"\f1b5"}.fa-steam:before{content:"\f1b6"}.fa-steam-square:before{content:"\f1b7"}.fa-recycle:before{content:"\f1b8"}.fa-automobile:before,.fa-car:before{content:"\f1b9"}.fa-cab:before,.fa-taxi:before{content:"\f1ba"}.fa-tree:before{content:"\f1bb"}.fa-spotify:before{content:"\f1bc"}.fa-deviantart:before{content:"\f1bd"}.fa-soundcloud:before{content:"\f1be"}.fa-database:before{content:"\f1c0"}.fa-file-pdf-o:before{content:"\f1c1"}.fa-file-word-o:before{content:"\f1c2"}.fa-file-excel-o:before{content:"\f1c3"}.fa-file-powerpoint-o:before{content:"\f1c4"}.fa-file-image-o:before,.fa-file-photo-o:before,.fa-file-picture-o:before{content:"\f1c5"}.fa-file-archive-o:before,.fa-file-zip-o:before{content:"\f1c6"}.fa-file-audio-o:before,.fa-file-sound-o:before{content:"\f1c7"}.fa-file-movie-o:before,.fa-file-video-o:before{content:"\f1c8"}.fa-file-code-o:before{content:"\f1c9"}.fa-vine:before{content:"\f1ca"}.fa-codepen:before{content:"\f1cb"}.fa-jsfiddle:before{content:"\f1cc"}.fa-life-bouy:before,.fa-life-buoy:before,.fa-life-ring:before,.fa-life-saver:before,.fa-support:before{content:"\f1cd"}.fa-circle-o-notch:before{content:"\f1ce"}.fa-ra:before,.fa-rebel:before{content:"\f1d0"}.fa-empire:before,.fa-ge:before{content:"\f1d1"}.fa-git-square:before{content:"\f1d2"}.fa-git:before{content:"\f1d3"}.fa-hacker-news:before{content:"\f1d4"}.fa-tencent-weibo:before{content:"\f1d5"}.fa-qq:before{content:"\f1d6"}.fa-wechat:before,.fa-weixin:before{content:"\f1d7"}.fa-paper-plane:before,.fa-send:before{content:"\f1d8"}.fa-paper-plane-o:before,.fa-send-o:before{content:"\f1d9"}.fa-history:before{content:"\f1da"}.fa-circle-thin:before{content:"\f1db"}.fa-header:before{content:"\f1dc"}.fa-paragraph:before{content:"\f1dd"}.fa-sliders:before{content:"\f1de"}.fa-share-alt:before{content:"\f1e0"}.fa-share-alt-square:before{content:"\f1e1"}.fa-bomb:before{content:"\f1e2"}.fa-futbol-o:before,.fa-soccer-ball-o:before{content:"\f1e3"}.fa-tty:before{content:"\f1e4"}.fa-binoculars:before{content:"\f1e5"}.fa-plug:before{content:"\f1e6"}.fa-slideshare:before{content:"\f1e7"}.fa-twitch:before{content:"\f1e8"}.fa-yelp:before{content:"\f1e9"}.fa-newspaper-o:before{content:"\f1ea"}.fa-wifi:before{content:"\f1eb"}.fa-calculator:before{content:"\f1ec"}.fa-paypal:before{content:"\f1ed"}.fa-google-wallet:before{content:"\f1ee"}.fa-cc-visa:before{content:"\f1f0"}.fa-cc-mastercard:before{content:"\f1f1"}.fa-cc-discover:before{content:"\f1f2"}.fa-cc-amex:before{content:"\f1f3"}.fa-cc-paypal:before{content:"\f1f4"}.fa-cc-stripe:before{content:"\f1f5"}.fa-bell-slash:before{content:"\f1f6"}.fa-bell-slash-o:before{content:"\f1f7"}.fa-trash:before{content:"\f1f8"}.fa-copyright:before{content:"\f1f9"}.fa-at:before{content:"\f1fa"}.fa-eyedropper:before{content:"\f1fb"}.fa-paint-brush:before{content:"\f1fc"}.fa-birthday-cake:before{content:"\f1fd"}.fa-area-chart:before{content:"\f1fe"}.fa-pie-chart:before{content:"\f200"}.fa-line-chart:before{content:"\f201"}.fa-lastfm:before{content:"\f202"}.fa-lastfm-square:before{content:"\f203"}.fa-toggle-off:before{content:"\f204"}.fa-toggle-on:before{content:"\f205"}.fa-bicycle:before{content:"\f206"}.fa-bus:before{content:"\f207"}.fa-ioxhost:before{content:"\f208"}.fa-angellist:before{content:"\f209"}.fa-cc:before{content:"\f20a"}.fa-ils:before,.fa-shekel:before,.fa-sheqel:before{content:"\f20b"}.fa-meanpath:before{content:"\f20c"}/*!
*
* IPython base
*
*/.modal.fade .modal-dialog{-webkit-transform:translate(0,0);-ms-transform:translate(0,0);-o-transform:translate(0,0);transform:translate(0,0)}code{color:#000}pre{font-size:inherit;line-height:inherit}label{font-weight:400}.border-box-sizing{box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box}.corner-all{border-radius:2px}.no-padding{padding:0}.hbox{display:-webkit-box;-webkit-box-orient:horizontal;display:-moz-box;-moz-box-orient:horizontal;display:box;box-orient:horizontal;box-align:stretch;display:flex;flex-direction:row;align-items:stretch}.hbox>*{-webkit-box-flex:0;-moz-box-flex:0;box-flex:0;flex:none}.vbox{display:-webkit-box;-webkit-box-orient:vertical;display:-moz-box;-moz-box-orient:vertical;display:box;box-orient:vertical;box-align:stretch;display:flex;flex-direction:column;align-items:stretch}.vbox>*{-webkit-box-flex:0;-moz-box-flex:0;box-flex:0;flex:none}.hbox.reverse,.reverse,.vbox.reverse{-webkit-box-direction:reverse;-moz-box-direction:reverse;box-direction:reverse;flex-direction:row-reverse}.box-flex0,.hbox.box-flex0,.vbox.box-flex0{-webkit-box-flex:0;-moz-box-flex:0;box-flex:0;flex:none;width:auto}.box-flex1,.hbox.box-flex1,.vbox.box-flex1{-webkit-box-flex:1;-moz-box-flex:1;box-flex:1;flex:1}.box-flex,.hbox.box-flex,.vbox.box-flex{-webkit-box-flex:1;-moz-box-flex:1;box-flex:1;flex:1}.box-flex2,.hbox.box-flex2,.vbox.box-flex2{-webkit-box-flex:2;-moz-box-flex:2;box-flex:2;flex:2}.box-group1{-webkit-box-flex-group:1;-moz-box-flex-group:1;box-flex-group:1}.box-group2{-webkit-box-flex-group:2;-moz-box-flex-group:2;box-flex-group:2}.hbox.start,.start,.vbox.start{-webkit-box-pack:start;-moz-box-pack:start;box-pack:start;justify-content:flex-start}.end,.hbox.end,.vbox.end{-webkit-box-pack:end;-moz-box-pack:end;box-pack:end;justify-content:flex-end}.center,.hbox.center,.vbox.center{-webkit-box-pack:center;-moz-box-pack:center;box-pack:center;justify-content:center}.baseline,.hbox.baseline,.vbox.baseline{-webkit-box-pack:baseline;-moz-box-pack:baseline;box-pack:baseline;justify-content:baseline}.hbox.stretch,.stretch,.vbox.stretch{-webkit-box-pack:stretch;-moz-box-pack:stretch;box-pack:stretch;justify-content:stretch}.align-start,.hbox.align-start,.vbox.align-start{-webkit-box-align:start;-moz-box-align:start;box-align:start;align-items:flex-start}.align-end,.hbox.align-end,.vbox.align-end{-webkit-box-align:end;-moz-box-align:end;box-align:end;align-items:flex-end}.align-center,.hbox.align-center,.vbox.align-center{-webkit-box-align:center;-moz-box-align:center;box-align:center;align-items:center}.align-baseline,.hbox.align-baseline,.vbox.align-baseline{-webkit-box-align:baseline;-moz-box-align:baseline;box-align:baseline;align-items:baseline}.align-stretch,.hbox.align-stretch,.vbox.align-stretch{-webkit-box-align:stretch;-moz-box-align:stretch;box-align:stretch;align-items:stretch}div.error{margin:2em;text-align:center}div.error>h1{font-size:500%;line-height:normal}div.error>p{font-size:200%;line-height:normal}div.traceback-wrapper{text-align:left;max-width:800px;margin:auto}body{position:absolute;left:0;right:0;top:0;bottom:0;overflow:visible}#header{display:none;background-color:#fff;position:relative;z-index:100}#header #header-container{padding-bottom:5px;padding-top:5px;box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box}#header .header-bar{width:100%;height:1px;background:#e7e7e7;margin-bottom:-1px}#header-spacer{width:100%;visibility:hidden}@media print{#header{display:none!important}#header-spacer{display:none}}#ipython_notebook{padding-left:0;padding-top:1px;padding-bottom:1px}@media (max-width:991px){#ipython_notebook{margin-left:10px}}#noscript{width:auto;padding-top:16px;padding-bottom:16px;text-align:center;font-size:22px;color:red;font-weight:700}#ipython_notebook img{height:28px}#site{width:100%;display:none;box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box;overflow:auto}@media print{#site{height:auto!important}}.ui-button .ui-button-text{padding:.2em .8em;font-size:77%}input.ui-button{padding:.3em .9em}span#login_widget{float:right}#logout,span#login_widget>.button{color:#333;background-color:#fff;border-color:#ccc}#logout.active,#logout.focus,#logout:active,#logout:focus,#logout:hover,.open>.dropdown-toggle#logout,.open>.dropdown-togglespan#login_widget>.button,span#login_widget>.button.active,span#login_widget>.button.focus,span#login_widget>.button:active,span#login_widget>.button:focus,span#login_widget>.button:hover{color:#333;background-color:#e6e6e6;border-color:#adadad}#logout.active,#logout:active,.open>.dropdown-toggle#logout,.open>.dropdown-togglespan#login_widget>.button,span#login_widget>.button.active,span#login_widget>.button:active{background-image:none}#logout.disabled,#logout.disabled.active,#logout.disabled.focus,#logout.disabled:active,#logout.disabled:focus,#logout.disabled:hover,#logout[disabled],#logout[disabled].active,#logout[disabled].focus,#logout[disabled]:active,#logout[disabled]:focus,#logout[disabled]:hover,fieldset[disabled] #logout,fieldset[disabled] #logout.active,fieldset[disabled] #logout.focus,fieldset[disabled] #logout:active,fieldset[disabled] #logout:focus,fieldset[disabled] #logout:hover,fieldset[disabled] span#login_widget>.button,fieldset[disabled] span#login_widget>.button.active,fieldset[disabled] span#login_widget>.button.focus,fieldset[disabled] span#login_widget>.button:active,fieldset[disabled] span#login_widget>.button:focus,fieldset[disabled] span#login_widget>.button:hover,span#login_widget>.button.disabled,span#login_widget>.button.disabled.active,span#login_widget>.button.disabled.focus,span#login_widget>.button.disabled:active,span#login_widget>.button.disabled:focus,span#login_widget>.button.disabled:hover,span#login_widget>.button[disabled],span#login_widget>.button[disabled].active,span#login_widget>.button[disabled].focus,span#login_widget>.button[disabled]:active,span#login_widget>.button[disabled]:focus,span#login_widget>.button[disabled]:hover{background-color:#fff;border-color:#ccc}#logout .badge,span#login_widget>.button .badge{color:#fff;background-color:#333}.nav-header{text-transform:none}#header>span{margin-top:10px}.modal_stretch .modal-dialog{display:-webkit-box;-webkit-box-orient:vertical;display:-moz-box;-moz-box-orient:vertical;display:box;box-orient:vertical;box-align:stretch;display:flex;flex-direction:column;align-items:stretch;min-height:80vh}.modal_stretch .modal-dialog .modal-body{max-height:calc(100vh - 200px);overflow:auto;flex:1}@media (min-width:768px){.modal .modal-dialog{width:700px}select.form-control{margin-left:12px;margin-right:12px}}/*!
*
* IPython auth
*
*/.center-nav{display:inline-block;margin-bottom:-4px}/*!
*
* IPython tree view
*
*/.alternate_upload{background-color:none;display:inline}.alternate_upload.form{padding:0;margin:0}.alternate_upload input.fileinput{text-align:center;vertical-align:middle;display:inline;opacity:0;z-index:2;width:12ex;margin-right:-12ex}.alternate_upload .btn-upload{height:22px}ul#tabs{margin-bottom:4px}ul#tabs a{padding-top:6px;padding-bottom:4px}ul.breadcrumb a:focus,ul.breadcrumb a:hover{text-decoration:none}ul.breadcrumb i.icon-home{font-size:16px;margin-right:4px}ul.breadcrumb span{color:#5e5e5e}.list_toolbar{padding:4px 0;vertical-align:middle}.list_toolbar .tree-buttons{padding-top:1px}.dynamic-buttons{padding-top:3px;display:inline-block}.list_toolbar [class*=span]{min-height:24px}.list_header{font-weight:700;background-color:#eee}.list_placeholder{font-weight:700;padding:4px 7px}.list_container{margin-top:4px;margin-bottom:20px;border:1px solid #ddd;border-radius:2px}.list_container>div{border-bottom:1px solid #ddd}.list_container>div:hover .list-item{background-color:red}.list_container>div:last-child{border:none}.list_item:hover .list_item{background-color:#ddd}.list_item a{text-decoration:none}.list_item:hover{background-color:#fafafa}.action_col{text-align:right}.list_header>div,.list_item>div{line-height:22px;padding:4px 7px}.list_header>div input,.list_item>div input{margin-right:7px;margin-left:14px;vertical-align:baseline;line-height:22px;position:relative;top:-1px}.list_header>div .item_link,.list_item>div .item_link{margin-left:-1px;vertical-align:baseline;line-height:22px}.new-file input[type=checkbox]{visibility:hidden}.item_name{line-height:22px;height:24px}.item_icon{font-size:14px;color:#5e5e5e;margin-right:7px;margin-left:7px;line-height:22px;vertical-align:baseline}.item_buttons{line-height:1em;margin-left:-5px}.item_buttons .btn-group,.item_buttons .input-group{float:left}.item_buttons>.btn,.item_buttons>.btn-group,.item_buttons>.input-group{margin-left:5px}.item_buttons .btn{min-width:13ex}.item_buttons .running-indicator{padding-top:4px;color:#5cb85c}.toolbar_info{height:24px;line-height:24px}input.engine_num_input,input.nbname_input{padding-top:3px;padding-bottom:3px;height:22px;line-height:14px;margin:0}input.engine_num_input{width:60px}.highlight_text{color:#00f}#project_name{display:inline-block;padding-left:7px;margin-left:-2px}#project_name>.breadcrumb{padding:0;margin-bottom:0;background-color:transparent;font-weight:700}#tree-selector{padding-right:0}#button-select-all{min-width:50px}#select-all{margin-left:7px;margin-right:2px}.menu_icon{margin-right:2px}.tab-content .row{margin-left:0;margin-right:0}.folder_icon:before{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;content:"\f114"}.folder_icon:before.pull-left{margin-right:.3em}.folder_icon:before.pull-right{margin-left:.3em}.notebook_icon:before{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;content:"\f02d";position:relative;top:-1px}.notebook_icon:before.pull-left{margin-right:.3em}.notebook_icon:before.pull-right{margin-left:.3em}.running_notebook_icon:before{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;content:"\f02d";position:relative;top:-1px;color:#5cb85c}.running_notebook_icon:before.pull-left{margin-right:.3em}.running_notebook_icon:before.pull-right{margin-left:.3em}.file_icon:before{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;content:"\f016";position:relative;top:-2px}.file_icon:before.pull-left{margin-right:.3em}.file_icon:before.pull-right{margin-left:.3em}#notebook_toolbar .pull-right{padding-top:0;margin-right:-1px}ul#new-menu{left:auto;right:0}.kernel-menu-icon{padding-right:12px;width:24px;content:"\f096"}.kernel-menu-icon:before{content:"\f096"}.kernel-menu-icon-current:before{content:"\f00c"}#tab_content{padding-top:20px}#running .panel-group .panel{margin-top:3px;margin-bottom:1em}#running .panel-group .panel .panel-heading{background-color:#eee;line-height:22px;padding:4px 7px}#running .panel-group .panel .panel-heading a:focus,#running .panel-group .panel .panel-heading a:hover{text-decoration:none}#running .panel-group .panel .panel-body{padding:0}#running .panel-group .panel .panel-body .list_container{margin-top:0;margin-bottom:0;border:0;border-radius:0}#running .panel-group .panel .panel-body .list_container .list_item{border-bottom:1px solid #ddd}#running .panel-group .panel .panel-body .list_container .list_item:last-child{border-bottom:0}.delete-button,.duplicate-button,.rename-button,.shutdown-button{display:none}.dynamic-instructions{display:inline-block;padding-top:4px}/*!
*
* IPython text editor webapp
*
*/.selected-keymap i.fa{padding:0 5px}.selected-keymap i.fa:before{content:"\f00c"}#mode-menu{overflow:auto;max-height:20em}.edit_app #header{-webkit-box-shadow:0 0 12px 1px rgba(87,87,87,.2);box-shadow:0 0 12px 1px rgba(87,87,87,.2)}.edit_app #menubar .navbar{margin-bottom:-1px}.dirty-indicator{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;width:20px}.dirty-indicator.pull-left{margin-right:.3em}.dirty-indicator.pull-right{margin-left:.3em}.dirty-indicator-dirty{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;width:20px}.dirty-indicator-dirty.pull-left{margin-right:.3em}.dirty-indicator-dirty.pull-right{margin-left:.3em}.dirty-indicator-clean{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;width:20px}.dirty-indicator-clean.pull-left{margin-right:.3em}.dirty-indicator-clean.pull-right{margin-left:.3em}.dirty-indicator-clean:before{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;content:"\f00c"}.dirty-indicator-clean:before.pull-left{margin-right:.3em}.dirty-indicator-clean:before.pull-right{margin-left:.3em}#filename{font-size:16pt;display:table;padding:0 5px}#current-mode{padding-left:5px;padding-right:5px}#texteditor-backdrop{padding-top:20px;padding-bottom:20px}@media not print{#texteditor-backdrop{background-color:#eee}}@media print{#texteditor-backdrop #texteditor-container .CodeMirror-gutter,#texteditor-backdrop #texteditor-container .CodeMirror-gutters{background-color:#fff}}@media not print{#texteditor-backdrop #texteditor-container .CodeMirror-gutter,#texteditor-backdrop #texteditor-container .CodeMirror-gutters{background-color:#fff}#texteditor-backdrop #texteditor-container{padding:0;background-color:#fff;-webkit-box-shadow:0 0 12px 1px rgba(87,87,87,.2);box-shadow:0 0 12px 1px rgba(87,87,87,.2)}}/*!
*
* IPython notebook
*
*/.ansibold{font-weight:700}.ansiblack{color:#000}.ansired{color:#8b0000}.ansigreen{color:#006400}.ansiyellow{color:#c4a000}.ansiblue{color:#00008b}.ansipurple{color:#9400d3}.ansicyan{color:#4682b4}.ansigray{color:gray}.ansibgblack{background-color:#000}.ansibgred{background-color:red}.ansibggreen{background-color:green}.ansibgyellow{background-color:#ff0}.ansibgblue{background-color:#00f}.ansibgpurple{background-color:#ff00ff}.ansibgcyan{background-color:#0ff}.ansibggray{background-color:gray}div.cell{border:1px solid transparent;display:-webkit-box;-webkit-box-orient:vertical;display:-moz-box;-moz-box-orient:vertical;display:box;box-orient:vertical;box-align:stretch;display:flex;flex-direction:column;align-items:stretch;border-radius:2px;box-sizing:border-box;-moz-box-sizing:border-box;border-width:thin;border-style:solid;width:100%;padding:5px;margin:0;outline:0}div.cell.selected{border-color:#ababab}@media print{div.cell.selected{border-color:transparent}}.edit_mode div.cell.selected{border-color:green}.prompt{min-width:14ex;padding:.4em;margin:0;font-family:monospace;text-align:right;line-height:1.21429em}div.inner_cell{display:-webkit-box;-webkit-box-orient:vertical;display:-moz-box;-moz-box-orient:vertical;display:box;box-orient:vertical;box-align:stretch;display:flex;flex-direction:column;align-items:stretch;-webkit-box-flex:1;-moz-box-flex:1;box-flex:1;flex:1}@-moz-document url-prefix(){div.inner_cell{overflow-x:hidden}}div.input_area{border:1px solid #cfcfcf;border-radius:2px;background:#f7f7f7;line-height:1.21429em}div.prompt:empty{padding-top:0;padding-bottom:0}div.unrecognized_cell{padding:5px 5px 5px 0;display:-webkit-box;-webkit-box-orient:horizontal;display:-moz-box;-moz-box-orient:horizontal;display:box;box-orient:horizontal;box-align:stretch;display:flex;flex-direction:row;align-items:stretch}div.unrecognized_cell .inner_cell{border-radius:2px;padding:5px;font-weight:700;color:red;border:1px solid #cfcfcf;background:#eaeaea}div.unrecognized_cell .inner_cell a,div.unrecognized_cell .inner_cell a:hover{color:inherit;text-decoration:none}@media (max-width:540px){.prompt{text-align:left}div.unrecognized_cell>div.prompt{display:none}}div.code_cell{}div.input{page-break-inside:avoid;display:-webkit-box;-webkit-box-orient:horizontal;display:-moz-box;-moz-box-orient:horizontal;display:box;box-orient:horizontal;box-align:stretch;display:flex;flex-direction:row;align-items:stretch}@media (max-width:540px){div.input{-webkit-box-orient:vertical;-moz-box-orient:vertical;box-orient:vertical;box-align:stretch;display:flex;flex-direction:column;align-items:stretch}}div.input_prompt{color:navy;border-top:1px solid transparent}div.input_area>div.highlight{margin:.4em;border:none;padding:0;background-color:transparent}div.input_area>div.highlight>pre{margin:0;border:none;padding:0;background-color:transparent}.CodeMirror{line-height:1.21429em;font-size:14px;height:auto;background:0 0}.CodeMirror-scroll{overflow-y:hidden;overflow-x:auto}.CodeMirror-lines{padding:.4em}.CodeMirror-linenumber{padding:0 8px 0 4px}.CodeMirror-gutters{border-bottom-left-radius:2px;border-top-left-radius:2px}.CodeMirror pre{padding:0;border:0;border-radius:0}.highlight-base,.highlight-variable{color:#000}.highlight-variable-2{color:#1a1a1a}.highlight-variable-3{color:#333}.highlight-string{color:#BA2121}.highlight-comment{color:#408080;font-style:italic}.highlight-number{color:#080}.highlight-atom{color:#88F}.highlight-keyword{color:green;font-weight:700}.highlight-builtin{color:green}.highlight-error{color:red}.highlight-operator{color:#A2F;font-weight:700}.highlight-meta{color:#A2F}.highlight-def{color:#00f}.highlight-string-2{color:#f50}.highlight-qualifier{color:#555}.highlight-bracket{color:#997}.highlight-tag{color:#170}.highlight-attribute{color:#00c}.highlight-header{color:#00f}.highlight-quote{color:#090}.highlight-link{color:#00c}.cm-s-ipython span.cm-keyword{color:green;font-weight:700}.cm-s-ipython span.cm-atom{color:#88F}.cm-s-ipython span.cm-number{color:#080}.cm-s-ipython span.cm-def{color:#00f}.cm-s-ipython span.cm-variable{color:#000}.cm-s-ipython span.cm-operator{color:#A2F;font-weight:700}.cm-s-ipython span.cm-variable-2{color:#1a1a1a}.cm-s-ipython span.cm-variable-3{color:#333}.cm-s-ipython span.cm-comment{color:#408080;font-style:italic}.cm-s-ipython span.cm-string{color:#BA2121}.cm-s-ipython span.cm-string-2{color:#f50}.cm-s-ipython span.cm-meta{color:#A2F}.cm-s-ipython span.cm-qualifier{color:#555}.cm-s-ipython span.cm-builtin{color:green}.cm-s-ipython span.cm-bracket{color:#997}.cm-s-ipython span.cm-tag{color:#170}.cm-s-ipython span.cm-attribute{color:#00c}.cm-s-ipython span.cm-header{color:#00f}.cm-s-ipython span.cm-quote{color:#090}.cm-s-ipython span.cm-link{color:#00c}.cm-s-ipython span.cm-error{color:red}.cm-s-ipython span.cm-tab{background:url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAMCAYAAAAkuj5RAAAAAXNSR0IArs4c6QAAAGFJREFUSMft1LsRQFAQheHPowAKoACx3IgEKtaEHujDjORSgWTH/ZOdnZOcM/sgk/kFFWY0qV8foQwS4MKBCS3qR6ixBJvElOobYAtivseIE120FaowJPN75GMu8j/LfMwNjh4HUpwg4LUAAAAASUVORK5CYII=')right no-repeat}div.output_wrapper{display:-webkit-box;-webkit-box-align:stretch;display:-moz-box;-moz-box-align:stretch;display:box;box-orient:vertical;box-align:stretch;display:flex;flex-direction:column;align-items:stretch;z-index:1}div.output_scroll{height:24em;width:100%;overflow:auto;border-radius:2px;-webkit-box-shadow:inset 0 2px 8px rgba(0,0,0,.8);box-shadow:inset 0 2px 8px rgba(0,0,0,.8);display:block}div.output_collapsed{margin:0;padding:0;display:-webkit-box;-webkit-box-orient:vertical;display:-moz-box;-moz-box-orient:vertical;display:box;box-orient:vertical;box-align:stretch;display:flex;flex-direction:column;align-items:stretch}div.out_prompt_overlay{height:100%;padding:0 .4em;position:absolute;border-radius:2px}div.out_prompt_overlay:hover{-webkit-box-shadow:inset 0 0 1px #000;box-shadow:inset 0 0 1px #000;background:rgba(240,240,240,.5)}div.output_prompt{color:#8b0000}div.output_area{padding:0;page-break-inside:avoid;display:-webkit-box;-webkit-box-orient:horizontal;display:-moz-box;-moz-box-orient:horizontal;display:box;box-orient:horizontal;box-align:stretch;display:flex;flex-direction:row;align-items:stretch}div.output_area .MathJax_Display{text-align:left!important}div.output_area .rendered_html img,div.output_area .rendered_html table{margin-left:0;margin-right:0}div.output_area img,div.output_area svg{max-width:100%;height:auto}div.output_area img.unconfined,div.output_area svg.unconfined{max-width:none}.output{display:-webkit-box;-webkit-box-orient:vertical;display:-moz-box;-moz-box-orient:vertical;display:box;box-orient:vertical;box-align:stretch;display:flex;flex-direction:column;align-items:stretch}@media (max-width:540px){div.output_area{-webkit-box-orient:vertical;-moz-box-orient:vertical;box-orient:vertical;box-align:stretch;display:flex;flex-direction:column;align-items:stretch}}div.output_area pre{margin:0;padding:0;border:0;vertical-align:baseline;color:#000;background-color:transparent;border-radius:0}div.output_subarea{overflow-x:auto;padding:.4em;-webkit-box-flex:1;-moz-box-flex:1;box-flex:1;flex:1;max-width:calc(100% - 14ex)}div.output_text{text-align:left;color:#000;line-height:1.21429em}div.output_stderr{background:#fdd}div.output_latex{text-align:left}div.output_javascript:empty{padding:0}.js-error{color:#8b0000}div.raw_input_container{font-family:monospace;padding-top:5px}span.raw_input_prompt{}input.raw_input{font-family:inherit;font-size:inherit;color:inherit;width:auto;vertical-align:baseline;padding:0 .25em;margin:0 .25em}input.raw_input:focus{box-shadow:none}p.p-space{margin-bottom:10px}div.output_unrecognized{padding:5px;font-weight:700;color:red}div.output_unrecognized a,div.output_unrecognized a:hover{color:inherit;text-decoration:none}.rendered_html{color:#000}.rendered_html em{font-style:italic}.rendered_html strong{font-weight:700}.rendered_html :link,.rendered_html :visited,.rendered_html u{text-decoration:underline}.rendered_html h1{font-size:185.7%;margin:1.08em 0 0;font-weight:700;line-height:1}.rendered_html h2{font-size:157.1%;margin:1.27em 0 0;font-weight:700;line-height:1}.rendered_html h3{font-size:128.6%;margin:1.55em 0 0;font-weight:700;line-height:1}.rendered_html h4{font-size:100%;margin:2em 0 0;font-weight:700;line-height:1}.rendered_html h5,.rendered_html h6{font-size:100%;margin:2em 0 0;font-weight:700;line-height:1;font-style:italic}.rendered_html h1:first-child{margin-top:.538em}.rendered_html h2:first-child{margin-top:.636em}.rendered_html h3:first-child{margin-top:.777em}.rendered_html h4:first-child,.rendered_html h5:first-child,.rendered_html h6:first-child{margin-top:1em}.rendered_html ul{list-style:disc;margin:0 2em;padding-left:0}.rendered_html ul ul{list-style:square;margin:0 2em}.rendered_html ul ul ul{list-style:circle;margin:0 2em}.rendered_html ol{list-style:decimal;margin:0 2em;padding-left:0}.rendered_html ol ol{list-style:upper-alpha;margin:0 2em}.rendered_html ol ol ol{list-style:lower-alpha;margin:0 2em}.rendered_html ol ol ol ol{list-style:lower-roman;margin:0 2em}.rendered_html ol ol ol ol ol{list-style:decimal;margin:0 2em}.rendered_html *+ol,.rendered_html *+ul{margin-top:1em}.rendered_html hr{color:#000;background-color:#000}.rendered_html pre{margin:1em 2em}.rendered_html code,.rendered_html pre{border:0;background-color:#fff;color:#000;font-size:100%;padding:0}.rendered_html blockquote{margin:1em 2em}.rendered_html table{margin-left:auto;margin-right:auto;border:1px solid #000;border-collapse:collapse}.rendered_html td,.rendered_html th,.rendered_html tr{border:1px solid #000;border-collapse:collapse;margin:1em 2em}.rendered_html td,.rendered_html th{text-align:left;vertical-align:middle;padding:4px}.rendered_html th{font-weight:700}.rendered_html *+table{margin-top:1em}.rendered_html p{text-align:left}.rendered_html *+p{margin-top:1em}.rendered_html img{display:block;margin-left:auto;margin-right:auto}.rendered_html *+img{margin-top:1em}.rendered_html img,.rendered_html svg{max-width:100%;height:auto}.rendered_html img.unconfined,.rendered_html svg.unconfined{max-width:none}div.text_cell{display:-webkit-box;-webkit-box-orient:horizontal;display:-moz-box;-moz-box-orient:horizontal;display:box;box-orient:horizontal;box-align:stretch;display:flex;flex-direction:row;align-items:stretch}@media (max-width:540px){div.text_cell>div.prompt{display:none}}div.text_cell_render{outline:0;resize:none;width:inherit;border-style:none;padding:.5em .5em .5em .4em;color:#000;box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box}a.anchor-link:link{text-decoration:none;padding:0 20px;visibility:hidden}h1:hover .anchor-link,h2:hover .anchor-link,h3:hover .anchor-link,h4:hover .anchor-link,h5:hover .anchor-link,h6:hover .anchor-link{visibility:visible}.text_cell.rendered .input_area{display:none}.text_cell.rendered .rendered_html{overflow-x:auto}.text_cell.unrendered .text_cell_render{display:none}.cm-header-1,.cm-header-2,.cm-header-3,.cm-header-4,.cm-header-5,.cm-header-6{font-weight:700;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif}.cm-header-1{font-size:185.7%}.cm-header-2{font-size:157.1%}.cm-header-3{font-size:128.6%}.cm-header-4{font-size:110%}.cm-header-5,.cm-header-6{font-size:100%;font-style:italic}/*!
*
* IPython notebook webapp
*
*/@media (max-width:767px){.notebook_app{padding-left:0;padding-right:0}}#ipython-main-app{box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box;height:100%}div#notebook_panel{margin:0;padding:0;box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box;height:100%}#notebook{font-size:14px;line-height:20px;overflow-y:hidden;overflow-x:auto;width:100%;padding-top:20px;margin:0;outline:0;box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box;min-height:100%}@media not print{#notebook-container{padding:15px;background-color:#fff;min-height:0;-webkit-box-shadow:0 0 12px 1px rgba(87,87,87,.2);box-shadow:0 0 12px 1px rgba(87,87,87,.2)}}div.ui-widget-content{border:1px solid #ababab;outline:0}pre.dialog{background-color:#f7f7f7;border:1px solid #ddd;border-radius:2px;padding:.4em .4em .4em 2em}p.dialog{padding:.2em}code,kbd,pre,samp{white-space:pre-wrap}#fonttest{font-family:monospace}p{margin-bottom:0}.end_space{min-height:100px;transition:height .2s ease}.notebook_app #header{-webkit-box-shadow:0 0 12px 1px rgba(87,87,87,.2);box-shadow:0 0 12px 1px rgba(87,87,87,.2)}@media not print{.notebook_app{background-color:#eee}}.celltoolbar{border:thin solid #CFCFCF;border-bottom:none;background:#EEE;border-radius:2px 2px 0 0;width:100%;height:29px;padding-right:4px;-webkit-box-orient:horizontal;-moz-box-orient:horizontal;box-orient:horizontal;box-align:stretch;display:flex;flex-direction:row;align-items:stretch;-webkit-box-pack:end;-moz-box-pack:end;box-pack:end;justify-content:flex-end;font-size:87%;padding-top:3px}@media print{.edit_mode div.cell.selected{border-color:transparent}div.code_cell{page-break-inside:avoid}#notebook-container{width:100%}.celltoolbar{display:none}}.ctb_hideshow{display:none;vertical-align:bottom}.ctb_global_show .ctb_show.ctb_hideshow{display:block}.ctb_global_show .ctb_show+.input_area,.ctb_global_show .ctb_show+div.text_cell_input,.ctb_global_show .ctb_show~div.text_cell_render{border-top-right-radius:0;border-top-left-radius:0}.ctb_global_show .ctb_show~div.text_cell_render{border:1px solid #cfcfcf}.celltoolbar select{color:#555;background-color:#fff;background-image:none;border:1px solid #ccc;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075);box-shadow:inset 0 1px 1px rgba(0,0,0,.075);-webkit-transition:border-color ease-in-out .15s,box-shadow ease-in-out .15s;-o-transition:border-color ease-in-out .15s,box-shadow ease-in-out .15s;transition:border-color ease-in-out .15s,box-shadow ease-in-out .15s;line-height:1.5;border-radius:1px;width:inherit;font-size:inherit;height:22px;padding:0;display:inline-block}.celltoolbar select:focus{border-color:#66afe9;outline:0;-webkit-box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 8px rgba(102,175,233,.6);box-shadow:inset 0 1px 1px rgba(0,0,0,.075),0 0 8px rgba(102,175,233,.6)}.celltoolbar select::-moz-placeholder{color:#999;opacity:1}.celltoolbar select:-ms-input-placeholder{color:#999}.celltoolbar select::-webkit-input-placeholder{color:#999}.celltoolbar select[disabled],.celltoolbar select[readonly],fieldset[disabled] .celltoolbar select{background-color:#eee;opacity:1}.celltoolbar select[disabled],fieldset[disabled] .celltoolbar select{cursor:not-allowed}textarea.celltoolbar select{height:auto}select.celltoolbar select{height:30px;line-height:30px}select[multiple].celltoolbar select,textarea.celltoolbar select{height:auto}.celltoolbar label{margin-left:5px;margin-right:5px}.completions{position:absolute;z-index:10;overflow:hidden;border:1px solid #ababab;border-radius:2px;-webkit-box-shadow:0 6px 10px -1px #adadad;box-shadow:0 6px 10px -1px #adadad;line-height:1}.completions select{background:#fff;outline:0;border:none;padding:0;margin:0;overflow:auto;font-family:monospace;font-size:110%;color:#000;width:auto}.completions select option.context{color:#286090}#kernel_logo_widget{float:right!important;float:right}#kernel_logo_widget .current_kernel_logo{display:none;margin-top:-1px;margin-bottom:-1px;width:32px;height:32px}#menubar{box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box;margin-top:1px}#menubar .navbar{border-top:1px;border-radius:0 0 2px 2px;margin-bottom:0}#menubar .navbar-toggle{float:left;padding-top:7px;padding-bottom:7px;border:none}#menubar .navbar-collapse{clear:left}.nav-wrapper{border-bottom:1px solid #e7e7e7}i.menu-icon{padding-top:4px}ul#help_menu li a{overflow:hidden;padding-right:2.2em}ul#help_menu li a i{margin-right:-1.2em}.dropdown-submenu{position:relative}.dropdown-submenu>.dropdown-menu{top:0;left:100%;margin-top:-6px;margin-left:-1px}.dropdown-submenu:hover>.dropdown-menu{display:block}.dropdown-submenu>a:after{font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;display:block;content:"\f0da";float:right;color:#333;margin-top:2px;margin-right:-10px}.dropdown-submenu>a:after.pull-left{margin-right:.3em}.dropdown-submenu>a:after.pull-right{margin-left:.3em}.dropdown-submenu:hover>a:after{color:#262626}.dropdown-submenu.pull-left{float:none}.dropdown-submenu.pull-left>.dropdown-menu{left:-100%;margin-left:10px}#notification_area{float:right!important;float:right;z-index:10}.indicator_area{float:right!important;float:right;color:#777;margin-left:5px;margin-right:5px;z-index:10;text-align:center;width:auto}#kernel_indicator{float:right!important;float:right;color:#777;margin-left:5px;margin-right:5px;z-index:10;text-align:center;width:auto;border-left:1px solid}#kernel_indicator .kernel_indicator_name{padding-left:5px;padding-right:5px}#modal_indicator{float:right!important;float:right;color:#777;margin-left:5px;margin-right:5px;z-index:10;text-align:center;width:auto}#readonly-indicator{float:right!important;float:right;color:#777;z-index:10;text-align:center;width:auto;display:none;margin:2px 0 0}.modal_indicator:before{width:1.28571429em;text-align:center}.edit_mode .modal_indicator:before{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;content:"\f040"}.edit_mode .modal_indicator:before.pull-left{margin-right:.3em}.edit_mode .modal_indicator:before.pull-right{margin-left:.3em}.command_mode .modal_indicator:before{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;content:' '}.command_mode .modal_indicator:before.pull-left{margin-right:.3em}.command_mode .modal_indicator:before.pull-right{margin-left:.3em}.kernel_idle_icon:before{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;content:"\f10c"}.kernel_idle_icon:before.pull-left{margin-right:.3em}.kernel_idle_icon:before.pull-right{margin-left:.3em}.kernel_busy_icon:before{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;content:"\f111"}.kernel_busy_icon:before.pull-left{margin-right:.3em}.kernel_busy_icon:before.pull-right{margin-left:.3em}.kernel_dead_icon:before{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;content:"\f1e2"}.kernel_dead_icon:before.pull-left{margin-right:.3em}.kernel_dead_icon:before.pull-right{margin-left:.3em}.kernel_disconnected_icon:before{display:inline-block;font:normal normal normal 14px/1 FontAwesome;font-size:inherit;text-rendering:auto;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;content:"\f127"}.kernel_disconnected_icon:before.pull-left{margin-right:.3em}.kernel_disconnected_icon:before.pull-right{margin-left:.3em}.notification_widget{z-index:10;background:rgba(240,240,240,.5);margin-right:4px;color:#333;background-color:#fff;border-color:#ccc}.notification_widget.active,.notification_widget.focus,.notification_widget:active,.notification_widget:focus,.notification_widget:hover,.open>.dropdown-toggle.notification_widget{color:#333;background-color:#e6e6e6;border-color:#adadad}.notification_widget.active,.notification_widget:active,.open>.dropdown-toggle.notification_widget{background-image:none}.notification_widget.disabled,.notification_widget.disabled.active,.notification_widget.disabled.focus,.notification_widget.disabled:active,.notification_widget.disabled:focus,.notification_widget.disabled:hover,.notification_widget[disabled],.notification_widget[disabled].active,.notification_widget[disabled].focus,.notification_widget[disabled]:active,.notification_widget[disabled]:focus,.notification_widget[disabled]:hover,fieldset[disabled] .notification_widget,fieldset[disabled] .notification_widget.active,fieldset[disabled] .notification_widget.focus,fieldset[disabled] .notification_widget:active,fieldset[disabled] .notification_widget:focus,fieldset[disabled] .notification_widget:hover{background-color:#fff;border-color:#ccc}.notification_widget .badge{color:#fff;background-color:#333}.notification_widget.warning{color:#fff;background-color:#f0ad4e;border-color:#eea236}.notification_widget.warning.active,.notification_widget.warning.focus,.notification_widget.warning:active,.notification_widget.warning:focus,.notification_widget.warning:hover,.open>.dropdown-toggle.notification_widget.warning{color:#fff;background-color:#ec971f;border-color:#d58512}.notification_widget.warning.active,.notification_widget.warning:active,.open>.dropdown-toggle.notification_widget.warning{background-image:none}.notification_widget.warning.disabled,.notification_widget.warning.disabled.active,.notification_widget.warning.disabled.focus,.notification_widget.warning.disabled:active,.notification_widget.warning.disabled:focus,.notification_widget.warning.disabled:hover,.notification_widget.warning[disabled],.notification_widget.warning[disabled].active,.notification_widget.warning[disabled].focus,.notification_widget.warning[disabled]:active,.notification_widget.warning[disabled]:focus,.notification_widget.warning[disabled]:hover,fieldset[disabled] .notification_widget.warning,fieldset[disabled] .notification_widget.warning.active,fieldset[disabled] .notification_widget.warning.focus,fieldset[disabled] .notification_widget.warning:active,fieldset[disabled] .notification_widget.warning:focus,fieldset[disabled] .notification_widget.warning:hover{background-color:#f0ad4e;border-color:#eea236}.notification_widget.warning .badge{color:#f0ad4e;background-color:#fff}.notification_widget.success{color:#fff;background-color:#5cb85c;border-color:#4cae4c}.notification_widget.success.active,.notification_widget.success.focus,.notification_widget.success:active,.notification_widget.success:focus,.notification_widget.success:hover,.open>.dropdown-toggle.notification_widget.success{color:#fff;background-color:#449d44;border-color:#398439}.notification_widget.success.active,.notification_widget.success:active,.open>.dropdown-toggle.notification_widget.success{background-image:none}.notification_widget.success.disabled,.notification_widget.success.disabled.active,.notification_widget.success.disabled.focus,.notification_widget.success.disabled:active,.notification_widget.success.disabled:focus,.notification_widget.success.disabled:hover,.notification_widget.success[disabled],.notification_widget.success[disabled].active,.notification_widget.success[disabled].focus,.notification_widget.success[disabled]:active,.notification_widget.success[disabled]:focus,.notification_widget.success[disabled]:hover,fieldset[disabled] .notification_widget.success,fieldset[disabled] .notification_widget.success.active,fieldset[disabled] .notification_widget.success.focus,fieldset[disabled] .notification_widget.success:active,fieldset[disabled] .notification_widget.success:focus,fieldset[disabled] .notification_widget.success:hover{background-color:#5cb85c;border-color:#4cae4c}.notification_widget.success .badge{color:#5cb85c;background-color:#fff}.notification_widget.info{color:#fff;background-color:#5bc0de;border-color:#46b8da}.notification_widget.info.active,.notification_widget.info.focus,.notification_widget.info:active,.notification_widget.info:focus,.notification_widget.info:hover,.open>.dropdown-toggle.notification_widget.info{color:#fff;background-color:#31b0d5;border-color:#269abc}.notification_widget.info.active,.notification_widget.info:active,.open>.dropdown-toggle.notification_widget.info{background-image:none}.notification_widget.info.disabled,.notification_widget.info.disabled.active,.notification_widget.info.disabled.focus,.notification_widget.info.disabled:active,.notification_widget.info.disabled:focus,.notification_widget.info.disabled:hover,.notification_widget.info[disabled],.notification_widget.info[disabled].active,.notification_widget.info[disabled].focus,.notification_widget.info[disabled]:active,.notification_widget.info[disabled]:focus,.notification_widget.info[disabled]:hover,fieldset[disabled] .notification_widget.info,fieldset[disabled] .notification_widget.info.active,fieldset[disabled] .notification_widget.info.focus,fieldset[disabled] .notification_widget.info:active,fieldset[disabled] .notification_widget.info:focus,fieldset[disabled] .notification_widget.info:hover{background-color:#5bc0de;border-color:#46b8da}.notification_widget.info .badge{color:#5bc0de;background-color:#fff}.notification_widget.danger{color:#fff;background-color:#d9534f;border-color:#d43f3a}.notification_widget.danger.active,.notification_widget.danger.focus,.notification_widget.danger:active,.notification_widget.danger:focus,.notification_widget.danger:hover,.open>.dropdown-toggle.notification_widget.danger{color:#fff;background-color:#c9302c;border-color:#ac2925}.notification_widget.danger.active,.notification_widget.danger:active,.open>.dropdown-toggle.notification_widget.danger{background-image:none}.notification_widget.danger.disabled,.notification_widget.danger.disabled.active,.notification_widget.danger.disabled.focus,.notification_widget.danger.disabled:active,.notification_widget.danger.disabled:focus,.notification_widget.danger.disabled:hover,.notification_widget.danger[disabled],.notification_widget.danger[disabled].active,.notification_widget.danger[disabled].focus,.notification_widget.danger[disabled]:active,.notification_widget.danger[disabled]:focus,.notification_widget.danger[disabled]:hover,fieldset[disabled] .notification_widget.danger,fieldset[disabled] .notification_widget.danger.active,fieldset[disabled] .notification_widget.danger.focus,fieldset[disabled] .notification_widget.danger:active,fieldset[disabled] .notification_widget.danger:focus,fieldset[disabled] .notification_widget.danger:hover{background-color:#d9534f;border-color:#d43f3a}.notification_widget.danger .badge{color:#d9534f;background-color:#fff}div#pager{background-color:#fff;font-size:14px;line-height:20px;overflow:hidden;display:none;position:fixed;bottom:0;width:100%;max-height:50%;padding-top:8px;-webkit-box-shadow:0 0 12px 1px rgba(87,87,87,.2);box-shadow:0 0 12px 1px rgba(87,87,87,.2);z-index:100;top:auto!important}div#pager pre{line-height:1.21429em;color:#000;background-color:#f7f7f7;padding:.4em}div#pager #pager-button-area{position:absolute;top:8px;right:20px}div#pager #pager-contents{position:relative;overflow:auto;width:100%;height:100%}div#pager #pager-contents #pager-container{position:relative;padding:15px 0;box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box}div#pager .ui-resizable-handle{top:0;height:8px;background:#f7f7f7;border-top:1px solid #cfcfcf;border-bottom:1px solid #cfcfcf}div#pager .ui-resizable-handle::after{content:'';top:2px;left:50%;height:3px;width:30px;margin-left:-15px;position:absolute;border-top:1px solid #cfcfcf}.quickhelp{display:-webkit-box;-webkit-box-orient:horizontal;display:-moz-box;-moz-box-orient:horizontal;display:box;box-orient:horizontal;box-align:stretch;display:flex;flex-direction:row;align-items:stretch}.shortcut_key{display:inline-block;width:20ex;text-align:right;font-family:monospace}.shortcut_descr{display:inline-block;-webkit-box-flex:1;-moz-box-flex:1;box-flex:1;flex:1}span.save_widget{margin-top:6px}span.save_widget span.filename{height:1em;line-height:1em;padding:3px;margin-left:16px;border:none;font-size:146.5%;border-radius:2px}span.save_widget span.filename:hover{background-color:#e6e6e6}span.autosave_status,span.checkpoint_status{font-size:small}@media (max-width:767px){span.save_widget{font-size:small}span.autosave_status,span.checkpoint_status{display:none}}@media (min-width:768px)and (max-width:991px){span.checkpoint_status{display:none}span.autosave_status{font-size:x-small}}.toolbar{padding:0;margin-left:-5px;margin-top:2px;margin-bottom:5px;box-sizing:border-box;-moz-box-sizing:border-box;-webkit-box-sizing:border-box}.toolbar label,.toolbar select{width:auto;vertical-align:middle;margin-bottom:0;display:inline;font-size:92%;margin-left:.3em;margin-right:.3em;padding:3px 0 0}.toolbar .btn{padding:2px 8px}.toolbar .btn-group{margin-top:0;margin-left:5px}#maintoolbar{margin-bottom:-3px;margin-top:-8px;border:0;min-height:27px;margin-left:0;padding-top:11px;padding-bottom:3px}#maintoolbar .navbar-text{float:none;vertical-align:middle;text-align:right;margin-left:5px;margin-right:0;margin-top:0}.select-xs{height:24px}@-moz-keyframes fadeOut{from{opacity:1}to{opacity:0}}@-webkit-keyframes fadeOut{from{opacity:1}to{opacity:0}}@-moz-keyframes fadeIn{from{opacity:0}to{opacity:1}}@-webkit-keyframes fadeIn{from{opacity:0}to{opacity:1}}.bigtooltip{overflow:auto;height:200px;-webkit-transition-property:height;-webkit-transition-duration:500ms;-moz-transition-property:height;-moz-transition-duration:500ms;transition-property:height;transition-duration:500ms}.smalltooltip{-webkit-transition-property:height;-webkit-transition-duration:500ms;-moz-transition-property:height;-moz-transition-duration:500ms;transition-property:height;transition-duration:500ms;text-overflow:ellipsis;overflow:hidden;height:80px}.tooltipbuttons{position:absolute;padding-right:15px;top:0;right:0}.tooltiptext{padding-right:30px}.ipython_tooltip{max-width:700px;animation:fadeOut 400ms;-webkit-animation:fadeIn 400ms;-moz-animation:fadeIn 400ms;animation:fadeIn 400ms;vertical-align:middle;background-color:#f7f7f7;overflow:visible;border:1px solid #ababab;outline:0;padding:3px 3px 3px 7px;padding-left:7px;font-family:monospace;min-height:50px;-moz-box-shadow:0 6px 10px -1px #adadad;-webkit-box-shadow:0 6px 10px -1px #adadad;box-shadow:0 6px 10px -1px #adadad;border-radius:2px;position:absolute;z-index:1000}.ipython_tooltip a{float:right}.ipython_tooltip .tooltiptext pre{border:0;border-radius:0;font-size:100%;background-color:#f7f7f7}.pretooltiparrow{left:0;margin:0;top:-16px;width:40px;height:16px;overflow:hidden;position:absolute}.pretooltiparrow:before{background-color:#f7f7f7;border:1px solid #ababab;z-index:11;content:"";position:absolute;left:15px;top:10px;width:25px;height:25px;-webkit-transform:rotate(45deg);-moz-transform:rotate(45deg);-ms-transform:rotate(45deg);-o-transform:rotate(45deg)}.terminal-app{background:#eee}.terminal-app #header{background:#fff;-webkit-box-shadow:0 0 12px 1px rgba(87,87,87,.2);box-shadow:0 0 12px 1px rgba(87,87,87,.2)}.terminal-app .terminal{float:left;font-family:monospace;color:#fff;background:#000;padding:.4em;border-radius:2px;-webkit-box-shadow:0 0 12px 1px rgba(87,87,87,.4);box-shadow:0 0 12px 1px rgba(87,87,87,.4)}.terminal-app .terminal,.terminal-app .terminal dummy-screen{line-height:1em;font-size:14px}.terminal-app .terminal-cursor{color:#000;background:#fff}.terminal-app #terminado-container{margin-top:20px}
/*# sourceMappingURL=style.min.css.map */
</style>
<style type="text/css">
.highlight .hll { background-color: #ffffcc }
.highlight { background: #f8f8f8; }
.highlight .c { color: #408080; font-style: italic } /* Comment */
.highlight .err { border: 1px solid #FF0000 } /* Error */
.highlight .k { color: #008000; font-weight: bold } /* Keyword */
.highlight .o { color: #666666 } /* Operator */
.highlight .cm { color: #408080; font-style: italic } /* Comment.Multiline */
.highlight .cp { color: #BC7A00 } /* Comment.Preproc */
.highlight .c1 { color: #408080; font-style: italic } /* Comment.Single */
.highlight .cs { color: #408080; font-style: italic } /* Comment.Special */
.highlight .gd { color: #A00000 } /* Generic.Deleted */
.highlight .ge { font-style: italic } /* Generic.Emph */
.highlight .gr { color: #FF0000 } /* Generic.Error */
.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */
.highlight .gi { color: #00A000 } /* Generic.Inserted */
.highlight .go { color: #888888 } /* Generic.Output */
.highlight .gp { color: #000080; font-weight: bold } /* Generic.Prompt */
.highlight .gs { font-weight: bold } /* Generic.Strong */
.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */
.highlight .gt { color: #0044DD } /* Generic.Traceback */
.highlight .kc { color: #008000; font-weight: bold } /* Keyword.Constant */
.highlight .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */
.highlight .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */
.highlight .kp { color: #008000 } /* Keyword.Pseudo */
.highlight .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */
.highlight .kt { color: #B00040 } /* Keyword.Type */
.highlight .m { color: #666666 } /* Literal.Number */
.highlight .s { color: #BA2121 } /* Literal.String */
.highlight .na { color: #7D9029 } /* Name.Attribute */
.highlight .nb { color: #008000 } /* Name.Builtin */
.highlight .nc { color: #0000FF; font-weight: bold } /* Name.Class */
.highlight .no { color: #880000 } /* Name.Constant */
.highlight .nd { color: #AA22FF } /* Name.Decorator */
.highlight .ni { color: #999999; font-weight: bold } /* Name.Entity */
.highlight .ne { color: #D2413A; font-weight: bold } /* Name.Exception */
.highlight .nf { color: #0000FF } /* Name.Function */
.highlight .nl { color: #A0A000 } /* Name.Label */
.highlight .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */
.highlight .nt { color: #008000; font-weight: bold } /* Name.Tag */
.highlight .nv { color: #19177C } /* Name.Variable */
.highlight .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */
.highlight .w { color: #bbbbbb } /* Text.Whitespace */
.highlight .mb { color: #666666 } /* Literal.Number.Bin */
.highlight .mf { color: #666666 } /* Literal.Number.Float */
.highlight .mh { color: #666666 } /* Literal.Number.Hex */
.highlight .mi { color: #666666 } /* Literal.Number.Integer */
.highlight .mo { color: #666666 } /* Literal.Number.Oct */
.highlight .sb { color: #BA2121 } /* Literal.String.Backtick */
.highlight .sc { color: #BA2121 } /* Literal.String.Char */
.highlight .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */
.highlight .s2 { color: #BA2121 } /* Literal.String.Double */
.highlight .se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */
.highlight .sh { color: #BA2121 } /* Literal.String.Heredoc */
.highlight .si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */
.highlight .sx { color: #008000 } /* Literal.String.Other */
.highlight .sr { color: #BB6688 } /* Literal.String.Regex */
.highlight .s1 { color: #BA2121 } /* Literal.String.Single */
.highlight .ss { color: #19177C } /* Literal.String.Symbol */
.highlight .bp { color: #008000 } /* Name.Builtin.Pseudo */
.highlight .vc { color: #19177C } /* Name.Variable.Class */
.highlight .vg { color: #19177C } /* Name.Variable.Global */
.highlight .vi { color: #19177C } /* Name.Variable.Instance */
.highlight .il { color: #666666 } /* Literal.Number.Integer.Long */
</style>
<style type="text/css">
/* Overrides of notebook CSS for static HTML export */
body {
overflow: visible;
padding: 8px;
}
div#notebook {
overflow: visible;
border-top: none;
}
@media print {
div.cell {
display: block;
page-break-inside: avoid;
}
div.output_wrapper {
display: block;
page-break-inside: avoid;
}
div.output {
display: block;
page-break-inside: avoid;
}
}
</style>
<!-- Custom stylesheet, it must be in the same directory as the html file -->
<link rel="stylesheet" href="custom.css">
<!-- Loading mathjax macro -->
<!-- Load mathjax -->
<script src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML"></script>
<!-- MathJax configuration -->
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
tex2jax: {
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
processEscapes: true,
processEnvironments: true
},
// Center justify equations in code and markdown cells. Elsewhere
// we use CSS to left justify single line equations in code cells.
displayAlign: 'center',
"HTML-CSS": {
styles: {'.MathJax_Display': {"margin": 0}},
linebreaks: { automatic: true }
}
});
</script>
<!-- End of mathjax configuration --></head>
<body>
<div tabindex="-1" id="notebook" class="border-box-sizing">
<div class="container" id="notebook-container">
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>In two <a href="http://netprophetblog.blogspot.com/2015/09/massey-example.html">previous</a> <a href="http://netprophetblog.blogspot.com/2015/09/calculating-massey-rating-part-2.html">postings</a>, I took a look at Massey ratings and showed how to calculate the rating in Python. In this posting, we'll create a more complex version of the Massey rating and show how to calculate it.</p>
<p>Recall that the basic notion of the Massey rating is that the expected outcome of a game (in terms of margin of victory) is equal to the difference in ratings between the two teams:</p>
<p><p></p>
<p><center>$e(MOV_{ij}) = R_i - R_j$</center></p>
<p><p>
So in some sense the rating $R$ is the "strength" of a team, and the stronger team is expected to win by about the difference in ratings between the two teams. An obvious extension of this approach is to think of teams as having not just one strength rating, but two: one for their offensive strength and one for their defensive strength. We expect a team's score to be it's offensive rating minus the defensive rating of its opponent:</p>
<p><p></p>
<p><center>$e(Score_{ij}) = O_i - D_j$</center></p>
<p><p>
There's one more wrinkle to add before we jump into modeling this rating. Since we're now looking at scores instead of margin of victory, we have to account for the home court advantage. To do this, we'll use two equations, one for the home team and one for the away team:</p>
<p><p>
$$e(Score_h) = O_h - D_a + HCA$$</p>
<p><p>
$$e(Score_a) = O_a - D_h$$</p>
<p><p>
The home team's score is expected to be boosted by the $HCA$ while the away team is not.</p>
<p>Given some game outcomes, how can we calculate the team ratings? Let's start with some game data.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [2]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="n">games</span> <span class="o">=</span> <span class="p">[[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">83</span><span class="p">,</span> <span class="mi">80</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">91</span><span class="p">,</span> <span class="mi">70</span><span class="p">],</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">80</span><span class="p">,</span> <span class="mi">85</span><span class="p">],</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">73</span><span class="p">,</span> <span class="mi">60</span><span class="p">],</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">73</span><span class="p">,</span> <span class="mi">60</span><span class="p">],</span>
<span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">85</span><span class="p">,</span> <span class="mi">60</span><span class="p">],</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">70</span><span class="p">,</span> <span class="mi">78</span><span class="p">],</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">85</span><span class="p">,</span> <span class="mi">70</span><span class="p">],</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">91</span><span class="p">,</span> <span class="mi">70</span><span class="p">],</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">60</span><span class="p">,</span> <span class="mi">68</span><span class="p">]]</span>
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Here I've invented data for ten games between six teams. Each game is represented by four elements:</p>
$$[Team_h, Team_a, Score_h, Score_a]$$<p>From each of these games we can generate two equations corresponding to the equations above. For example, for the first game:</p>
$$83 = O_0 - D_3 + HCA$$$$80 = O_3 - D_0$$<p>Altogether we'll have twenty equations. To represent each equation, we'll use a row that looks like this:</p>
$$[O_0, O_1, O_2, ... D_0, D_1, D_2, ... HCA]$$<p>To represent an equation, we place a 1 in the spot for the Offensive rating, a -1 in the spot for a Defensive rating, and a 1 in the HCA spot if this is the home team's score. Everything else is a zero. So the first game we get the following two rows:</p>
$$[ 1, 0, 0, 0, 0, 0, 0, 0, 0, -1, 0, 0, 1.]$$$$[ 0, 0, 0, 1, 0, 0, -1, 0, 0, 0, 0, 0, 0.]$$<p>Do this for all the games and we end up with a 20x13 matrix. Here's the code to create that matrix:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [29]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="kn">import</span> <span class="nn">numpy</span> <span class="kn">as</span> <span class="nn">np</span>
<span class="k">def</span> <span class="nf">buildGamesMatrix2</span><span class="p">(</span><span class="n">games</span><span class="p">,</span> <span class="n">num_teams</span><span class="p">):</span>
<span class="n">l</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">games</span><span class="p">)</span>
<span class="n">M</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">([</span><span class="n">l</span><span class="o">*</span><span class="mi">2</span><span class="p">,</span> <span class="n">num_teams</span><span class="o">*</span><span class="mi">2</span><span class="o">+</span><span class="mi">1</span><span class="p">])</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="n">l</span><span class="p">):</span>
<span class="n">g</span> <span class="o">=</span> <span class="n">games</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
<span class="n">M</span><span class="p">[</span><span class="n">i</span><span class="p">,</span> <span class="n">g</span><span class="p">[</span><span class="mi">0</span><span class="p">]]</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="n">M</span><span class="p">[</span><span class="n">i</span><span class="p">,</span> <span class="n">g</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">+</span><span class="n">num_teams</span><span class="p">]</span> <span class="o">+=</span> <span class="o">-</span><span class="mi">1</span>
<span class="n">M</span><span class="p">[</span><span class="n">i</span><span class="p">,</span> <span class="n">num_teams</span><span class="o">*</span><span class="mi">2</span><span class="p">]</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="n">M</span><span class="p">[</span><span class="n">l</span><span class="o">+</span><span class="n">i</span><span class="p">,</span> <span class="n">g</span><span class="p">[</span><span class="mi">1</span><span class="p">]]</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="n">M</span><span class="p">[</span><span class="n">l</span><span class="o">+</span><span class="n">i</span><span class="p">,</span> <span class="n">g</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">+</span><span class="n">num_teams</span><span class="p">]</span> <span class="o">+=</span> <span class="o">-</span><span class="mi">1</span>
<span class="k">return</span> <span class="n">M</span>
<span class="n">M2</span> <span class="o">=</span> <span class="n">buildGamesMatrix2</span><span class="p">(</span><span class="n">games</span><span class="p">,</span><span class="mi">6</span><span class="p">)</span>
<span class="k">print</span> <span class="n">M2</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>[[ 1. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. 0. 1.]
[ 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. 1.]
[ 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 1.]
[ 0. 0. 1. 0. 0. 0. -1. 0. 0. 0. 0. 0. 1.]
[ 0. 0. 1. 0. 0. 0. 0. 0. 0. -1. 0. 0. 1.]
[ 0. 0. 0. 1. 0. 0. -1. 0. 0. 0. 0. 0. 1.]
[ 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. -1. 1.]
[ 0. 0. 0. 0. 1. 0. 0. 0. 0. -1. 0. 0. 1.]
[ 0. 0. 0. 0. 0. 1. 0. -1. 0. 0. 0. 0. 1.]
[ 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. -1. 0. 1.]
[ 0. 0. 0. 1. 0. 0. -1. 0. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 1. 0. 0. -1. 0. 0. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 1. 0. -1. 0. 0. 0. 0. 0.]
[ 1. 0. 0. 0. 0. 0. 0. 0. -1. 0. 0. 0. 0.]
[ 0. 0. 0. 1. 0. 0. 0. 0. -1. 0. 0. 0. 0.]
[ 1. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0. 0. 0.]
[ 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. -1. 0. 0.]
[ 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. -1. 0. 0.]
[ 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. -1. 0.]
[ 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. -1. 0.]]
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The way I built this array all the home teams are first, but the order is irrelevant. The next thing we need is a corresponding array with the 20 scores. Here's the code to build that:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [4]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="k">def</span> <span class="nf">buildOutcomes2</span><span class="p">(</span><span class="n">games</span><span class="p">,</span> <span class="n">n</span><span class="p">):</span>
<span class="n">l</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">games</span><span class="p">)</span>
<span class="n">E</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">([</span><span class="n">l</span><span class="o">*</span><span class="mi">2</span><span class="p">])</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="n">l</span><span class="p">):</span>
<span class="n">g</span> <span class="o">=</span> <span class="n">games</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
<span class="n">E</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">+=</span> <span class="n">g</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
<span class="n">E</span><span class="p">[</span><span class="n">l</span><span class="o">+</span><span class="n">i</span><span class="p">]</span> <span class="o">+=</span> <span class="n">g</span><span class="p">[</span><span class="mi">3</span><span class="p">]</span>
<span class="k">return</span> <span class="n">E</span>
<span class="n">E2</span> <span class="o">=</span> <span class="n">buildOutcomes2</span><span class="p">(</span><span class="n">games</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span>
<span class="k">print</span> <span class="n">E2</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>[ 83. 91. 80. 73. 73. 85. 70. 85. 91. 60. 80. 70. 85. 60. 60.
60. 78. 70. 70. 68.]
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Again, this looks like a row, but Python is happy to treat it as a column.</p>
<p>The full set of equations can be written as the matrix equation:</p>
$$M * R = E$$<p>The problem is to solve this equation for $R$, the vector of team ratings. Since this is a set of linear equations, we can use Numpy least squares solver to find the answer:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [5]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="n">R</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linalg</span><span class="o">.</span><span class="n">lstsq</span><span class="p">(</span><span class="n">M2</span><span class="p">,</span><span class="n">E2</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">print</span> <span class="n">R</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>[ 29.45226244 47.04230769 24.77895928 36.20520362 35.44954751
38.06945701 -42.32149321 -42.49615385 -27.17126697 -40.27443439
-31.4418552 -27.29253394 6.92307692]
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The vector is the team ratings. For example, Team 0's offensive rating is 29.45226244 and its defensive rating is -42.32149321. The last term is the HCA. Let's make a little pretty print function to make it easier to examine:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [6]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="k">def</span> <span class="nf">ppResults</span><span class="p">(</span><span class="n">r</span><span class="p">,</span> <span class="n">n</span><span class="p">,</span> <span class="n">M</span><span class="p">,</span> <span class="n">E</span><span class="p">):</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="k">print</span> <span class="s">"Team[</span><span class="si">%d</span><span class="s">]: (</span><span class="si">%0.2f</span><span class="s">, </span><span class="si">%0.2f</span><span class="s">)"</span> <span class="o">%</span> <span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">r</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">r</span><span class="p">[</span><span class="n">i</span><span class="o">+</span><span class="n">n</span><span class="p">])</span>
<span class="k">print</span> <span class="s">"HCA: </span><span class="si">%0.2f</span><span class="s"> RMSE: </span><span class="si">%0.2f</span><span class="s"> MAE: </span><span class="si">%0.2f</span><span class="s">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">r</span><span class="p">[</span><span class="n">n</span><span class="o">*</span><span class="mi">2</span><span class="p">],</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">((</span><span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">M</span><span class="p">,</span><span class="n">r</span><span class="p">)</span><span class="o">-</span><span class="n">E</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span><span class="o">**</span><span class="mf">0.5</span><span class="p">,</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">abs</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">M</span><span class="p">,</span><span class="n">r</span><span class="p">)</span><span class="o">-</span><span class="n">E</span><span class="p">)))</span>
<span class="n">ppResults</span><span class="p">(</span><span class="n">R</span><span class="p">,</span><span class="mi">6</span><span class="p">,</span> <span class="n">M2</span><span class="p">,</span> <span class="n">E2</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>Team[0]: (29.45, -42.32)
Team[1]: (47.04, -42.50)
Team[2]: (24.78, -27.17)
Team[3]: (36.21, -40.27)
Team[4]: (35.45, -31.44)
Team[5]: (38.07, -27.29)
HCA: 6.92 RMSE: 5.87 MAE: 4.45
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>I've also shown a couple of error measures, based upon the difference between the actual results ($E$) and the predicted results ($M*R$). You'll also note that the defensive ratings are a little strange. Since they're negative, they're actually mean something like "the number of points this team adds to the other team's score." The math doesn't care, but if it bothers us we can adjust the scores so that the defensive ratings are all positive:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [30]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="n">R</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linalg</span><span class="o">.</span><span class="n">lstsq</span><span class="p">(</span><span class="n">M2</span><span class="p">,</span><span class="n">E2</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">correction</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">min</span><span class="p">(</span><span class="n">R</span><span class="p">[</span><span class="mi">6</span><span class="p">:</span><span class="mi">12</span><span class="p">])</span>
<span class="n">R</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">12</span><span class="p">]</span> <span class="o">-=</span> <span class="n">correction</span>
<span class="n">ppResults</span><span class="p">(</span><span class="n">R</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="n">M2</span><span class="p">,</span> <span class="n">E2</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>Team[0]: (71.95, 0.17)
Team[1]: (89.54, 0.00)
Team[2]: (67.28, 15.32)
Team[3]: (78.70, 2.22)
Team[4]: (77.95, 11.05)
Team[5]: (80.57, 15.20)
HCA: 6.92 RMSE: 5.87 MAE: 4.45
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>(Note that we have to be careful not to "correct" the HCA term in R[12]!) Now it is more apparent that Teams 2 and 5 play very good defense, and Teams 0 and 1 not so much.</p>
<p>We can also solve this problem using the LinearRegression function from SciKit Learn:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [13]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="kn">from</span> <span class="nn">sklearn</span> <span class="kn">import</span> <span class="n">linear_model</span>
<span class="n">clf</span> <span class="o">=</span> <span class="n">linear_model</span><span class="o">.</span><span class="n">LinearRegression</span><span class="p">(</span><span class="n">fit_intercept</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">clf</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">M2</span><span class="p">,</span><span class="n">E2</span><span class="p">)</span>
<span class="n">correction</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">min</span><span class="p">(</span><span class="n">clf</span><span class="o">.</span><span class="n">coef_</span><span class="p">[</span><span class="mi">6</span><span class="p">:</span><span class="mi">12</span><span class="p">])</span>
<span class="n">clf</span><span class="o">.</span><span class="n">coef_</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">12</span><span class="p">]</span> <span class="o">-=</span> <span class="n">correction</span>
<span class="n">ppResults</span><span class="p">(</span><span class="n">clf</span><span class="o">.</span><span class="n">coef_</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="n">M2</span><span class="p">,</span> <span class="n">E2</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>Team[0]: (71.95, 0.17)
Team[1]: (89.54, 0.00)
Team[2]: (67.28, 15.32)
Team[3]: (78.70, 2.22)
Team[4]: (77.95, 11.05)
Team[5]: (80.57, 15.20)
HCA: 6.92 RMSE: 5.87 MAE: 4.45
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>This generates exactly the same answer. Which shouldn't be surprising, because under the covers the LinearRegression function calls the least squares solver. The advantage of doing it this way is that we can easily swap out the LinearRegression for a different model, for example, a Ridge regression:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [21]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="kn">from</span> <span class="nn">sklearn</span> <span class="kn">import</span> <span class="n">linear_model</span>
<span class="n">clf</span> <span class="o">=</span> <span class="n">linear_model</span><span class="o">.</span><span class="n">Ridge</span><span class="p">(</span><span class="n">fit_intercept</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=</span><span class="mf">0.1</span><span class="p">)</span>
<span class="n">clf</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">M2</span><span class="p">,</span><span class="n">E2</span><span class="p">)</span>
<span class="n">correction</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">min</span><span class="p">(</span><span class="n">clf</span><span class="o">.</span><span class="n">coef_</span><span class="p">[</span><span class="mi">6</span><span class="p">:</span><span class="mi">12</span><span class="p">])</span>
<span class="n">clf</span><span class="o">.</span><span class="n">coef_</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">12</span><span class="p">]</span> <span class="o">-=</span> <span class="n">correction</span>
<span class="n">ppResults</span><span class="p">(</span><span class="n">clf</span><span class="o">.</span><span class="n">coef_</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="n">M2</span><span class="p">,</span> <span class="n">E2</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>Team[0]: (71.63, 2.81)
Team[1]: (85.36, 0.00)
Team[2]: (65.39, 16.40)
Team[3]: (78.62, 4.20)
Team[4]: (75.55, 11.07)
Team[5]: (78.16, 14.12)
HCA: 9.95 RMSE: 6.10 MAE: 4.69
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>As you can see, this returns slightly different results. You can read about Ridge regression in detail <a href="https://tamino.wordpress.com/2011/02/12/ridge-regression/">elsewhere</a>, but the gist is that it addresses some problems in least squares regression that often occur in real-world data. It's not important in this toy example, but if you actually want to calculate this rating for NCAA teams, it can be useful.</p>
<p>There's yet another approach we can use to solve this problem: <a href="https://en.wikipedia.org/wiki/Stochastic_gradient_descent">stochastic gradient descent</a> (SGD). SGD is a method for finding the minimum of a function. For example, consider the following function (taken from <a href="http://www.scipy-lectures.org/intro/scipy.html#optimization-and-fit-scipy-optimize">here</a>):</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [22]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="kn">from</span> <span class="nn">scipy</span> <span class="kn">import</span> <span class="n">optimize</span>
<span class="kn">import</span> <span class="nn">pylab</span> <span class="kn">as</span> <span class="nn">plt</span>
<span class="o">%</span><span class="k">matplotlib</span> inline
<span class="kn">import</span> <span class="nn">numpy</span> <span class="kn">as</span> <span class="nn">np</span>
<span class="k">def</span> <span class="nf">f</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="k">return</span> <span class="n">x</span><span class="o">**</span><span class="mi">2</span> <span class="o">+</span> <span class="mi">10</span><span class="o">*</span><span class="n">np</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="o">-</span><span class="mi">10</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mf">0.1</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz
AAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm81fO+x/HXpxIiKseQQnHKKUPmk8NlG0qmOKaMlZmu
2aF2XPYxNqAMh8NFN0MRGXKR4rSKwxWVpDSpfU6REGXIsGt/7x+ftdm22u29pt9av/V+Ph770Vq/
vX6/36fd6rO/6zt8vhZCQERE4qlB1AGIiEj2KMmLiMSYkryISIwpyYuIxJiSvIhIjCnJi4jEWJ2S
vJk9bGZLzWxGtWODzexDM5tuZs+Y2abVvldqZvPMbLaZdc1G4CIism51bckPA7rVODYO2CmE0AmY
C5QCmFlHoAfQMXnOvWamTwwiIhGoU/INIbwOfFXj2PgQQmXy6dtA6+TjY4CRIYSKEEI5MB/YJzPh
iohIfWSqhX0W8FLy8dbA4mrfWwy0ytB9RESkHtJO8mZ2DfBTCGFELS9T7QQRkQg0SudkM+sNHAEc
Uu3wx8A21Z63Th6rea4Sv4hICkIIVtfXptySN7NuwFXAMSGEH6p9awxwspk1NrO2QDtg8loC1VeG
vq6//vrIY4jTl36e+nnm61d91aklb2YjgQOB35nZIuB6fDZNY2C8mQG8FULoE0KYZWajgFnAKqBP
SCUyERFJW52SfAjhlDUcfriW198C3JJqUCIikhmavx4TJSUlUYcQK/p5ZpZ+ntGxqHpSzEy9OCIi
9WRmhFwMvIqISP5TkhcRiTEleRGRGFOSFxGJMSV5EZEYU5IXEYkxJXkRkRhTkhcRiTEleRGRGFOS
FxGJMSV5EZEYU5IXEYkxJXkRkRhTkhcRiTEleRGRGFOSFxGJsUiTfGVllHcXEYm/SJP8pElR3l1E
JP4iTfKPPhrl3UVE4i/SPV6bNQt88glsuGEkIYiIFJyC2uN1r71gzJgoIxARibc6JXkze9jMlprZ
jGrHWpjZeDOba2bjzKxZte+Vmtk8M5ttZl3Xdt0zzlCXjYhINtW1JT8M6FbjWD9gfAihPfBa8jlm
1hHoAXRMnnOvma3xPscdB2+8AZ99lkroIiKyLnVK8iGE14GvahzuDgxPPh4OHJt8fAwwMoRQEUIo
B+YD+6zpuhtvDEcdBU88Ud+wRUSkLtLpk98yhLA0+XgpsGXy8dbA4mqvWwy0WttF1GUjIpI9jTJx
kRBCMLPapums8XtlZWVUVsLs2fDIIyX07FmSiXBERGIjkUiQSCRSPr/OUyjNrA3wQghhl+Tz2UBJ
COFTM2sJTAgh/MHM+gGEEAYkXzcWuD6E8HaN64Wqe//lL7D++nDzzSn/PUREikIup1COAXolH/cC
nqt2/GQza2xmbYF2wOTaLnTGGfDYYypzICKSaXWdQjkSeBPY0cwWmdmZwACgi5nNBQ5OPieEMAsY
BcwCXgb6hHV8XOjUCTbdFF5/PfW/iIiI/FakK16r33vwYO+bf+ihSMIRESkI9e2uyZsk//HHsPPO
qMyBiEgtCqqsQXWtWqnMgYhIpuVNkgfNmRcRybS86a4B+PZbaN0a5s6FLbaIJCwRkbxWsN01oDIH
IiKZlldJHtRlIyJS06pVUFYGK1bU/9y8S/KHHAKLF/t0ShGRYrdqFfTsCW+95ZUB6ivvknyjRnDq
qWrNi4hUVHjvxhdfwHPPwQYb1P8aeTXwWuW99+CYY2DhQmiQd7+GRESyb+VKOPFEaNgQnnzyl/VD
BT3wWkVlDkSkmK1YAd26QfPmMHp0egtE8zLJm2kAVkSK0yefwEEHwa67wiOPwHrrpXe9vOyuAZU5
EJHiM3Wqd1VfeCGUlnqDt6ZYdNeAyhyISHF55hk47DAYOhT6919zgk9F3iZ5UJeNiMRfCHDrrXDp
pTB2LBx/fGavn7fdNaAyByISbz/+COedBx984L0Wrda6G/YvYtNdA17moHt3GDEi6khERDLr8899
8ee338KkSXVL8KnI6yQP0KsXDB8edRQiIpkzcyb88Y9w4IHw1FOw0UbZu1feJ/mDDoJly+D996OO
REQkfePGeV7761/h5puzv+Az75N8gwY+AKvWvIgUugce8Do0zzzjeS0X8nrgtcqcOf6xZvFir20j
IlJIKit93vuzz8KLL0K7dqlfK1YDr1V23BHatoVXXok6EhGR+vn+e+jRA9580ytJppPgU1EQSR40
ACsihefzz+Hgg6FxY3j1Vdhss9zHUBDdNQBffeWt+YULvWiPiEg+W7wYunSB446Dm27K3ArWnHfX
mFmpmc00sxlmNsLM1jezFmY23szmmtk4M2uW7n2aN4euXb3kpohIPvvoIzjgADjrLJ9Bk6kEn4q0
kryZtQHOBfYIIewCNAROBvoB40MI7YHXks/Tpi4bEcl3M2f6RJGrr4arroo6mvRb8l8DFUATM2sE
NAE+AboDVel4OHBsmvcBvHjPwoU+20ZEJN/Mnu1dNAMGwAUXRB2NSyvJhxC+BG4H/o0n9+UhhPHA
liGEpcmXLQW2TCvKpEaN4LTTVLRMRPLPwoWe4G+5BU4/PepofpHWrHMz2wG4DGgDrACeMrNf/fVC
CMHM1jjCWlZW9vPjkpISSkpK1nnPXr3gqKPghhu0NaCI5IfFi70OTWkp9O6d2WsnEgkSiUTK56c1
u8bMegBdQgjnJJ+fAXQGDgYOCiF8amYtgQkhhD/UOLdes2uq23NPL83ZtWvKoYuIZMSyZbDffnD2
2bnpg8/17JrZQGcz29DMDDgUmAW8APRKvqYX8Fya9/mVc86BBx/M5BVFROrvxx/hz3/2arn5MMi6
JmnPkzezq/FEXglMBc4BmgKjgG2BcuCkEMLyGuel3JJfsQLatPE685tvnnrsIiKpCsG7j7/7zitJ
5qr7uL4t+YJZDFVT796wyy5w5ZWZi0lEpK5uvNE3+pg4EZo0yd19Y1m7Zk2qumwi+h0lIkXs6ac9
/4wZk9sEn4qCTfL77ed//vOf0cYhIsVlzhy48EKvKNmyZdTRrFvBJnkzDcCKSG6tXAknnOC1aPbY
I+po6qZg++TBK7y1awfl5dAs7eo4IiJrFwKceSasWuULMqOqR1M0ffLgM2u6doWRI6OORETibtgw
eOcd+Pvfoy04Vl8F3ZIH3y+xtBSmTMlAUCIiazB/PnTuDJMmQceO0cZSVC15gEMP9RVnU6dGHYmI
xNHq1T5l+9pro0/wqSj4JN+ggS8n1gCsiGTD7bfDeuvBJZdEHUlqCr67Brw40K67wqJFsNFGGbmk
iAgzZvj2fe++C9ttF3U0rui6awBat/ZdWB5/POpIRCQuKiqgZ08YODB/EnwqYpHkAS66CO65Rytg
RSQz7rgDttrKp00Wslh014An944d4f77vVUvIpKqhQth771h8mTYfvuoo/m1ouyuAZ+3WtWaFxFJ
VQieS668Mv8SfCpi05IH+OYb7zubMQNatcropUWkSIweDdddB9OmQePGUUfzW0Xbkgdo2tT3gL3/
/qgjEZFC9PXXcNllvqo1HxN8KmLVkgffLb2kBP71L1h//YxfXkRi7C9/gS+/hIcfjjqStSuaTUNq
06WLT30644ysXF5EYmjePNh3X5g5E7bcMupo1q6ou2uqXH65r1LTdEoRqaurr/aWfD4n+FTEMskf
frgvZHjttagjEZFCMGECvPee98fHTSyTvJn/Rh48OOpIRCTfrV7tn/4HD4YNNog6msyLZZIHOPVU
+OADmD496khEJJ8NGwabbALHHx91JNkRy4HXKgMHeqJ/9NGs3kZECtTKlb673PPPw157RR1N3Wh2
TTXLl/uKtenTYZttsnorESlAAwb4XhSjRkUdSd3lPMmbWTPgQWAnIABnAvOAJ4HtgHLgpBDC8hrn
ZT3JA1xxhdecv+22rN9KRArIV19B+/bwxhuw445RR1N3UST54cDEEMLDZtYI2Ai4BvgihDDIzPoC
zUMI/Wqcl5Mkv2gR7LabL5LafPOs305ECkRpKXzxBfz3f0cdSf3kNMmb2abAtBDC9jWOzwYODCEs
NbOtgEQI4Q81XpOTJA/Qp4+XPBg4MCe3E5E8t2QJ7LQTvP++70dRSHKd5HcD7gdmAZ2AKcBlwOIQ
QvPkawz4sup5tXNzluQXL4ZOneDDD2GLLXJySxHJY336QJMmhdmNW98k3yjN+zUC9gAuCiG8Y2ZD
gV91y4QQgpmtMZuXlZX9/LikpISSkpI0w1mz1q3hlFN8HqzmzosUtwUL4MknYc6cqCOpm0QiQSKR
SPn8dFvyWwFvhRDaJp/vD5QC2wMHhRA+NbOWwIQou2vgl31gP/wwfsuWRaTuzjkHtt4abrgh6khS
k9PaNSGET4FFZtY+eehQYCbwAtAreawX8Fw698mE1q29DLFa8iLFa8ECeO65eJYvWJtMzK7phE+h
bAx8hE+hbAiMArYl4imU1X38Meyyiy+Q2nrrnN5aRPLAOedAy5Zw441RR5I6LYZah759fdrUQw/l
/NYiEqEFC3zf1nnzoEWLqKNJnZL8OqxY4QsfXn4Zdt8957cXkYjEoRUPSvJ1ct998NRTXorY6vyj
EpFCFZdWPGjTkDo591xYuhReeCHqSEQkFwYNggsuKPwEn4qibMkDjB0Ll1zig7Bx2bBXRH5ryRLo
2NHnxcdhMaRa8nXUrZtXqLzrrqgjEZFsuvNOnz4dhwSfiqJtycMvG/dOnuwJX0TiZcUK/7/97rvQ
tm3U0WSGWvL10K6db957/vna9Fskjv7+d9/zOS4JPhVF3ZIHWLUK/vhHuOgiOPPMqKMRkUz54QdP
7q+84iVN4kIt+Xpq1MgXRvXt6wM0IhIPw4fDHnvEK8Gnouhb8lX69/eNRUaP1tx5kUK3erUvehw2
DP7jP6KOJrPUkk/RddfBRx+p3IFIHIwe7dVm998/6kiip5Z8NR9+CAccABMn+rxaESk8IcCee0JZ
GXTvHnU0maeWfBo6dPAtAo8/Hr75JupoRCQV48fDjz/CUUdFHUl+UEt+Dc4/3ytVPv20+udFCs0h
h0CvXtCzZ9SRZIda8hlw110+0+a666KORETqY/JkmD/ft/sUpyS/Buuv77vHjBgBDz8cdTQiUlcD
B8KVV8J660UdSf5Qd00t5syBkhL429/guOOijkZEajNnjk+XXLgQNtoo6miyp77dNY2yGUyh23FH
eOklL2bWuLEGckTy2eDB8J//Ge8Enwq15Otg8mQ4+mjvq+/RI+poRKSmqv2b582DzTaLOprsUks+
C/bZx6dlHX44fPKJ7/SuWTci+WPIEJ9RE/cEnwq15OvhX//yLpvOneHuu2GDDaKOSES++gp22AGm
T4dttok6muzTFMos2m47ePNNWL7c69DPmRN1RCJy772+srUYEnwqlOTrqWlTGDUKzjsP9tsPbrvN
yxWLSO6tXOljZVdfHXUk+SsjSd7MGprZNDN7Ifm8hZmNN7O5ZjbOzJpl4j75wgwuvNAHZMeNg512
8oJIBdb7JFLwhg3zT9WqNbV2GemTN7MrgD2BpiGE7mY2CPgihDDIzPoCzUMI/WqcU3B98msSgg/K
9u3r0yzLyuCww6CBPiOJZNWqVb6724gRnuiLRc775M2sNXAE8CBQdePuwPDk4+HAseneJ1+ZQdeu
MGUKXHopXHMN/P73MGAALF0adXQi8TVqFGy7bXEl+FRkor05BLgKqKx2bMsQQlWKWwpsmYH75LUG
DeDUUz3ZP/mkz9dt3x4OOgiGDvVVeCKSGSF4Q6pfv3W/ttilNU/ezI4CPgshTDOzkjW9JoQQzGyN
/TJlZWU/Py4pKaGkZI2XKChmsPfe/nX33fDqq/D883DLLbDJJl6vvuqrbVvNtxdJxUsv+f+dbt2i
jiT7EokEiUQi5fPT6pM3s1uAM4BVwAbAJsAzwN5ASQjhUzNrCUwIIfyhxrmx6JOvq8pKmDULXn8d
Jk3yjUkaNPBaG/vv73/uvLP68kXqYv/94aKL4OSTo44k9+rbJ5+xxVBmdiDwlxDC0cmB12UhhIFm
1g9oFteB11SF4NsNvvGGJ/7XX4fPP4c//Qm6dIEjj/RBJRH5tddfhzPP9D2ZGxXhmv2ok/yVydk1
LYBRwLZAOXBSCGF5jdcXdZJfk6VL/Q08dqx/HN14Yzj2WF+uvdNOUUcnkh+OOML/X5x3XtSRRCOy
JF9fSvK1CwHee89nEDzyCGy9tbdeeveGJk2ijk4kGu+9559yFyzwfR+KkZJ8DK1e7QO499/vZRUu
vdRLqm6ySdSRieRWjx5eMPDKK6OOJDpK8jE3cybceqt36fTr5wlfu+BIMZg3z8esFizw8iLFSkm+
SMyZAxdfDJ9+Cvfd53V0ROLs3HO92/Kvf406kmgpyReREOCpp+CKK7wE8pAhsOGGUUclknnFtCnI
uqjUcBExg5NO8vn3K1Z4nfu5c6OOSiTz7rjDJx0Ue4JPhVryMRGCD8z+13/BPfdom0KJj2XLfM3I
++9D69ZRRxM9ddcUualT4fjj4ZxzoH9/lU2QwnfttfDZZ/DAA1FHkh+U5IUlS7ymxwEHwJ13qlSC
FK5ly7zQ35Qp0KZN1NHkB/XJCy1ben2cGTPglFPgxx+jjkgkNUOG+CdTJfjUqSUfYz/84Em+ahaO
5tNLIfnyS++LVyv+19SSl59tsIHXtv/pJy+JUFm57nNE8sWQIXDccUrw6VJLvgh8/z0cfjh06OA7
22swVvJdVSv+3Xd93wX5hVry8hsbbghjxvjH3v79o45GZN3uuMMrTSrBp08t+SKybJkvmOrf37tv
RPLR0qXQsaNPB95uu6ijyT+aQim1mj3bp1aOHu27UYnkm4sv9s1AhgyJOpL8pCQv6zRunG9E8uab
+jgs+WXBAt8fefZs2HzzqKPJT+qTl3Xq2hVKS+Hoo+Hbb6OORuQX110Hl1yiBJ9JaskXqRDg7LOh
osJ3ntKMG4na9OneAJk/v7jrxa+LWvJSJ2ZeyGzaNHjooaijEfEJAf37K8FnWhHudS5VmjTxlbAH
HOBbqu26a9QRSbEaN843wnnmmagjiR+15Itchw4+i+HEE+Gbb6KORopRRQVcdpnPjS/WzbmzSX3y
AvjWahUV8D//E3UkUmzuvBNefBFeeUVjQ3WhKZSSkm+/hd13hwEDvOqfSC58/rkvfJo40f+Udcvp
wKuZbWNmE8xsppl9YGaXJI+3MLPxZjbXzMaZWbN07iPZt/HG8Nhj0KeP16MXyYVrr4XTTlOCz6a0
WvJmthWwVQjhPTPbGJgCHAucCXwRQhhkZn2B5iGEfjXOVUs+D11/PUyeDC+9pI/Okl1Tp3rhvNmz
oXnzqKMpHDltyYcQPg0hvJd8/C3wIdAK6A4MT75sOJ74pQBce63XuLn33qgjkTirqPB1GoMHK8Fn
W8b65M2sDTAR2Bn4dwihefK4AV9WPa/2erXk89TcufCnP3mLfvvto45G4ujmm+GNN/SJMRX1bcln
ZJ58sqtmNHBpCOEbq/avFkIIZrbGbF5WVvbz45KSEkpKSjIRjqSpfXvo1883A3/tNf0nlMyaNcun
7U6dqvdWXSQSCRKJRMrnp92SN7P1gP8FXg4hDE0emw2UhBA+NbOWwIQQwh9qnKeWfB5bvdpb82ef
DeedF3U0EherV3v109NP90F+qb9cz64x4CFgVlWCTxoD9Eo+7gU8l859JPcaNoSHH4ZrroFFi6KO
RuLirru8jPAFF0QdSfFId3bN/sAk4H2g6kKlwGRgFLAtUA6cFEJYXuNcteQLwE03eUniF1/UR2tJ
z7vvwhFHwFtvwQ47RB1N4dJiKMmoigqva3P55dCzZ9TRSKFavhz23NMX2514YtTRFDYlecm4adPg
sMO8FGzLllFHI4UmBDjhBH/v3HNP1NEUPpUalozbfXevbdOnj/+HFamPe+6B8nK4/faoIylOaslL
nfz4I+y2G9x4o7fKROpiwgTo0UP98Jmk7hrJmn/+0/tTP/gAWrSIOhrJd7Nnw4EHwsiRcPDBUUcT
H0ryklUXX+wVK4cNizoSyWf//rdvRlNWBr17Rx1NvCjJS1Z98w3ssgs88IDvxymZ8e233vItL/fy
u99/78c32wx+9zvYYgvYcUfYZJNIw6yTJUu8Bd+nj28GIpmlJC9ZN3YsXHghzJjhJYql/ioqIJGA
MWP8zwULvJxE27ae0Js08UHuZcs86S9d6tvjtWoFe+zhq5EPPxzatYv6b/Jr5eXQpQucdRaUlkYd
TTwpyUtO9Ozp/fJDh677tfKLxYt91ecjj0CbNnDssXDIIT6ovd56tZ+7apUn+ilTfJONl1/2XwZH
HgknnwydO0e7YG3qVDjmGLj6au/Wk+xQkpecWLYMdt4Znn3Wk4vUbvFiXwg0cqT3UZ9/vrfc0xEC
vP8+PP88jBgBP/0Ep57qm3B06JCRsOvsiSc8sd93n2ZfZZuSvOTMk0/CDTd4C04bMK/ZypU++Pjg
g17s7aqrvDsm00LwRWsjRvgvki22gDPO8KS/1VaZv1+Vr7/2fvc33vD3w+67Z+9e4rQYSnLmpJPg
97+HW2+NOpL89OqrPkj98cdeXnfw4OwkePBumj32gNtu85ktt9/uYyYdOkC3bvD44/Ddd5m7X2Wl
X7NDB+9mmjpVCT5fqSUvafn4Y+9P/sc/PKGJz5S55BL/mdx3nw+QRmXlSu/OefRRLzR3yCHeb37k
kT5zJ5XrPfUUDBrkg+5Dh8K++2Y+blk7dddIzj3wADz0kCeRhg2jjiZas2Z5n3TnznDnndC0adQR
/eKLL7ya6HPP+WYw7dv7LJ199/XiYW3aQOPGvz5n1SrfKeydd2DcOB/s7dzZf4kddpgqk0ZBSV5y
rrLSW4jdu3u1ymL12GP+9x80CM48M+poavfDDz5L5803veTA9Ok+ONy0KWy6qffxf/cdfPmlT+vc
fXc49FAvFdyqVdTRFzcleYnE/PnewivGfWFXr/Zpgy+8AE8/DbvuGnVEqVm1ypP61197C71JE9h8
c9/kQ/KHkrxEZvBgeOUVGD++eD7Gf/edT1lcsQKeeQaaN1/3OSLp0Owaiczll/vmEMVS12bJEigp
gWbN/JebErzkIyV5yZhGjXwAtl8/n8YXZwsWwH77wdFH+y+1mgOWIvlCSV4yqlMnuOIK6NXLB2Tj
aOZMr7B41VVw3XXF0zUlhUlJXjLuqqt8MPKOO6KOJPPeecdnEg0c6EXaRPKdBl4lK8rLYe+9fdVn
p05RR5MZEyf6pikPPujTRUWioIFXyQtt2vgS+9NO8znZhe6llzzBjxypBC+FRS15yZoQ4JRTfHHN
/fdHHU3qRo/2DTCef14VNyV6edOSN7NuZjbbzOaZWd9s3Ufyl5mXPEgkvHZKIXr8cbjoIp8iqQQv
hSgrLXkzawjMAQ4FPgbeAU4JIXxY7TVqyReJGTN8I+dEAnbaKepo6u7BB71M8Lhx0LFj1NGIuHxp
ye8DzA8hlIcQKoAngGOydC/Jc7vs4qthTzjBKzQWgrvvhhtvhAkTlOClsGUrybcCFlV7vjh5TIpU
796+eKhnz/yfPz9woFeQnDQp//ZQFamvbJUeqlM/TFlZ2c+PS0pKKCkpyVI4kg/+9jfo2tWLed12
W9TR/FYI3j0zapRPl1S1RckHiUSCRCKR8vnZ6pPvDJSFELoln5cClSGEgdVeoz75IvTll17D/NJL
82sxUWWl195JJLzAWrZ2cBJJV3375LPVkn8XaGdmbYBPgB7AKVm6lxSQFi1844r994fttvP65FH7
4QfvRvrsM2/BN2sWdUQimZOVPvkQwirgIuAVYBbwZPWZNVLcdtjBy/L27u0bQEdp+XLfAxVg7Fgl
eIkfLYaSyIwf7ytin38+mn1C586FY4+FLl1gyBBooPXfUgDyZQqlyDp16QKPPOIbS7/6am7v/b//
611Gl13mM2mU4CWu1JKXyE2a5HPohwzxln02rV4NN9/sK3GfeiqaTxAi6ciXgVeROjvgAPjHP7zw
14wZcNNN2dlXdO5c32B7vfV8L9qtt878PUTyjT6kSl7YeWd4+22YMsW31Csvz9y1Kyth6FCfutmj
h/9CUYKXYqEkL3lj8829ENgxx8Bee8GgQfDTT+ld87XX/FqjR8P//R9ccon636W4qE9e8tL8+T4o
OmMGlJbC6afDxhvX7dyffoIXXvCdqT77DG65xfv8tU2fxEF9++SV5CWvvfWW15KZOBGOPBIOO8x3
nNphB+9bB6iogI8+gmnTfJbOmDHQoYOXCD7+eGjYMNq/g0gmKclLLC1Z4sl7/HiYPt377DfYwFvn
K1f66tldd/WSxkceCdtvH3XEItmhJC9FYfVq+O47LyrWtKn62aV4KMmLiMSYVryKiMjPlORFRGJM
SV5EJMaU5EVEYkxJXkQkxpTkRURiTEleRCTGlORFRGJMSV5EJMaU5EVEYkxJXkQkxpTkRURiLOUk
b2aDzexDM5tuZs+Y2abVvldqZvPMbLaZdc1MqCIiUl/ptOTHATuFEDoBc4FSADPrCPQAOgLdgHvN
TJ8YsiyRSEQdQqzo55lZ+nlGJ+XkG0IYH0KoTD59G2idfHwMMDKEUBFCKAfmA/ukFaWsk/4TZZZ+
npmln2d0MtXCPgt4Kfl4a2Bxte8tBlpl6D4iIlIPjWr7ppmNB7Zaw7f6hxBeSL7mGuCnEMKIWi6l
3UFERCKQ1s5QZtYbOBc4JITwQ/JYP4AQwoDk87HA9SGEt2ucq8QvIpKCnGz/Z2bdgNuBA0MIX1Q7
3hEYgffDtwJeBX6vvf5ERHKv1u6adbgbaAyMNzOAt0IIfUIIs8xsFDALWAX0UYIXEYlGZBt5i4hI
9uV8/rqZnWhmM81stZntUeN7WkSVBjMrM7PFZjYt+dUt6pgKjZl1S77/5plZ36jjKXRmVm5m7yff
j5OjjqfQmNnDZrbUzGZUO9bCzMab2VwzG2dmzWq7RhSLlGYAfwYmVT+oRVQZEYA7Qgi7J7/GRh1Q
ITGzhsA9+PuvI3CKmXWINqqCF4CS5PtR62Xqbxj+fqyuHzA+hNAeeC35fK1ynkRDCLNDCHPX8C0t
osqMOo9tE01YAAABv0lEQVS6y2/sA8wPIZSHECqAJ/D3paRH78kUhRBeB76qcbg7MDz5eDhwbG3X
yKeWshZRZcbFyXpCD63rY5z8RitgUbXneg+mLwCvmtm7ZnZu1MHExJYhhKXJx0uBLWt7cTqza9aq
Louo6kijwjXU8rO9BrgPuCH5/EZ8iuvZOQotDvR+y7z9QghLzGxzfCbe7GTrVDIghBDWteYoK0k+
hNAlhdM+Brap9rx18phUU9efrZk9CNTnF6r89j24Db/+dCn1FEJYkvzzczN7Fu8SU5JPz1Iz2yqE
8KmZtQQ+q+3FUXfXVO+rGwOcbGaNzawt0A7QaHw9JP/Bq/wZH+SWunsXaGdmbcysMT4RYEzEMRUs
M2tiZk2TjzcCuqL3ZCaMAXolH/cCnqvtxVlpydfGzP4M3AX8DnjRzKaFEA7XIqqMGGhmu+HdDguB
8yOOp6CEEFaZ2UXAK0BD4KEQwocRh1XItgSeTS6WbAQ8HkIYF21IhcXMRgIHAr8zs0XAdcAAYJSZ
nQ2UAyfVeg3lURGR+Iq6u0ZERLJISV5EJMaU5EVEYkxJXkQkxpTkRURiTEleRCTGlORFRGJMSV5E
JMb+H7+Fyo7kZvNhAAAAAElFTkSuQmCC
"
>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The minimum point of this function is around -1.3. How could we figure that out? Imagine that we could place a ball bearing on the graph of this function. When we let go, it will roll downhill, back and forth, and eventually settle to the lowest point. That's essentially how SGD works. It picks a point somewhere on the graph, measures the slope at that point, steps off a bit in the "downhill" direction, and then repeats. And because this is done numerically, it works for any function. Let's try it:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [23]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="k">print</span> <span class="s">"Minimum found at: </span><span class="si">%0.2f</span><span class="s">"</span> <span class="o">%</span> <span class="n">optimize</span><span class="o">.</span><span class="n">fmin_bfgs</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="mi">0</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>Optimization terminated successfully.
Current function value: -7.945823
Iterations: 5
Function evaluations: 24
Gradient evaluations: 8
Minimum found at: -1.31
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>That's pretty straightforward, so let's try this for our ratings. First we have to write the function that we're trying to minimize. If you think about it a moment, you'll realize that what we're trying to do is minimize the error in our ratings. That's just the term from our pretty-print function, so we can pull it out and create an objective function:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [25]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="k">def</span> <span class="nf">objf</span><span class="p">(</span><span class="n">BH</span><span class="p">,</span> <span class="n">M2</span><span class="p">,</span> <span class="n">E2</span><span class="p">):</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">((</span><span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">M2</span><span class="p">,</span><span class="n">BH</span><span class="p">)</span><span class="o">-</span><span class="n">E2</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">)</span><span class="o">**</span><span class="mf">0.5</span>
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>We also need to provide a first guess at values for BH. We'll take a simple way out and guess all ones. Then we can stick it all into the optimize function and see what happens.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [24]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="n">first_guess</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">ones</span><span class="p">(</span><span class="mi">13</span><span class="p">)</span>
<span class="n">answer</span> <span class="o">=</span> <span class="n">optimize</span><span class="o">.</span><span class="n">fmin_bfgs</span><span class="p">(</span><span class="n">objf</span><span class="p">,</span> <span class="n">first_guess</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">M2</span><span class="p">,</span> <span class="n">E2</span><span class="p">))</span>
<span class="n">correction</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">min</span><span class="p">(</span><span class="n">answer</span><span class="p">[</span><span class="mi">6</span><span class="p">:</span><span class="mi">12</span><span class="p">])</span>
<span class="n">answer</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">12</span><span class="p">]</span> <span class="o">-=</span> <span class="n">correction</span>
<span class="n">ppResults</span><span class="p">(</span><span class="n">answer</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="n">M2</span><span class="p">,</span> <span class="n">E2</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>Optimization terminated successfully.
Current function value: 5.873501
Iterations: 74
Function evaluations: 1245
Gradient evaluations: 83
Team[0]: (71.95, 0.17)
Team[1]: (89.54, 0.00)
Team[2]: (67.28, 15.32)
Team[3]: (78.70, 2.22)
Team[4]: (77.95, 11.05)
Team[5]: (80.57, 15.20)
HCA: 6.92 RMSE: 5.87 MAE: 4.45
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>And sure enough, it finds the same answer.</p>
<p>There are several drawbacks to solving our ratings this way. First of all, it can be slower (possibly much slower) than the linear least squares solution. Secondly, it can sometimes fail to find the true minimum of the function. It can get "trapped" into what is called a local minimum. Recall the graph of the example function:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [26]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">f</span><span class="p">(</span><span class="n">x</span><span class="p">))</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz
AAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm81fO+x/HXpxIiKseQQnHKKUPmk8NlG0qmOKaMlZmu
2aF2XPYxNqAMh8NFN0MRGXKR4rSKwxWVpDSpfU6REGXIsGt/7x+ftdm22u29pt9av/V+Ph770Vq/
vX6/36fd6rO/6zt8vhZCQERE4qlB1AGIiEj2KMmLiMSYkryISIwpyYuIxJiSvIhIjCnJi4jEWJ2S
vJk9bGZLzWxGtWODzexDM5tuZs+Y2abVvldqZvPMbLaZdc1G4CIism51bckPA7rVODYO2CmE0AmY
C5QCmFlHoAfQMXnOvWamTwwiIhGoU/INIbwOfFXj2PgQQmXy6dtA6+TjY4CRIYSKEEI5MB/YJzPh
iohIfWSqhX0W8FLy8dbA4mrfWwy0ytB9RESkHtJO8mZ2DfBTCGFELS9T7QQRkQg0SudkM+sNHAEc
Uu3wx8A21Z63Th6rea4Sv4hICkIIVtfXptySN7NuwFXAMSGEH6p9awxwspk1NrO2QDtg8loC1VeG
vq6//vrIY4jTl36e+nnm61d91aklb2YjgQOB35nZIuB6fDZNY2C8mQG8FULoE0KYZWajgFnAKqBP
SCUyERFJW52SfAjhlDUcfriW198C3JJqUCIikhmavx4TJSUlUYcQK/p5ZpZ+ntGxqHpSzEy9OCIi
9WRmhFwMvIqISP5TkhcRiTEleRGRGFOSFxGJMSV5EZEYU5IXEYkxJXkRkRhTkhcRiTEleRGRGFOS
FxGJMSV5EZEYU5IXEYkxJXkRkRhTkhcRiTEleRGRGFOSFxGJsUiTfGVllHcXEYm/SJP8pElR3l1E
JP4iTfKPPhrl3UVE4i/SPV6bNQt88glsuGEkIYiIFJyC2uN1r71gzJgoIxARibc6JXkze9jMlprZ
jGrHWpjZeDOba2bjzKxZte+Vmtk8M5ttZl3Xdt0zzlCXjYhINtW1JT8M6FbjWD9gfAihPfBa8jlm
1hHoAXRMnnOvma3xPscdB2+8AZ99lkroIiKyLnVK8iGE14GvahzuDgxPPh4OHJt8fAwwMoRQEUIo
B+YD+6zpuhtvDEcdBU88Ud+wRUSkLtLpk98yhLA0+XgpsGXy8dbA4mqvWwy0WttF1GUjIpI9jTJx
kRBCMLPapums8XtlZWVUVsLs2fDIIyX07FmSiXBERGIjkUiQSCRSPr/OUyjNrA3wQghhl+Tz2UBJ
COFTM2sJTAgh/MHM+gGEEAYkXzcWuD6E8HaN64Wqe//lL7D++nDzzSn/PUREikIup1COAXolH/cC
nqt2/GQza2xmbYF2wOTaLnTGGfDYYypzICKSaXWdQjkSeBPY0cwWmdmZwACgi5nNBQ5OPieEMAsY
BcwCXgb6hHV8XOjUCTbdFF5/PfW/iIiI/FakK16r33vwYO+bf+ihSMIRESkI9e2uyZsk//HHsPPO
qMyBiEgtCqqsQXWtWqnMgYhIpuVNkgfNmRcRybS86a4B+PZbaN0a5s6FLbaIJCwRkbxWsN01oDIH
IiKZlldJHtRlIyJS06pVUFYGK1bU/9y8S/KHHAKLF/t0ShGRYrdqFfTsCW+95ZUB6ivvknyjRnDq
qWrNi4hUVHjvxhdfwHPPwQYb1P8aeTXwWuW99+CYY2DhQmiQd7+GRESyb+VKOPFEaNgQnnzyl/VD
BT3wWkVlDkSkmK1YAd26QfPmMHp0egtE8zLJm2kAVkSK0yefwEEHwa67wiOPwHrrpXe9vOyuAZU5
EJHiM3Wqd1VfeCGUlnqDt6ZYdNeAyhyISHF55hk47DAYOhT6919zgk9F3iZ5UJeNiMRfCHDrrXDp
pTB2LBx/fGavn7fdNaAyByISbz/+COedBx984L0Wrda6G/YvYtNdA17moHt3GDEi6khERDLr8899
8ee338KkSXVL8KnI6yQP0KsXDB8edRQiIpkzcyb88Y9w4IHw1FOw0UbZu1feJ/mDDoJly+D996OO
REQkfePGeV7761/h5puzv+Az75N8gwY+AKvWvIgUugce8Do0zzzjeS0X8nrgtcqcOf6xZvFir20j
IlJIKit93vuzz8KLL0K7dqlfK1YDr1V23BHatoVXXok6EhGR+vn+e+jRA9580ytJppPgU1EQSR40
ACsihefzz+Hgg6FxY3j1Vdhss9zHUBDdNQBffeWt+YULvWiPiEg+W7wYunSB446Dm27K3ArWnHfX
mFmpmc00sxlmNsLM1jezFmY23szmmtk4M2uW7n2aN4euXb3kpohIPvvoIzjgADjrLJ9Bk6kEn4q0
kryZtQHOBfYIIewCNAROBvoB40MI7YHXks/Tpi4bEcl3M2f6RJGrr4arroo6mvRb8l8DFUATM2sE
NAE+AboDVel4OHBsmvcBvHjPwoU+20ZEJN/Mnu1dNAMGwAUXRB2NSyvJhxC+BG4H/o0n9+UhhPHA
liGEpcmXLQW2TCvKpEaN4LTTVLRMRPLPwoWe4G+5BU4/PepofpHWrHMz2wG4DGgDrACeMrNf/fVC
CMHM1jjCWlZW9vPjkpISSkpK1nnPXr3gqKPghhu0NaCI5IfFi70OTWkp9O6d2WsnEgkSiUTK56c1
u8bMegBdQgjnJJ+fAXQGDgYOCiF8amYtgQkhhD/UOLdes2uq23NPL83ZtWvKoYuIZMSyZbDffnD2
2bnpg8/17JrZQGcz29DMDDgUmAW8APRKvqYX8Fya9/mVc86BBx/M5BVFROrvxx/hz3/2arn5MMi6
JmnPkzezq/FEXglMBc4BmgKjgG2BcuCkEMLyGuel3JJfsQLatPE685tvnnrsIiKpCsG7j7/7zitJ
5qr7uL4t+YJZDFVT796wyy5w5ZWZi0lEpK5uvNE3+pg4EZo0yd19Y1m7Zk2qumwi+h0lIkXs6ac9
/4wZk9sEn4qCTfL77ed//vOf0cYhIsVlzhy48EKvKNmyZdTRrFvBJnkzDcCKSG6tXAknnOC1aPbY
I+po6qZg++TBK7y1awfl5dAs7eo4IiJrFwKceSasWuULMqOqR1M0ffLgM2u6doWRI6OORETibtgw
eOcd+Pvfoy04Vl8F3ZIH3y+xtBSmTMlAUCIiazB/PnTuDJMmQceO0cZSVC15gEMP9RVnU6dGHYmI
xNHq1T5l+9pro0/wqSj4JN+ggS8n1gCsiGTD7bfDeuvBJZdEHUlqCr67Brw40K67wqJFsNFGGbmk
iAgzZvj2fe++C9ttF3U0rui6awBat/ZdWB5/POpIRCQuKiqgZ08YODB/EnwqYpHkAS66CO65Rytg
RSQz7rgDttrKp00Wslh014An944d4f77vVUvIpKqhQth771h8mTYfvuoo/m1ouyuAZ+3WtWaFxFJ
VQieS668Mv8SfCpi05IH+OYb7zubMQNatcropUWkSIweDdddB9OmQePGUUfzW0Xbkgdo2tT3gL3/
/qgjEZFC9PXXcNllvqo1HxN8KmLVkgffLb2kBP71L1h//YxfXkRi7C9/gS+/hIcfjjqStSuaTUNq
06WLT30644ysXF5EYmjePNh3X5g5E7bcMupo1q6ou2uqXH65r1LTdEoRqaurr/aWfD4n+FTEMskf
frgvZHjttagjEZFCMGECvPee98fHTSyTvJn/Rh48OOpIRCTfrV7tn/4HD4YNNog6msyLZZIHOPVU
+OADmD496khEJJ8NGwabbALHHx91JNkRy4HXKgMHeqJ/9NGs3kZECtTKlb673PPPw157RR1N3Wh2
TTXLl/uKtenTYZttsnorESlAAwb4XhSjRkUdSd3lPMmbWTPgQWAnIABnAvOAJ4HtgHLgpBDC8hrn
ZT3JA1xxhdecv+22rN9KRArIV19B+/bwxhuw445RR1N3UST54cDEEMLDZtYI2Ai4BvgihDDIzPoC
zUMI/Wqcl5Mkv2gR7LabL5LafPOs305ECkRpKXzxBfz3f0cdSf3kNMmb2abAtBDC9jWOzwYODCEs
NbOtgEQI4Q81XpOTJA/Qp4+XPBg4MCe3E5E8t2QJ7LQTvP++70dRSHKd5HcD7gdmAZ2AKcBlwOIQ
QvPkawz4sup5tXNzluQXL4ZOneDDD2GLLXJySxHJY336QJMmhdmNW98k3yjN+zUC9gAuCiG8Y2ZD
gV91y4QQgpmtMZuXlZX9/LikpISSkpI0w1mz1q3hlFN8HqzmzosUtwUL4MknYc6cqCOpm0QiQSKR
SPn8dFvyWwFvhRDaJp/vD5QC2wMHhRA+NbOWwIQou2vgl31gP/wwfsuWRaTuzjkHtt4abrgh6khS
k9PaNSGET4FFZtY+eehQYCbwAtAreawX8Fw698mE1q29DLFa8iLFa8ECeO65eJYvWJtMzK7phE+h
bAx8hE+hbAiMArYl4imU1X38Meyyiy+Q2nrrnN5aRPLAOedAy5Zw441RR5I6LYZah759fdrUQw/l
/NYiEqEFC3zf1nnzoEWLqKNJnZL8OqxY4QsfXn4Zdt8957cXkYjEoRUPSvJ1ct998NRTXorY6vyj
EpFCFZdWPGjTkDo591xYuhReeCHqSEQkFwYNggsuKPwEn4qibMkDjB0Ll1zig7Bx2bBXRH5ryRLo
2NHnxcdhMaRa8nXUrZtXqLzrrqgjEZFsuvNOnz4dhwSfiqJtycMvG/dOnuwJX0TiZcUK/7/97rvQ
tm3U0WSGWvL10K6db957/vna9Fskjv7+d9/zOS4JPhVF3ZIHWLUK/vhHuOgiOPPMqKMRkUz54QdP
7q+84iVN4kIt+Xpq1MgXRvXt6wM0IhIPw4fDHnvEK8Gnouhb8lX69/eNRUaP1tx5kUK3erUvehw2
DP7jP6KOJrPUkk/RddfBRx+p3IFIHIwe7dVm998/6kiip5Z8NR9+CAccABMn+rxaESk8IcCee0JZ
GXTvHnU0maeWfBo6dPAtAo8/Hr75JupoRCQV48fDjz/CUUdFHUl+UEt+Dc4/3ytVPv20+udFCs0h
h0CvXtCzZ9SRZIda8hlw110+0+a666KORETqY/JkmD/ft/sUpyS/Buuv77vHjBgBDz8cdTQiUlcD
B8KVV8J660UdSf5Qd00t5syBkhL429/guOOijkZEajNnjk+XXLgQNtoo6miyp77dNY2yGUyh23FH
eOklL2bWuLEGckTy2eDB8J//Ge8Enwq15Otg8mQ4+mjvq+/RI+poRKSmqv2b582DzTaLOprsUks+
C/bZx6dlHX44fPKJ7/SuWTci+WPIEJ9RE/cEnwq15OvhX//yLpvOneHuu2GDDaKOSES++gp22AGm
T4dttok6muzTFMos2m47ePNNWL7c69DPmRN1RCJy772+srUYEnwqlOTrqWlTGDUKzjsP9tsPbrvN
yxWLSO6tXOljZVdfHXUk+SsjSd7MGprZNDN7Ifm8hZmNN7O5ZjbOzJpl4j75wgwuvNAHZMeNg512
8oJIBdb7JFLwhg3zT9WqNbV2GemTN7MrgD2BpiGE7mY2CPgihDDIzPoCzUMI/WqcU3B98msSgg/K
9u3r0yzLyuCww6CBPiOJZNWqVb6724gRnuiLRc775M2sNXAE8CBQdePuwPDk4+HAseneJ1+ZQdeu
MGUKXHopXHMN/P73MGAALF0adXQi8TVqFGy7bXEl+FRkor05BLgKqKx2bMsQQlWKWwpsmYH75LUG
DeDUUz3ZP/mkz9dt3x4OOgiGDvVVeCKSGSF4Q6pfv3W/ttilNU/ezI4CPgshTDOzkjW9JoQQzGyN
/TJlZWU/Py4pKaGkZI2XKChmsPfe/nX33fDqq/D883DLLbDJJl6vvuqrbVvNtxdJxUsv+f+dbt2i
jiT7EokEiUQi5fPT6pM3s1uAM4BVwAbAJsAzwN5ASQjhUzNrCUwIIfyhxrmx6JOvq8pKmDULXn8d
Jk3yjUkaNPBaG/vv73/uvLP68kXqYv/94aKL4OSTo44k9+rbJ5+xxVBmdiDwlxDC0cmB12UhhIFm
1g9oFteB11SF4NsNvvGGJ/7XX4fPP4c//Qm6dIEjj/RBJRH5tddfhzPP9D2ZGxXhmv2ok/yVydk1
LYBRwLZAOXBSCGF5jdcXdZJfk6VL/Q08dqx/HN14Yzj2WF+uvdNOUUcnkh+OOML/X5x3XtSRRCOy
JF9fSvK1CwHee89nEDzyCGy9tbdeeveGJk2ijk4kGu+9559yFyzwfR+KkZJ8DK1e7QO499/vZRUu
vdRLqm6ySdSRieRWjx5eMPDKK6OOJDpK8jE3cybceqt36fTr5wlfu+BIMZg3z8esFizw8iLFSkm+
SMyZAxdfDJ9+Cvfd53V0ROLs3HO92/Kvf406kmgpyReREOCpp+CKK7wE8pAhsOGGUUclknnFtCnI
uqjUcBExg5NO8vn3K1Z4nfu5c6OOSiTz7rjDJx0Ue4JPhVryMRGCD8z+13/BPfdom0KJj2XLfM3I
++9D69ZRRxM9ddcUualT4fjj4ZxzoH9/lU2QwnfttfDZZ/DAA1FHkh+U5IUlS7ymxwEHwJ13qlSC
FK5ly7zQ35Qp0KZN1NHkB/XJCy1ben2cGTPglFPgxx+jjkgkNUOG+CdTJfjUqSUfYz/84Em+ahaO
5tNLIfnyS++LVyv+19SSl59tsIHXtv/pJy+JUFm57nNE8sWQIXDccUrw6VJLvgh8/z0cfjh06OA7
22swVvJdVSv+3Xd93wX5hVry8hsbbghjxvjH3v79o45GZN3uuMMrTSrBp08t+SKybJkvmOrf37tv
RPLR0qXQsaNPB95uu6ijyT+aQim1mj3bp1aOHu27UYnkm4sv9s1AhgyJOpL8pCQv6zRunG9E8uab
+jgs+WXBAt8fefZs2HzzqKPJT+qTl3Xq2hVKS+Hoo+Hbb6OORuQX110Hl1yiBJ9JaskXqRDg7LOh
osJ3ntKMG4na9OneAJk/v7jrxa+LWvJSJ2ZeyGzaNHjooaijEfEJAf37K8FnWhHudS5VmjTxlbAH
HOBbqu26a9QRSbEaN843wnnmmagjiR+15Itchw4+i+HEE+Gbb6KORopRRQVcdpnPjS/WzbmzSX3y
AvjWahUV8D//E3UkUmzuvBNefBFeeUVjQ3WhKZSSkm+/hd13hwEDvOqfSC58/rkvfJo40f+Udcvp
wKuZbWNmE8xsppl9YGaXJI+3MLPxZjbXzMaZWbN07iPZt/HG8Nhj0KeP16MXyYVrr4XTTlOCz6a0
WvJmthWwVQjhPTPbGJgCHAucCXwRQhhkZn2B5iGEfjXOVUs+D11/PUyeDC+9pI/Okl1Tp3rhvNmz
oXnzqKMpHDltyYcQPg0hvJd8/C3wIdAK6A4MT75sOJ74pQBce63XuLn33qgjkTirqPB1GoMHK8Fn
W8b65M2sDTAR2Bn4dwihefK4AV9WPa/2erXk89TcufCnP3mLfvvto45G4ujmm+GNN/SJMRX1bcln
ZJ58sqtmNHBpCOEbq/avFkIIZrbGbF5WVvbz45KSEkpKSjIRjqSpfXvo1883A3/tNf0nlMyaNcun
7U6dqvdWXSQSCRKJRMrnp92SN7P1gP8FXg4hDE0emw2UhBA+NbOWwIQQwh9qnKeWfB5bvdpb82ef
DeedF3U0EherV3v109NP90F+qb9cz64x4CFgVlWCTxoD9Eo+7gU8l859JPcaNoSHH4ZrroFFi6KO
RuLirru8jPAFF0QdSfFId3bN/sAk4H2g6kKlwGRgFLAtUA6cFEJYXuNcteQLwE03eUniF1/UR2tJ
z7vvwhFHwFtvwQ47RB1N4dJiKMmoigqva3P55dCzZ9TRSKFavhz23NMX2514YtTRFDYlecm4adPg
sMO8FGzLllFHI4UmBDjhBH/v3HNP1NEUPpUalozbfXevbdOnj/+HFamPe+6B8nK4/faoIylOaslL
nfz4I+y2G9x4o7fKROpiwgTo0UP98Jmk7hrJmn/+0/tTP/gAWrSIOhrJd7Nnw4EHwsiRcPDBUUcT
H0ryklUXX+wVK4cNizoSyWf//rdvRlNWBr17Rx1NvCjJS1Z98w3ssgs88IDvxymZ8e233vItL/fy
u99/78c32wx+9zvYYgvYcUfYZJNIw6yTJUu8Bd+nj28GIpmlJC9ZN3YsXHghzJjhJYql/ioqIJGA
MWP8zwULvJxE27ae0Js08UHuZcs86S9d6tvjtWoFe+zhq5EPPxzatYv6b/Jr5eXQpQucdRaUlkYd
TTwpyUtO9Ozp/fJDh677tfKLxYt91ecjj0CbNnDssXDIIT6ovd56tZ+7apUn+ilTfJONl1/2XwZH
HgknnwydO0e7YG3qVDjmGLj6au/Wk+xQkpecWLYMdt4Znn3Wk4vUbvFiXwg0cqT3UZ9/vrfc0xEC
vP8+PP88jBgBP/0Ep57qm3B06JCRsOvsiSc8sd93n2ZfZZuSvOTMk0/CDTd4C04bMK/ZypU++Pjg
g17s7aqrvDsm00LwRWsjRvgvki22gDPO8KS/1VaZv1+Vr7/2fvc33vD3w+67Z+9e4rQYSnLmpJPg
97+HW2+NOpL89OqrPkj98cdeXnfw4OwkePBumj32gNtu85ktt9/uYyYdOkC3bvD44/Ddd5m7X2Wl
X7NDB+9mmjpVCT5fqSUvafn4Y+9P/sc/PKGJz5S55BL/mdx3nw+QRmXlSu/OefRRLzR3yCHeb37k
kT5zJ5XrPfUUDBrkg+5Dh8K++2Y+blk7dddIzj3wADz0kCeRhg2jjiZas2Z5n3TnznDnndC0adQR
/eKLL7ya6HPP+WYw7dv7LJ199/XiYW3aQOPGvz5n1SrfKeydd2DcOB/s7dzZf4kddpgqk0ZBSV5y
rrLSW4jdu3u1ymL12GP+9x80CM48M+poavfDDz5L5803veTA9Ok+ONy0KWy6qffxf/cdfPmlT+vc
fXc49FAvFdyqVdTRFzcleYnE/PnewivGfWFXr/Zpgy+8AE8/DbvuGnVEqVm1ypP61197C71JE9h8
c9/kQ/KHkrxEZvBgeOUVGD++eD7Gf/edT1lcsQKeeQaaN1/3OSLp0Owaiczll/vmEMVS12bJEigp
gWbN/JebErzkIyV5yZhGjXwAtl8/n8YXZwsWwH77wdFH+y+1mgOWIvlCSV4yqlMnuOIK6NXLB2Tj
aOZMr7B41VVw3XXF0zUlhUlJXjLuqqt8MPKOO6KOJPPeecdnEg0c6EXaRPKdBl4lK8rLYe+9fdVn
p05RR5MZEyf6pikPPujTRUWioIFXyQtt2vgS+9NO8znZhe6llzzBjxypBC+FRS15yZoQ4JRTfHHN
/fdHHU3qRo/2DTCef14VNyV6edOSN7NuZjbbzOaZWd9s3Ufyl5mXPEgkvHZKIXr8cbjoIp8iqQQv
hSgrLXkzawjMAQ4FPgbeAU4JIXxY7TVqyReJGTN8I+dEAnbaKepo6u7BB71M8Lhx0LFj1NGIuHxp
ye8DzA8hlIcQKoAngGOydC/Jc7vs4qthTzjBKzQWgrvvhhtvhAkTlOClsGUrybcCFlV7vjh5TIpU
796+eKhnz/yfPz9woFeQnDQp//ZQFamvbJUeqlM/TFlZ2c+PS0pKKCkpyVI4kg/+9jfo2tWLed12
W9TR/FYI3j0zapRPl1S1RckHiUSCRCKR8vnZ6pPvDJSFELoln5cClSGEgdVeoz75IvTll17D/NJL
82sxUWWl195JJLzAWrZ2cBJJV3375LPVkn8XaGdmbYBPgB7AKVm6lxSQFi1844r994fttvP65FH7
4QfvRvrsM2/BN2sWdUQimZOVPvkQwirgIuAVYBbwZPWZNVLcdtjBy/L27u0bQEdp+XLfAxVg7Fgl
eIkfLYaSyIwf7ytin38+mn1C586FY4+FLl1gyBBooPXfUgDyZQqlyDp16QKPPOIbS7/6am7v/b//
611Gl13mM2mU4CWu1JKXyE2a5HPohwzxln02rV4NN9/sK3GfeiqaTxAi6ciXgVeROjvgAPjHP7zw
14wZcNNN2dlXdO5c32B7vfV8L9qtt878PUTyjT6kSl7YeWd4+22YMsW31Csvz9y1Kyth6FCfutmj
h/9CUYKXYqEkL3lj8829ENgxx8Bee8GgQfDTT+ld87XX/FqjR8P//R9ccon636W4qE9e8tL8+T4o
OmMGlJbC6afDxhvX7dyffoIXXvCdqT77DG65xfv8tU2fxEF9++SV5CWvvfWW15KZOBGOPBIOO8x3
nNphB+9bB6iogI8+gmnTfJbOmDHQoYOXCD7+eGjYMNq/g0gmKclLLC1Z4sl7/HiYPt377DfYwFvn
K1f66tldd/WSxkceCdtvH3XEItmhJC9FYfVq+O47LyrWtKn62aV4KMmLiMSYVryKiMjPlORFRGJM
SV5EJMaU5EVEYkxJXkQkxpTkRURiTEleRCTGlORFRGJMSV5EJMaU5EVEYkxJXkQkxpTkRURiLOUk
b2aDzexDM5tuZs+Y2abVvldqZvPMbLaZdc1MqCIiUl/ptOTHATuFEDoBc4FSADPrCPQAOgLdgHvN
TJ8YsiyRSEQdQqzo55lZ+nlGJ+XkG0IYH0KoTD59G2idfHwMMDKEUBFCKAfmA/ukFaWsk/4TZZZ+
npmln2d0MtXCPgt4Kfl4a2Bxte8tBlpl6D4iIlIPjWr7ppmNB7Zaw7f6hxBeSL7mGuCnEMKIWi6l
3UFERCKQ1s5QZtYbOBc4JITwQ/JYP4AQwoDk87HA9SGEt2ucq8QvIpKCnGz/Z2bdgNuBA0MIX1Q7
3hEYgffDtwJeBX6vvf5ERHKv1u6adbgbaAyMNzOAt0IIfUIIs8xsFDALWAX0UYIXEYlGZBt5i4hI
9uV8/rqZnWhmM81stZntUeN7WkSVBjMrM7PFZjYt+dUt6pgKjZl1S77/5plZ36jjKXRmVm5m7yff
j5OjjqfQmNnDZrbUzGZUO9bCzMab2VwzG2dmzWq7RhSLlGYAfwYmVT+oRVQZEYA7Qgi7J7/GRh1Q
ITGzhsA9+PuvI3CKmXWINqqCF4CS5PtR62Xqbxj+fqyuHzA+hNAeeC35fK1ynkRDCLNDCHPX8C0t
osqMOo9tE01YAAABv0lEQVS6y2/sA8wPIZSHECqAJ/D3paRH78kUhRBeB76qcbg7MDz5eDhwbG3X
yKeWshZRZcbFyXpCD63rY5z8RitgUbXneg+mLwCvmtm7ZnZu1MHExJYhhKXJx0uBLWt7cTqza9aq
Louo6kijwjXU8rO9BrgPuCH5/EZ8iuvZOQotDvR+y7z9QghLzGxzfCbe7GTrVDIghBDWteYoK0k+
hNAlhdM+Brap9rx18phUU9efrZk9CNTnF6r89j24Db/+dCn1FEJYkvzzczN7Fu8SU5JPz1Iz2yqE
8KmZtQQ+q+3FUXfXVO+rGwOcbGaNzawt0A7QaHw9JP/Bq/wZH+SWunsXaGdmbcysMT4RYEzEMRUs
M2tiZk2TjzcCuqL3ZCaMAXolH/cCnqvtxVlpydfGzP4M3AX8DnjRzKaFEA7XIqqMGGhmu+HdDguB
8yOOp6CEEFaZ2UXAK0BD4KEQwocRh1XItgSeTS6WbAQ8HkIYF21IhcXMRgIHAr8zs0XAdcAAYJSZ
nQ2UAyfVeg3lURGR+Iq6u0ZERLJISV5EJMaU5EVEYkxJXkQkxpTkRURiTEleRCTGlORFRGJMSV5E
JMb+H7+Fyo7kZvNhAAAAAElFTkSuQmCC
"
>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>See the little valley around 4? That's a local minimum. If we happen to start in the wrong place, SGD can roll down into that minimum and get stuck there.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [27]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="k">print</span> <span class="s">"Minimum found at: </span><span class="si">%0.2f</span><span class="s">"</span> <span class="o">%</span> <span class="n">optimize</span><span class="o">.</span><span class="n">fmin_bfgs</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="n">disp</span><span class="o">=</span><span class="mi">0</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>Minimum found at: 3.84
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Oops. There are ways to address this problem, but I won't discuss them here.</p>
<p>Despite these drawbacks, SGD has a couple of big advantages over linear regression. First, linear regression always optimizes over mean square error (that's the RMSE metric), and that may not be what you want. For example, <a href="http://www.thepredictiontracker.com/bbresults.php">The Prediction Tracker</a> shows both mean square error and mean absolute error (that's MAE). My predictor was often better at RMSE and worse at MAE, which irked me. If it had bothered me enough, I could have used SGD to optimize on MAE instead of RMSE:</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [28]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre><span class="k">def</span> <span class="nf">ourf</span><span class="p">(</span><span class="n">BH</span><span class="p">,</span> <span class="n">M2</span><span class="p">,</span> <span class="n">E2</span><span class="p">):</span>
<span class="c"># Note the different return value here</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">abs</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">M2</span><span class="p">,</span><span class="n">BH</span><span class="p">)</span><span class="o">-</span><span class="n">E2</span><span class="p">))</span>
<span class="n">first_guess</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">ones</span><span class="p">(</span><span class="mi">13</span><span class="p">)</span>
<span class="n">answer</span> <span class="o">=</span> <span class="n">optimize</span><span class="o">.</span><span class="n">fmin_bfgs</span><span class="p">(</span><span class="n">ourf</span><span class="p">,</span> <span class="n">first_guess</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">M2</span><span class="p">,</span> <span class="n">E2</span><span class="p">))</span>
<span class="n">correction</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">min</span><span class="p">(</span><span class="n">answer</span><span class="p">[</span><span class="mi">6</span><span class="p">:</span><span class="mi">12</span><span class="p">])</span>
<span class="n">answer</span><span class="p">[</span><span class="mi">0</span><span class="p">:</span><span class="mi">12</span><span class="p">]</span> <span class="o">-=</span> <span class="n">correction</span>
<span class="n">ppResults</span><span class="p">(</span><span class="n">answer</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="n">M2</span><span class="p">,</span> <span class="n">E2</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area"><div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>Warning: Desired error not necessarily achieved due to precision loss.
Current function value: 3.200011
Iterations: 94
Function evaluations: 2802
Gradient evaluations: 186
Team[0]: (76.79, 0.00)
Team[1]: (88.79, 2.00)
Team[2]: (66.79, 18.32)
Team[3]: (78.47, 0.32)
Team[4]: (78.79, 8.47)
Team[5]: (86.47, 15.32)
HCA: 6.53 RMSE: 6.98 MAE: 3.20
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>This solution has a better MAE (and a worse RMSE). In general, you can choose any objective function to optimize through SGD, so that provides a lot of flexibility.</p>
<p>The second advantage of SGD is that it isn't limited to linear functions. You could use it to minimize a polynomial or other complex function. That isn't directly relevant for this rating system, but it could be for a different system.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [ ]:</div>
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython2"><pre>
</pre></div>
</div>
</div>
</div>
</div>
</div>
</div>
Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com0tag:blogger.com,1999:blog-5902620336509647050.post-37208256425757916312015-09-29T14:17:00.001-04:002015-09-29T14:17:50.306-04:00Missed It By *That* Much!After five years of working on predicting college basketball, I've honed a decent intuition for the problem. That applies doubly to my own predictor. When I add a feature I have a pretty good idea of how it will affect performance. Anything too significant and I immediately suspect that I've had data leakage or some other error. In fact, this happened to me enough in the first year or so that I eventually re-architected my predictor specifically to create separation between the processing of the season's games and the prediction of scheduled games, and I carried that architecture over into the Python rewrite.<br />
<br />
So I was a little dubious when I added a new feature to the predictor and saw a significant increase in performance. The increase wasn't enormous (although with tuning it did become even better) but it was enough to trigger my "Spidey sense". I commenced to carefully inspect my code, but I couldn't find any leaks. The code was parallel to a similar feature that I was confident was sound, and I hadn't done anything obvious to violate my separation architecture, so it was a bit puzzling.<br />
<br />
My ability to test the problem directly was limited (because portions of the architecture aren't yet built out in Python) but seemed to confirm there was a problem. Without many other options, I broke out the debugger and started stepping through the program while monitoring the data structures. Which is a laborious task when you're processing 32,000 games and your model has 500+ features.<br />
<br />
Eventually I found the problem. At one point in the code where I thought I was copying a value out of an array to save it elsewhere, I was actually getting a 1x1 array -- i.e., an array containing just one value. Python is so flexible with data structures that this actually worked fine throughout the rest of the code. It turns out that a 1x1 array of a value is just about interchangeable with the value itself.<br />
<br />
The rub is that Python doesn't copy arrays unless forced. So when I grabbed that 1x1 array, I was really just getting a view into the array it came from. When the value in the original array changed, so did the value in my little 1x1 array. The end result was that new values in the original array would leak back into what I thought were saved values. <br />
<br />
This sort of thing leads a lot of people to hate untyped scripting languages like Python and Javascript. The flexibility is great, but it can lead to difficult to diagnose errors (and also poor performance). I'm a little more pragmatic. All the choices in programming languages have plusses and minuses, and in my experience they tend to largely balance out.<br />
<br />
So in the end I didn't discover the secret of college basketball, but at least I found my bug. So that's something.<br />
<br />Scott Turnerhttp://www.blogger.com/profile/03393071448515738228noreply@blogger.com2