My system went 3-0-1 last weekend (Or 3-1 if you consider my prediction in the Bengals – Chargers game a loss, as opposed to “too close to pick”), so time to present playoff odds for the second round of the playoffs.

Divisional Round Playoff Odds
Home Team Visiting Team Score Diff Win Prob Est. Point Spread
Seattle Seahawks New Orleans Saints 0.676 0.663 5.0
Carolina Panthers SF 49ers -0.157 0.461 -1.2
Denver Broncos San Diego Chargers 0.411 0.601 3.0
New England Patriots Indianapolis Colts -0.060 0.485 -0.4

 

Odds that differ by less than a point in estimated point spread are probably not significant, and from my POV, a suggestion that you don’t bet that particular game.

The regular season has ended and the playoffs have begun. It would be useful to have a set of playoff grade data to do playoff probabilities, and though I’ve been down and out this season (no job at times, foot stress fracture at times, and a bad right shoulder), I currently have some time off my new job, a new laptop, and enough time to grind through some playoff numbers.

NFL stats at the end of the regular season:

week_17_2013_stats

To explain the columns above, Median is a median point spread, and can be used to get a feel for how good a team is without overly weighting a blowout win or blowout loss. HS is Brian Burke’s Homemade Sagarin, as implemented in Maggie Xiong’s PDL::Stats. Pred is the predicted Pythagorean expectation. The exponent for this measure is fitted to the data set itself. SOS, SRS, and MOV are the simple ranking components, analyzed via this Perl implementation. MOV is margin of victory, or point spread divided by games played. SOS is strength of schedule. SRS is the simple ranking.

Playoff Odds are calculated according to this model:

logit P  =  0.668 + 0.348*(delta SOS) + 0.434*(delta Playoff Experience)

The results are given below, as a “score” in logits:

2013 NFL Playoff Teams, C&F Playoff Model Worksheet.
NFC
Rank Name Home Field Advantage Prev. Playoff Experience Strength of Schedule Total Score
1 Seattle Seahawks 0.406 0.434 0.494 1.334
2 Carolina Panthers 0.406 0.0 0.484 0.889
3 Philadelphia Eagles 0.406 0.0 -0.661 -0.256
4 Green Bay Packers 0.406 0.434 -0.842 -0.003
5 San Francisco 49ers 0.0 0.434 0.612 1.046
6 New Orleans Saints 0.0 0.0 0.658 0.658
AFC
1 Denver Broncos 0.406 0.434 -0.546 0.293
2 NE Patriots 0.406 0.434 -0.258 0.582
3 Cancinnati Bengals 0.406 0.434 -0.856 -0.017
4 Indianapolis Colts 0.406 0.434 0.209 1.048
5 Kansas City Chiefs 0.0 0.0 -0.602 -0.602
6 San Diego Chargers 0.0 0.0 -0.118 -0.118

 

The total score of a particular team is used as a base. Subtract the score of the opponent and the result is the logit of the win probability for that game. You can use the inverse logit (see Wolfram Alpha to do this easily) to get the probability, and you can multiply the logit of the win probability by 7.4 to get the estimated point spread.

For the first week of the playoffs, I’ve done all this for you, in the table below. Odds are presented from the home team’s point of view.

First Round Playoff Odds
Home Team Visiting Team Score Diff Win Prob Est. Point Spread
Philadelphia Eagles New Orleans Saints -0.914 0.286 -6.8
Green Bay Packers SF 49ers -1.049 0.259 -7.8
Cincinnati Bengals San Diego Chargers 0.101 0.525 0.7
Indianapolis Colts Kansas City Chiefs 1.650 0.839 12.2

 

Some general conclusions from the data above: the teams my model favors most are the Seattle Seahawks, the Indianapolis Colts, the 49ers, the Carolina Panthers, and then the New Orleans Saints, mostly NFC teams. Since the Super Bowl itself does not have a home team, the odds change once you actually reach the Super Bowl. The sum of the SOS column and the Previous Playoff Experience column can be used to estimate odds of winning “the big one”. The strongest team in a Super Bowl setting would be the San Francisco 49ers, with a total score, less HFA, of 1.049. The Indianapolis Colts, with a total score of 0.643 less HFA, would be the strongest possible AFC contender.

A point I’d like the reader to consider is this question: should the New Orleans Saints be granted an exception to the previous playoff experience rule of “last year only counts” and given the 0.434 advantage of a playoff team? 2012 was an aberration as the coach was suspended. I’m not calculating this variation into the formula at this point, but I’ll note that this is an issue that you, the reader, need to resolve for yourself.

The road to the playoffs is not easy, a topic that can be studied by trying to calculate the path to the playoffs of the Indianapolis colts, a team that would be favored in every matchup along the way. Let’s calculate the odds of Indianapolis actually winning all three games.

Odds of Indianapolis reaching the Super Bowl
WP versus Kansas City WP versus Denver Broncos WP versus NE Pats Cume Probability
0.839 0.586 0.515 0.253

 

Three teams from the NFC would be favored over any possible AFC contender. Those are San Francisco, Seattle, and the New Orleans Saints. Carolina would be favored over any AFC contender except the Indianapolis Colts.

I suspect  to a first approximation almost no one other than Baltimore fans, such as Brian Burke, and this blog really believed that Baltimore had much of a chance(+). Well, I should mention Aaron Freeman of Falc Fans, who was rooting for Baltimore but still felt Denver would win. Looking, his article is no longer on the Falcfans site. Pity..

WP graph of Baltimore versus Denver. I tweeted that this graph was going to resemble a seismic chart of an earthquake. Not my work, just a screen shot off the excellent site Advanced NFL Stats.

WP graph of Baltimore versus Denver. I tweeted that this graph was going to resemble a seismic chart of an earthquake. Not my work, just a screen shot off the excellent site Advanced NFL Stats.

After a double overtime victory by 3 points, it’s awfully tempting to say, “I predicted this”, and if you look at the teams I’ve  favored, to this point* the streak of picks is 6-0. Let me point out though, that you can make a limiting assumption and from that assumption figure out how accurate I should have been. The limiting assumption is to assume the playoff model is 100% accurate** and see how well it predicted play. If the model is 100% accurate, the real results and the predicted results should merge.

I can tell you without adding up anything that only one of my favored picks had more than a 70% chance, and at least two were around 52-53%. So 6 times 70 percent is 4.2, and my model, in a perfect world, should have picked no more than 4 winners and 2 losers. A perfect model in a probabilistic world, where teams rarely have 65% chances to win, much less 100%, should be wrong sometimes. Instead, so far it’s on a 6-0 run. That means that luck is driving my success so far.

Is it possible, as I have argued, that strength of schedule is an under appreciated playoff stat, a playoff “Moneyball” stat, that teams that go through tough times are better than their offense and defensive stats suggest? It’s possible at this point. It’s also without question that I’ve been lucky in both the 2012 playoffs and the 2013 playoffs so far.

Potential Championship Scenarios:

 

Conference Championship Possibilities
Home Team Visiting Team Home Win Pct Est. Point Spread
NE BAL 0.523 0.7
HOU BAL 0.383 -3.5
ATL SF 0.306 -6.1
SF SEA 0.745 7.9

 

My model likes Seattle, which has the second best strength of schedule metric of all the playoff teams, but it absolutely loves San Francisco. It also likes Baltimore,  but not enough to say it has a free run throughout the playoffs. Like many modelers, I’m predicting that Atlanta and Seattle will be a close game.

~~~

+ I should also mention  that Bryan  Broaddus tweeted about a colleague of his who predicted a BAL victory.

* Sunday, January 13, 2013, about 10:00am.

** Such a limiting assumption is similar to assuming the NFL draft is rational; that the customers (NFL teams) have all the information they should, that they understand everything about the product they consume  (draft picks), and that their estimates of draft value thus form a normal distribution around the real value of draft picks, and that irrational exuberance, or trends, or GMs falling in love with players play no role in picking players. This, it turns out, makes model simulations much easier.

Though the results for the divisional round are embedded in the image of my playoff spreadsheet in my previous article, the table below is certainly easier to read.

 

Divisional Playoff Round
Home Team Visiting Team Home Win Pct Est. Point Spread
DEN BAL 0.477 -0.7
NE HOU 0.638 4.2
ATL SEA 0.462 -1.1
SF GB 0.700 6.3

 

I suspect other systems will rank Seattle as stronger than mine does, and Baltimore as weaker. That said, the Vegas line as of this Sunday gives Atlanta a 2 point advantage over Seattle, and my system slightly favors Seattle. We can calculate odds and points via other mechanisms, say, Pythagoreans, SRS and median point spreads, and if we do, what do we get?

 

Atlanta Versus Seattle
Technique Home Win Pct Est. Point Spread
Median Point Spread 0.632 4.0
Simple Ranking System 0.407 -2.8
Pythagorean Expectation 0.486 -0.4

 

Certainly different systems yield different emphases. For me, the one lasting impression I had was the Washington Seattle game was an almost picture perfect demonstration that home field advantage is strongest in the first quarter.

Of all the teams playing, my system likes San Francisco the best. I suspect it likes it more than others. We’ll learn more as other analytics oriented folks post their odds for the divisional round.

We can’t work with my playoff model without having a set of week 17 strength of schedule numbers, so we’ll present those first.

2012_stats_week_17

Between a difficult work schedule this last December and a very welcome vacation (I keep my stats on a stay at home machine), I haven’t been giving weekly updates recently. Hopefully some of my various thoughts will begin to make up for that.

Though with SOS values, you could crunch all the playoff numbers yourselves, this set of data should help in working out the possibilities:

Odds as calculated by my formula

Odds as calculated by my formula, with home field advantage adjusted to 60%. Point spread calculated with formula 3.0*logit(win probability)/logit(0.60). Click on image twice to expand.

What I find interesting is the difference between Vegas style lines, and my numbers, and the numbers recently posted by Brian Burke on the New York Times Fifth Down blog. My model is very different from Brian’s, but in three of the four wild card games, our percentage odds to win are within 2-3 percent of each other.

Point spreads were estimated as follows: if an effect of 60% were valued at 3 points (i.e. playoff home field advantage is about 60% and home field advantage is usually judged to be worth 3 points), then two effects of that magnitude should be worth 6 points. But it’s only on a logit scale that these effects can be added, so it only makes sense to relate probabilities of winning through their logits. As the logit of 0.60 is about 0.405465, then an estimated point spread can be had with the formula

point spread = 3.0*logit(win probability)/0.405465

Update (1/9/2012) – even simpler is:

est. point spread = 7.4*logit(win probability)

A simplified table of the wild card games, with percentages and estimated point spreads is:

Wild Card Playoff Round
Home Team Visiting Team Home Win Pct Est. Point Spread
GB MIN 0.682 5.6
WAS SEA 0.482 -0.5
HOU CIN 0.642 4.3
BAL IND 0.841 12.3

How many successes is a touchdown worth?

We’ve spoken about the potential relationships between success rates, adjusted yards per attempt, and stats like DVOA here, but to make any progress, you need to consider possible relationships between successes and yards. Let me point out the lower bound of the relationship is known, as 3 consecutive successes must yield at least 10 yards, and 30 consecutive successes must end up scoring a touchdown. In this case, the relationship is 1 success is equal to or greater than 3 1/3 yards.

Thus, if the surplus value of a touchdown is 20 yards, that’s 6 successes. If a turnover is worth 45 yards, that’s about 13.5 successes.

A smarter way to get at the mean value of this kind of relationship, as opposed to a limiting value, would be to add up the yards of all successful plays in the NFL and divide by the number of those plays. For now, that’s something to be pursued later.

Things that are easy to note: the teams with at least 9 wins are either guaranteed a playoff birth, or have, at worst, a 99% chance of making the playoffs. The teams with 8 wins have a very good chance of entering the playoffs. Those teams with 7 wins have at least a 50% chance of making the playoffs. Those with 6 wins have between a 5% to 30% chance of making the playoffs. Let’s say they are hoping to get in.

Data from week 12

2012_stats_week_12

Data from week 13

2012_stats_week_13

The methodology of these stats is discussed in previous posts in this series. If you’re wondering where I’m getting odds to go into the playoffs, see this post. If you’re wondering what chance your team has of winning in the playoffs, see this post on my logistic regression methods, based on studies of playoff games. How would your ranking in the playoffs affect your chances of getting into the Super Bowl? We studied that here.

I am not a proponent of the notion that regular season offensive stats are predictive in the post season. My studies suggest p on the order of 0.15 for offensive stats in the post season, and thus aren’t predictive enough for my tastes ( p <= 0.05). That hasn't stopped Football Outsiders from pretending that their proprietary stats are predictive and calculating playoff odds with their tools.

The three sites we noted last year: Cool Standings, Football Outsiders, and NFL Forecast, are at it again, providing predictions of who is going to be in the playoffs.

 

Cool Standings uses Pythagoreans to do their predictions (and for some reason in 2011, ignored home field advantage), FO uses their proprietary DVOA stats, and NFL Forecast uses Brian Burke’s predictive model.

Blogging the Beast has a terrific article on “the play”. If you watched any Dallas-Philadelphia games in 2011, you’ll know exactly what I mean, the way with a simple counter trap, LeSean McCoy treated the Cowboys line as if it were Swiss cheese.

Most important new link, perhaps, is a new Grantland article by Chris Brown of Smart Football. This article on Chip Kelly is really good. Not only is the writing good, but I love the photos:

Not my photo. This is from Chris Brown’s Chip Kelly article (see link in text).

as an example. Have you ever seen a better photo of the gap assignments of a defense?

In April of 2011 I published a playoff model, one that described  the odds of winning in terms of home field advantage, strength of schedule, and previous playoff experience. In the work I did then, I fixed the length of time of previous playoff experience to 2 years, and as I was working with (and changing) my experimental design (the “y” variable was initially playoff  winning percentage, which turned out to be a relatively insensitive parameter), once I had a result with two independent variables, at that point I called it a day and published.

Once the 2011 season rolled into the playoffs, while thinking about the upcoming game between Atlanta and New York, I realized I had never tested the span of time over which playoff experience mattered. I then proposed that New York could be considered to have playoff experience, since it had played in 2007, and if so, there would be marked changes to the odds associated with the New York Giants. This was a reasonable proposition at the time, because no testing had been done on my end to prove or disprove the idea.

Using this notion, the formula we published then racked up a 9-2 record for predicting games, or more cautiously, 7-2-2, as the results obtained for the San Francisco-New Orleans game (50-50 odds) and the NYG-Green Bay game (results yielding possible wins for both teams at the same time) really didn’t lend confidence in betting for either side of those two games.

Once the playoffs were over,  I uploaded the new 2011 playoff games and did logistic regressions of these data. I amended the program I used for my analysis to allow for playoff experience to be judged over 1,2,3, or 4 year intervals.  I also allowed the program to vary the range of years to be fitted. Please note, that there are a very small number of playoff games played in any particular year (11), and I’ve seen sources that claim you can really only resolve one parameter per 50 data points. If we cut the data set too short, we’re playing with fire in terms of resolving our data. But to explain the experimental protocol, I ran the data from 2001 to 2009, 2001 to 2010, and 2001 to 2011 through fits where playoff experience was judged over a 1, a 2, a 3, and a 4 year span. The results are given below, in a table.

Table Explanation:

These are data derived from logistic regression fits, using Maggie Xiong’s PDL::Stats, to NFL playoff data. The data were taken from NFL.com. Start year is the first year of playoff data, end year is the last year considered. HFA is the magnitude of the home field advantage. SOS is the strength of schedule metric, as derived from the simple ranking system algorithm. Playoff experience was determined by examining the data and seeing if the team played a playoff game within “playoff span” years of the year in question. Assignment was either a 1 or 0 value, depending on whether the question was true or false. Dm/(n-p) is the model deviance divided by the number of degrees of freedom of the data set. As explained here, this parameter should tend to the value of 1. The p of the parameters above are the confidence intervals of the various fit values. It is better when p is small, and desired is a p < 0.05. P values greater than 0.05 are highlighted in blue.

Note that the best fits are found when the playoff experience span is the smallest. The confidence limits on the playoff parameter are the smallest, the model deviance is the smallest, the confidence limit of the model deviance is the smallest.  The best models result from the  narrowest possible definition of “playoff experience”, and this result is consistent across the three yearly spans we tested.

So where does this place the idea that the New York Giants were a playoff experienced  team in 2011? It places it in the land of the educated guess, the gut call, a notion coming from the same portion of the brain that drew a snake swallowing its own tail in the dreams of August Kekule. Sometimes intuition counts. But in the land of curve fitting, you have to publish  your best model, not the one you happen to like for the sake of liking it. The best model I have to date would be the one for the 2001 to 2011 data set, with a playoff experience band defined in terms of  a single year. It yields the following logistic formula:

logit P  =  0.668 + 0.348*(delta SOS) + 0.434*(delta Playoff Experience)

Compared to the previous formula, the probability resulting from a one unit difference in SOS now becomes 0.58 instead of 0.57 (see the Wolfram Alpha article for an easy way to transform logits into probabilities), but the value of playoff experience now becomes 0.606, instead of 0.68.

If there were one area I’d like to work on with regard to this formula, it would be to find a way to calculate the (dis)advantage of having a true rookie quarterback. I suspect this kind of analysis could be best done with counting. I don’t think a curve fit is necessary in this instance. I suspect a rookie quarterback adjustment would have allowed this formula to more accurately determine the potential winner in the Houston Texans – Cinncinnati Bengals game. After all, 10-1 is better than 9-2.

The playoffs are a funny bit of business, where people tend to assume the #1 seed has a really good chance of making it to the Super Bowl. That is, unfortunately, not even close to the truth. If you ignore home field advantage, then it becomes easy to see that in these circumstances, the #1 and #2 seeds have 1 chance in 8 of winning (0.125), whereas seeds 3-6 have a 1 in 16 chance of winning (0.0625). But since in the playoffs, there is a home field advantage (at least until you reach the Super Bowl), the actual odds from Seeds 1 to 6 vary quite dramatically.

For now, we’re going to assume a home field advantage of 0.60. From 2001 to 2010, 100 non-Super Bowl playoff games were played, and the home team won 60 of them. This year, the home team won every time, unless the visitor was named the New York Giants, leading to a record of 8-2. So, I guess, the running total now, from 2001 to 2011,  has to be 68/110, or 61.8% or so.

That said, I’m still going to use 60% in my calculations below.

For the sake of making it easier to turn any calculations into code, we’ll assign the home field advantage to the variable U (for “upper”), and to 1 – U, we will assign the variable L (for “lower”). Given these assignments, we now have:

Temporary variables:

LL = L*L
T23 = U*L + L*U
T45 = LL*U + (1. – LL)*L

Calculations of playoff odds

Seed 1 = U*U*0.50
Seed 2 = U*T23*0.50
Seed 3 = U*L*T23*0.50
Seed 4 = U*L*T45*0.50
Seed 5 = L*L*T45*0.50
Seed 6 = L*L*L*0.50

T23 is necessary to calculate the second game of Seed 2 or the third game of Seed 3. In this game, these two teams could face Seed 1, Seed 4, Seed 5, or Seed 6. Critically, they will either face Seed 1, for which they would be the visiting team, or all others, for which they would be the home team. The odds therefore become (odds of Seed 1 winning)(vistor’s odds) + (1 – odds of Seed 1 winning)(home team odds).

T45 is necessary to calculate the third game of Seed 4 or 5. In this game, these two teams could face Seed 1, Seed 2, Seed 3, or Seed 6. As Seed 6 is the only team for which Seeds 4 and 5 would be the home team, it is easiest to calculate the odds of Seed 6 making it to the third game, and then subtract those odds for the probability of playing as the visitors. Since the odds of Seed 6 arriving at game 3 are L*L, you end up with the formula given above.

Choosing a value of 0.60 for the home field advantage, we end up with:

Seed 1 : 0.18
Seed 2 : 0.144
Seed 3 : 0.0576
Seed 4 : 0.05184
Seed 5 : 0.03456
Seed 6 : 0.032

The range, from 18% to about 3%, is considerably more broad than the naive 1/8 to 1/16 values. Home field has a marked effect on the ability of teams to reach and win the Super Bowl. But the sheer number of teams involved, 12, and the arrangement of the playoffs, means that a #1 seed has, with a HFA of 60%, about a 36% change of making it to the Bowl, and a 18% chance of winning.

Note: this link has a coded version of the calculations above.

When you try to think of the NFL playoffs as simply an extension of the regular season, you screw up. Advantages that reliably yield wins under regular season conditions – think of the dominance of the San Francisco 49ers defense, at times, in the NFC Championship game two weeks ago – aren’t consistent enough in the post season. A lot of games are decided by, well, small effects, perhaps intangibles, at this time of year.

Part of the reason is that  the gap in the classical offensive and defensive metrics is much more narrowed in the post season; you’re looking at such small differences in net offensive potential that other elements come into play.  The other component, as far as I can  tell, is that traditional analysts, focused on the analysis of the regular season, are loathe to abandon tools that worked so well  on the 16 regular season games. If it’s 66-75% accurate during the regular season, isn’t that enough in the post season?

In my  opinion, the answer is no. Regular tools fail because the playoff system has already selected for teams  that are good at scoring and preventing scoring. Those teams are, to a first approximation, already well matched. You can’t use regular season tools reliably.  You have to  analyze  for playoff specific causes of wins and losses.

This is the only reason I can  come up with for the recent analyses of the strength of schedule metric. Analysts have  noted (see here and here) that it is negatively correlated with winning. This year has particularly potent effects, using Football Outsider’s definition of the SOS metric. Jim Glass, in the FO article, nails the effect on the head when he states:

The fact that stronger teams play easier schedules and weaker teams play tougher ones results trivially from the fact that teams cannot play themselves. As teams cannot play themselves, in lieu of doing so the strongest teams must play the weaker and the weakest the stronger.

This,  of course, begs the question that my playoff results pose: if strength of schedule correlates with losing, then why do playoff teams with advantages in the strength of schedule metric win? The confidence limit  of this effect is larger than the one for playoff experience, in my measurements. Given the right experimental design, this is pretty much a given.

Back in  the early 1990s, I used to call this  the “NFC East effect” and it seemed as obvious to me as the  nose on my face. The NFC East was the toughest division  in football. Whatever team won the NFC East was bound to win the Super Bowl because they had faced such incredibly  hard competition, that anyone else was a patsy by comparison (with the possible exception of the San Francisco 49ers). And whether any division could again gain such dominance, I don’t know. The salary cap has made it hard to hold such powerful teams together.

I’m posting now because the 2007 (and now 2011) New York Giants are a poster child for this phenomenon. My formula gave the New York Giants a 61% advantage in the 2007 Super Bowl. It is giving the Giants an advantage in this Super Bowl as well, by 66%. By traditional metrics, the 2011 Giants shouldn’t have survived so much as  their first playoff game. They managed, this year, to win three. The largest  measurable advantage they had  in this year’s playoffs is their exceptional strength of schedule.

So, win or lose, the question is still out there. If regular season stats are so important, why are the Giants winning? And if you’re using a “regular season” model to  predict playoffs, perhaps you need to step back and start analyzing the playoffs on their own, without preconception.

Follow

Get every new post delivered to your Inbox.

Join 243 other followers