When you try to think of the NFL playoffs as simply an extension of the regular season, you screw up. Advantages that reliably yield wins under regular season conditions – think of the dominance of the San Francisco 49ers defense, at times, in the NFC Championship game two weeks ago – aren’t consistent enough in the post season. A lot of games are decided by, well, small effects, perhaps intangibles, at this time of year.
Part of the reason is that the gap in the classical offensive and defensive metrics is much more narrowed in the post season; you’re looking at such small differences in net offensive potential that other elements come into play. The other component, as far as I can tell, is that traditional analysts, focused on the analysis of the regular season, are loathe to abandon tools that worked so well on the 16 regular season games. If it’s 66-75% accurate during the regular season, isn’t that enough in the post season?
In my opinion, the answer is no. Regular tools fail because the playoff system has already selected for teams that are good at scoring and preventing scoring. Those teams are, to a first approximation, already well matched. You can’t use regular season tools reliably. You have to analyze for playoff specific causes of wins and losses.
This is the only reason I can come up with for the recent analyses of the strength of schedule metric. Analysts have noted (see here and here) that it is negatively correlated with winning. This year has particularly potent effects, using Football Outsider’s definition of the SOS metric. Jim Glass, in the FO article, nails the effect on the head when he states:
The fact that stronger teams play easier schedules and weaker teams play tougher ones results trivially from the fact that teams cannot play themselves. As teams cannot play themselves, in lieu of doing so the strongest teams must play the weaker and the weakest the stronger.
This, of course, begs the question that my playoff results pose: if strength of schedule correlates with losing, then why do playoff teams with advantages in the strength of schedule metric win? The confidence limit of this effect is larger than the one for playoff experience, in my measurements. Given the right experimental design, this is pretty much a given.
Back in the early 1990s, I used to call this the “NFC East effect” and it seemed as obvious to me as the nose on my face. The NFC East was the toughest division in football. Whatever team won the NFC East was bound to win the Super Bowl because they had faced such incredibly hard competition, that anyone else was a patsy by comparison (with the possible exception of the San Francisco 49ers). And whether any division could again gain such dominance, I don’t know. The salary cap has made it hard to hold such powerful teams together.
I’m posting now because the 2007 (and now 2011) New York Giants are a poster child for this phenomenon. My formula gave the New York Giants a 61% advantage in the 2007 Super Bowl. It is giving the Giants an advantage in this Super Bowl as well, by 66%. By traditional metrics, the 2011 Giants shouldn’t have survived so much as their first playoff game. They managed, this year, to win three. The largest measurable advantage they had in this year’s playoffs is their exceptional strength of schedule.
So, win or lose, the question is still out there. If regular season stats are so important, why are the Giants winning? And if you’re using a “regular season” model to predict playoffs, perhaps you need to step back and start analyzing the playoffs on their own, without preconception.