It isn’t about you, draft pundits: thoughts on the NFL draft.

The NFL draft is a kind of auction, with auction-like dynamics. It’s also akin to a marriage. It only takes one, not a crowd, to get married and the opinion of the one outweigh the many. When analyzing the draft, I’ve been known to say things like between three players of the same true value, the one that gets drafted is the one whose value is most overestimated (1). I’ve also said things like one scouting opinion isn’t important, but the envelope of opinions is. The distribution of those opinions is crucial to knowing when a player can be drafted (2).

The distribution of player rankings can affect the possible draft positions of a player. Hand drawn curves on a brand new pen tablet, so they’re not perfect curves. Imagine the purple curve with more extensive tails.

In the diagram above, there are three distributions, with different peaks, means and spreads. Player A, in black, has a tight distribution of values and barring any issues with uniqueness of position, there is a consensus where he will be drafted. Player B, in red, has a broader distribution, but is unlikely to suffer more than half a round of variance in draft position. Player C, in purple, has an envelope encompassing two whole rounds of the draft. It’s the player C types that create a lot of controversy.

Travis Frederick and ZZ Top: Separated at birth?

The player the Dallas Cowboys drafted in the first round, Travis Frederick, is exactly one of those types. He was highly ranked before the combine, but suffered because of his bad 40 time. People like Gil Brandt, who had him ranked 27th best at the time, dropped him because of his 40 time. Perusing various links, such as this one, you see rankings ranging from 31 (Gary Horton) into the 90s. Now please note that draft pundits really don’t count, NFL teams do. But for the sake of argument, we’re using media scouts as an estimator of the envelope of NFL opinions. And that envelope of values encompass two whole rounds of variance.

So, what happens when you must have a player whose valuation envelope is a broad distribution? This player must be taken pretty far from the mean, in the tails of the “high value” side, or else you risk losing him (3). What is guaranteed though, is that the pundits on the other side of the mean from you will undoubtedly scream bloody murder. That’s because a draft pundit’s opinion is his life’s blood, and they make their money validating and defending that opinion, usually in print, and sometimes on television. That it’s one of many doesn’t matter if that’s how you make your living. So of course pundits will scream.

2013 was a draft with few good players. If estimates are valid that there were only 16 or so players truly worth a first round pick, then by default you’re overdrafting your quality by a half round by the middle of the first round. If the span of Frederick’s valuations really ran from, say, mid second to the beginning of the fourth, then the so-called overdraft is not, it’s entirely the function of three things: first, the perceived need for the player and second, such a broad valuation envelope that Dallas had to draft him in the tails of the distribution. Third factor, the lack of talent overall in the draft that led to overdrafting in general.

~~~
Footnotes

1. Jonathan Bales, before he became a New York Times contributor, favored this comment (common sense, IMO) and used it to help validate a pet drafting theory of his. I never saw enough rigor in his theory to separate it from the notions of BPA or need, as it was more a collective efficiency concept. IMO the notion hardly led to the invalidation of BPA or needs based drafting.

2. In the early 2000s, I wrote a Monte Carlo simulator of the draft, which explicitly used those distributions to estimate where players would be drafted. More discussion of that code, released as a Sourceforge project, is here.

3. Let me note that in “must have” situations, teams whose draft record no one complains about .. New England say .. will draft players above their worth. Belichick’s rationale, given in the link, is instructive. An excerpt is:

Now, the question is always, “How much do they like him and where are they willing to buy?’ I’m sure for some teams it was the fourth round. For some teams it was the third round. But we just said, ‘Look we really want this guy. This is too high to pick him, but if we wait we might not get him, so we’re going to step up and take him.’

PS – tskyler, a Cowboys Zone forum contributor, has a very nuanced fan analysis of the Frederick draft here, one worth reading.

 

Super Bowl odds

Of all the teams in the NFC playoffs, the San Francisco 49ers had the best strength of schedule, as measured by the simple ranking system. Of all the teams in the AFC playoffs, the Baltimore Ravens had the best strength of schedule, as measured by the simple ranking system. But San Francisco’s SOS is markedly higher than Baltimore’s, to the point our system favors San Francisco by around 7.5 points.

 

2013 Super Bowl
NFC Team	AFC Team	NFC Win Pct	Est. Point Spread
SF	BAL	0.735	7.5

 

I suspect if Atlanta had won, we would be asking ourselves the question of whether SOS can be fooled. Advanced NFL Stats said, among other things, that Carolina was seriously underrated. If that were true of the whole NFC South, the Atlanta was actually playing better teams than their rankings suggested, and thus should have been more highly rated. But in the end, with 1:18 left to play, 3rd and 4 on the San Francisco 10 yard line, Atlanta was unable to get a first down, and San Francisco won a tough fought victory by 4 points. Two pivotal plays will markedly affect the narrative of this game.

Now to note, last year the New York Giants had the best strength of schedule of all the playoff teams, and they also won the Super Bowl. So I have to ask myself, at what point does this “coincidence” actually make it into the narrative of the average sports writer, or do they still keep talking about “teams of destiny” or other such vague language? Well, this kind of “sports journalist talk” hasn’t gone away in sports where analytics is an ever bigger factor in the game, sports like baseball or basketball. I suspect it doesn’t disappear here.

 

Baltimore; the perfect playoff model; conference championship odds.

I suspect  to a first approximation almost no one other than Baltimore fans, such as Brian Burke, and this blog really believed that Baltimore had much of a chance(+). Well, I should mention Aaron Freeman of Falc Fans, who was rooting for Baltimore but still felt Denver would win. Looking, his article is no longer on the Falcfans site. Pity..

WP graph of Baltimore versus Denver. I tweeted that this graph was going to resemble a seismic chart of an earthquake. Not my work, just a screen shot off the excellent site Advanced NFL Stats.

After a double overtime victory by 3 points, it’s awfully tempting to say, “I predicted this”, and if you look at the teams I’ve  favored, to this point* the streak of picks is 6-0. Let me point out though, that you can make a limiting assumption and from that assumption figure out how accurate I should have been. The limiting assumption is to assume the playoff model is 100% accurate** and see how well it predicted play. If the model is 100% accurate, the real results and the predicted results should merge.

I can tell you without adding up anything that only one of my favored picks had more than a 70% chance, and at least two were around 52-53%. So 6 times 70 percent is 4.2, and my model, in a perfect world, should have picked no more than 4 winners and 2 losers. A perfect model in a probabilistic world, where teams rarely have 65% chances to win, much less 100%, should be wrong sometimes. Instead, so far it’s on a 6-0 run. That means that luck is driving my success so far.

Is it possible, as I have argued, that strength of schedule is an under appreciated playoff stat, a playoff “Moneyball” stat, that teams that go through tough times are better than their offense and defensive stats suggest? It’s possible at this point. It’s also without question that I’ve been lucky in both the 2012 playoffs and the 2013 playoffs so far.

Potential Championship Scenarios:

 

Conference Championship Possibilities
Home Team	Visiting Team	Home Win Pct	Est. Point Spread
NE	BAL	0.523	0.7
HOU	BAL	0.383	-3.5
ATL	SF	0.306	-6.1
SF	SEA	0.745	7.9

 

My model likes Seattle, which has the second best strength of schedule metric of all the playoff teams, but it absolutely loves San Francisco. It also likes Baltimore,  but not enough to say it has a free run throughout the playoffs. Like many modelers, I’m predicting that Atlanta and Seattle will be a close game.

~~~

+ I should also mention  that Bryan  Broaddus tweeted about a colleague of his who predicted a BAL victory.

* Sunday, January 13, 2013, about 10:00am.

** Such a limiting assumption is similar to assuming the NFL draft is rational; that the customers (NFL teams) have all the information they should, that they understand everything about the product they consume  (draft picks), and that their estimates of draft value thus form a normal distribution around the real value of draft picks, and that irrational exuberance, or trends, or GMs falling in love with players play no role in picking players. This, it turns out, makes model simulations much easier.

 

2012 Season NFL Divisional Playoff Odds

Though the results for the divisional round are embedded in the image of my playoff spreadsheet in my previous article, the table below is certainly easier to read.

 

Divisional Playoff Round
Home Team	Visiting Team	Home Win Pct	Est. Point Spread
DEN	BAL	0.477	-0.7
NE	HOU	0.638	4.2
ATL	SEA	0.462	-1.1
SF	GB	0.700	6.3

 

I suspect other systems will rank Seattle as stronger than mine does, and Baltimore as weaker. That said, the Vegas line as of this Sunday gives Atlanta a 2 point advantage over Seattle, and my system slightly favors Seattle. We can calculate odds and points via other mechanisms, say, Pythagoreans, SRS and median point spreads, and if we do, what do we get?

 

Atlanta Versus Seattle
Technique	Home Win Pct	Est. Point Spread
Median Point Spread	0.632	4.0
Simple Ranking System	0.407	-2.8
Pythagorean Expectation	0.486	-0.4

 

Certainly different systems yield different emphases. For me, the one lasting impression I had was the Washington Seattle game was an almost picture perfect demonstration that home field advantage is strongest in the first quarter.

Of all the teams playing, my system likes San Francisco the best. I suspect it likes it more than others. We’ll learn more as other analytics oriented folks post their odds for the divisional round.

 

The beginnings of the NFL Playoffs. Some Odds.

We can’t work with my playoff model without having a set of week 17 strength of schedule numbers, so we’ll present those first.

Between a difficult work schedule this last December and a very welcome vacation (I keep my stats on a stay at home machine), I haven’t been giving weekly updates recently. Hopefully some of my various thoughts will begin to make up for that.

Though with SOS values, you could crunch all the playoff numbers yourselves, this set of data should help in working out the possibilities:

Odds as calculated by my formula, with home field advantage adjusted to 60%. Point spread calculated with formula 3.0*logit(win probability)/logit(0.60). Click on image twice to expand.

What I find interesting is the difference between Vegas style lines, and my numbers, and the numbers recently posted by Brian Burke on the New York Times Fifth Down blog. My model is very different from Brian’s, but in three of the four wild card games, our percentage odds to win are within 2-3 percent of each other.

Point spreads were estimated as follows: if an effect of 60% were valued at 3 points (i.e. playoff home field advantage is about 60% and home field advantage is usually judged to be worth 3 points), then two effects of that magnitude should be worth 6 points. But it’s only on a logit scale that these effects can be added, so it only makes sense to relate probabilities of winning through their logits. As the logit of 0.60 is about 0.405465, then an estimated point spread can be had with the formula

point spread = 3.0*logit(win probability)/0.405465

Update (1/9/2012) – even simpler is:

est. point spread = 7.4*logit(win probability)

A simplified table of the wild card games, with percentages and estimated point spreads is:

Wild Card Playoff Round
Home Team	Visiting Team	Home Win Pct	Est. Point Spread
GB	MIN	0.682	5.6
WAS	SEA	0.482	-0.5
HOU	CIN	0.642	4.3
BAL	IND	0.841	12.3

How many successes is a touchdown worth?

We’ve spoken about the potential relationships between success rates, adjusted yards per attempt, and stats like DVOA here, but to make any progress, you need to consider possible relationships between successes and yards. Let me point out the lower bound of the relationship is known, as 3 consecutive successes must yield at least 10 yards, and 30 consecutive successes must end up scoring a touchdown. In this case, the relationship is 1 success is equal to or greater than 3 1/3 yards.

Thus, if the surplus value of a touchdown is 20 yards, that’s 6 successes. If a turnover is worth 45 yards, that’s about 13.5 successes.

A smarter way to get at the mean value of this kind of relationship, as opposed to a limiting value, would be to add up the yards of all successful plays in the NFL and divide by the number of those plays. For now, that’s something to be pursued later.

 

Is DVOA just a kind of normalized “adjusted net success rate per attempt”?

Ok, this whole article is a kind of speculation on my part. DVOA is generally sold as a kind of generalization of the success rate concept, translated into a percentage above (or below) the norm. Components of DVOA include success rate, turnover adjustments, and scoring adjustments. For now, that’s enough to consider.

Adjusted yards per attempt, as we’ve shown, is derived from scoring models, in particular expected points models, and could be considered to be the linearization of a decidedly nonlinear EP curve. But if I wanted to, I could call AYA style stats the generalization of the yardage concept, one in which scoring and turnovers are all folded into a single number valued in terms of yards per attempt.

So, if I were to take AYA or its fancier cousin ANYA, and replace yards with success rate, and then refactor turnovers and scoring so that turnovers and scoring were scaled appropriately, I would end up with something like the “V” in DVOA. I could then add a SRS style defensive adjustment, and now I have “DV”. If I now calculate an average, and normalize all terms relative to my average, I’d end up with “Homemade DVOA”, wouldn’t I?

The point is, AYA or ANYA formulas are not really yardage stats, they are scoring stats whose units are in yards. So, if really, DVOA is ANYA in sheep’s clothing, where yardage has been replaced by success rate, with some after the fact defense adjustments and normalization from success rate “units”.. well, yes, then DVOA is a scoring stat, a kind of sophisticated and normalized “adjusted net success rate per attempt”.

 

At the 3/4 mark in the NFL season

Things that are easy to note: the teams with at least 9 wins are either guaranteed a playoff birth, or have, at worst, a 99% chance of making the playoffs. The teams with 8 wins have a very good chance of entering the playoffs. Those teams with 7 wins have at least a 50% chance of making the playoffs. Those with 6 wins have between a 5% to 30% chance of making the playoffs. Let’s say they are hoping to get in.

Data from week 12

Data from week 13

The methodology of these stats is discussed in previous posts in this series. If you’re wondering where I’m getting odds to go into the playoffs, see this post. If you’re wondering what chance your team has of winning in the playoffs, see this post on my logistic regression methods, based on studies of playoff games. How would your ranking in the playoffs affect your chances of getting into the Super Bowl? We studied that here.

I am not a proponent of the notion that regular season offensive stats are predictive in the post season. My studies suggest p on the order of 0.15 for offensive stats in the post season, and thus aren’t predictive enough for my tastes ( p <= 0.05). That hasn't stopped Football Outsiders from pretending that their proprietary stats are predictive and calculating playoff odds with their tools.

 

Thoughts on NFL week 11

Over some five years, the whole of the Matt Ryan – Mike Smith era, Atlanta has had a habit of outperforming its Pythagoreans:

Atlanta outperforming its Pythagoreans
Year	WL%	Pythag	Delta
2008	69	62	7
2009	56	56	0
2010	81	72	9
2011	63	59	4
2012 (to date)	90	71	19

 

But they’ve never outperformed their Pythagoreans as substantially as they have this year. It can’t be blamed on early season New Orleans collapse, as their only loss was inflicted by New Orleans. New Orleans has only hindered this process. Is it turnover that are causing all this? While the 2010 team had a +14 turnover ratio and the 2011 team had a +8 turnover ratio, the 2012 team has only a +5 turnover ratio at this point and the 2008 team had a -3 turnover ratio. No, it’s something else. For now, perhaps noting that this team tends to outperform its Pythagoreans is enough.

Week 11 scoring stats:

Chicago’s biggest weakness was on display this Monday night, as Aldon Smith had a career day. Aaron Schatz (@FO_Schatz) has sent digging into his archives for the biggest DVOA blowouts of all time. The 32-7 demolition of the Bears by the 49ers wasn’t the worst, but it clearly evoked the worst.

The game plan was heavy on traps and wham blocks, and would have warmed the hearts of anyone who ever played NFL Strategy against a blitz heavy opponent.

It does lead to the question of whether Chicago is in the same downward spiral they experienced last year. At this point, however, you would expect Jay Cutler to return and thus slow down the bleeding.

 

Playoff Predictions, 2012 style

The three sites we noted last year: Cool Standings, Football Outsiders, and NFL Forecast, are at it again, providing predictions of who is going to be in the playoffs.

 

Cool Standings uses Pythagoreans to do their predictions (and for some reason in 2011, ignored home field advantage), FO uses their proprietary DVOA stats, and NFL Forecast uses Brian Burke’s predictive model.

Blogging the Beast has a terrific article on “the play”. If you watched any Dallas-Philadelphia games in 2011, you’ll know exactly what I mean, the way with a simple counter trap, LeSean McCoy treated the Cowboys line as if it were Swiss cheese.

Most important new link, perhaps, is a new Grantland article by Chris Brown of Smart Football. This article on Chip Kelly is really good. Not only is the writing good, but I love the photos:

Not my photo. This is from Chris Brown’s Chip Kelly article (see link in text).

as an example. Have you ever seen a better photo of the gap assignments of a defense?

 

Week Ten thoughts

I believed, in the immediate aftermath of the 2011 season, that with Jason Peters at left tackle, the least of Philadelphia’s worries would have been the tackle position. Instead, he was injured in the off season. In September, Philadelphia center Jason Celce went down with a season ending injury. In the New Orleans game, Todd Herremans suffered a season ending injury, and going into the Dallas game, starting guard Danny Watkins had been out with a sprained ankle.

Losing Todd Herremans: deal breaker for the Eagles? (Image from Wikimedia).

So, in week 10, the Eagles had one healthy starting caliber player, and 4 backups playing on the offensive line. This loss of talent was profound, even in comparison with Dallas, which had 1 backup on the line – though Dallas RG Mackenzie Bernadeau has been pretty marginal as a starter. Simplified, losing tackles is much worse than losing a guard and a center. Result? A markedly ineffective Vick, a thoroughbred offense reduced to dog-sled pace.

No wonder announcers were hyping this as the “end of a season” for one of these teams. Most any cold blooded announcer could have figured out what was about to happen. The only question was how best to pitch it so people would actually watch.

Atlanta: I’ve been comparing the 2012 Atlanta Falcons to the 1976 Oakland Raiders, to make the case that Atlanta has a chance. But the 1976 Raiders had made it to three previous Conference Championship games, while the Mike Smith squads have never gone that far. They lack the deep playoff experience of those 1970s Raiders squads.

The fact is, all scoring stats suggest Atlanta has benefited from plenty of luck. I think, because of a better Julio Jones, that this is a better Falcons team than the 2011 team, but the coaching changes in New Orleans markedly benefited this squad. Yes, Atlanta can be beaten.

Week 9 scoring stats:

Week 10 scoring stats:

If we use the median point spread as a measure of how good Atlanta is, and select the teams within 2 points of their value, you end up with a group that includes San Francisco, New England, Minnesota, and the New York Giants. That’s a talented group of teams, but perhaps not as terrifying as Green Bay, Houston, Denver, and Chicago. Pythagoreans point out three elite teams in Houston, Chicago, and San Francisco, while simple rankings prefer the quartet of Houston, Chicago, Denver and San Francisco.

At this point, perhaps the more appropriate past comparison for the Falcons would be the 1973 Oakland Raiders. Atlanta needs to make some noise in the playoffs first.

Should anyone be worried about the Giants mid season slide? No. They always do this. The question is, will they fully recover in time to make a playoff run. That’s not something that will be entirely answered until week 17.