Draft


The NFL draft is a kind of auction, with auction-like dynamics. It’s also akin to a marriage. It only takes one, not a crowd, to get married and the opinion of the one outweigh the many. When analyzing the draft, I’ve been known to say things like between three players of the same true value, the one that gets drafted is the one whose value is most overestimated (1). I’ve also said things like one scouting opinion isn’t important, but the envelope of opinions is. The distribution of those opinions is crucial to knowing when a player can be drafted (2).

The distribution of player rankings can affect the possible draft positions of a player.

The distribution of player rankings can affect the possible draft positions of a player. Hand drawn curves on a brand new pen tablet, so they’re not perfect curves. Imagine the purple curve with more extensive tails.

In the diagram above, there are three distributions, with different peaks, means and spreads. Player A, in black, has a tight distribution of values and barring any issues with uniqueness of position, there is a consensus where he will be drafted. Player B, in red, has a broader distribution, but is unlikely to suffer more than half a round of variance in draft position. Player C, in purple, has an envelope encompassing two whole rounds of the draft. It’s the player C types that create a lot of controversy.

Did_Dallas_Draft_A_Player_Or_A_Band_Im_Not_So_Sure

Travis Frederick and ZZ Top: Separated at birth?

The player the Dallas Cowboys drafted in the first round, Travis Frederick, is exactly one of those types. He was highly ranked before the combine, but suffered because of his bad 40 time. People like Gil Brandt, who had him ranked 27th best at the time, dropped him because of his 40 time. Perusing various links, such as this one, you see rankings ranging from 31 (Gary Horton) into the 90s. Now please note that draft pundits really don’t count, NFL teams do. But for the sake of argument, we’re using media scouts as an estimator of the envelope of NFL opinions. And that envelope of values encompass two whole rounds of variance.

So, what happens when you must have a player whose valuation envelope is a broad distribution? This player must be taken pretty far from the mean, in the tails of the “high value” side, or else you risk losing him (3). What is guaranteed though, is that the pundits on the other side of the mean from you will undoubtedly scream bloody murder. That’s because a draft pundit’s opinion is his life’s blood, and they make their money validating and defending that opinion, usually in print, and sometimes on television. That it’s one of many doesn’t matter if that’s how you make your living. So of course pundits will scream.

2013 was a draft with few good players. If estimates are valid that there were only 16 or so players truly worth a first round pick, then by default you’re overdrafting your quality by a half round by the middle of the first round. If the span of Frederick’s valuations really ran from, say, mid second to the beginning of the fourth, then the so-called overdraft is not, it’s entirely the function of three things: first, the perceived need for the player and second, such a broad valuation envelope that Dallas had to draft him in the tails of the distribution. Third factor, the lack of talent overall in the draft that led to overdrafting in general.

~~~
Footnotes

1. Jonathan Bales, before he became a New York Times contributor, favored this comment (common sense, IMO) and used it to help validate a pet drafting theory of his. I never saw enough rigor in his theory to separate it from the notions of BPA or need, as it was more a collective efficiency concept. IMO the notion hardly led to the invalidation of BPA or needs based drafting.

2. In the early 2000s, I wrote a Monte Carlo simulator of the draft, which explicitly used those distributions to estimate where players would be drafted. More discussion of that code, released as a Sourceforge project, is here.

3. Let me note that in “must have” situations, teams whose draft record no one complains about .. New England say .. will draft players above their worth. Belichick’s rationale, given in the link, is instructive. An excerpt is:

Now, the question is always, “How much do they like him and where are they willing to buy?’ I’m sure for some teams it was the fourth round. For some teams it was the third round. But we just said, ‘Look we really want this guy. This is too high to pick him, but if we wait we might not get him, so we’re going to step up and take him.’

PS – tskyler, a Cowboys Zone forum contributor, has a very nuanced fan analysis of the Frederick draft here, one worth reading.

This has been part of an ongoing conversation among Dallas fans, and perhaps among any of the 9 teams, from the Redskins to Patriots to the Vikings, that traded up in the first round of the 2012 NFL draft. There are some new tools for the analyst and the fan, and these include: (1) Pro Football Reference’s average AV per draft choice list, (2)  Pro Sports Transactions’ NFL draft trade charts, and (3) The Jonathan Bales’ article on Dallas Cowboys.com where he analyzes a series of first round trades up from 2000 to 2010. He concludes that in general, the trade up does not return as much value as it gives.

I suspect that Jonathan’s conclusion is also evident in the fantasydouche.com plot we reposted here. The classic trade chart of Jimmy Johnson really does overvalue the high end draft choices. You’re not paying for proven value, but rather potential when you trade up. I suspect by the break even metric we chose, comparing relative average AVs, that many draft trades never pay off, in part because people pay too much for the  value they receive. This is most evident in trading a current second or third and a future first for a current first round draft choice. These trades tend almost to be failures by design, and smack ultimately of desperation, true even when the player obtained (e.g. Jason Campbell) actually has some skills.

That said, how many of these players exceed the average abilities of the slot in which they were drafted? Now that we have the PFR chart, this is another question that can be asked of the first round players. Note that Jonathan Bales’ study doesn’t really answer the question of how good the player becomes, in part because the time frame chosen doesn’t allow the player adequate development. I started in the year 2000 1995, ended in the year 2007. I identified 67 players in that time frame, and I compared the AV for each player as given by the weighted average on the PFR player page. I’ll note that the player page and the annual draft pages do not agree on players’ weighted career accumulated value, so I assumed the personal pages were more accurate.

As far as a scale, we’re using the following:

AV relative to average Ranking
-25 AV or more Bust
-24 to -15 AV Poor
-14 to -5 AV Disappointing
-4 to +4 AV Satisfactory
+5 to +14 AV Good
+15 to +24 AV Very Good
+25 AV and up Excellent

 
Note there are some issues with the scale. Plenty of players from 1995 through 2007 are still playing, and their rankings are almost certainly going to change. In particular, Eli Manning at +24 and Jay Cutler at +23 have a great chance to end up scored as Excellent before the next season is over. Jason Campbell is at +19, and if he starts for a team for one season, he will end up with a ranking of Excellent. Santonio Holmes (+19) also has a shot at the Excellent category.

Players in the years 2006 and 2007 in lower categories (Manny Lawson at +7, Joe Staley at +4, Anthony Spencer at 0 ) could end up as Very Good, perhaps even Excellent if their careers continue.

The scoring ended up as

Scale Number Percent as Good Percent as Bad
Excellent 14 20.9 100.0
Very Good 9 34.3 79.1
Good 13 53.7 65.7
Satisfactory 10 68.7 46.3
Disappointing 7 79.1 31.3
Poor 5 86.6 20.9
Bust 9 100.0 13.4

 
Data came from the sources above. A PDF of these raw data is here:

NFL Trade Ups

Update: Increased the dates of players considered from 2000-2007 to 1995-2007. Moved Ricky Williams back to 1999.

There were, of course, two substantial trades of Ricky Williams. The first netted the Washington Redskins the whole of the Saints 1999 draft, plus the Saint’s first and third round picks of 2000. Three years later, Ricky was traded to the Miami Dolphins for a pair of first rounders, plus change. The first was obviously not paid off. How did the Miami Dolphins fare in their trade, using our new risk metrics?

Risk Ratio no longer makes sense as a term when you’re talking about someone already drafted. The important term becomes the net risk term, 52 AV. That’s 1 more AV than the typical #1 draft choice, and that’s the amount of AV Ricky had to generate in order for this trade to break even. And note, these calculations are derived from weighted career AV, not raw AV. So any raw AV we apply to these numbers is a rough approximation (A typical career summing to, say, 95 AV, might end up around 76 or so WCAV).

That said, Ricky Williams had a great first season with the Dolphins, generating 19 AV in that season alone. His total ended up somewhere around 57 AV. I’d suggest the second trade approximately broke even.

End notes: I’ve seen a lot of discussion around  this set of data, discussing the quality of draft picks on a per pick basis, posted in of all places, a Cav’s board. If this board isn’t the original source of these graphs, please let me know. An excellent resource for high quality NFL draft trade information is here. And finally, a reader named Frank Dupont writes:

I wrote a book about decision making in the NFL.  It’s sort of a pop science book because it seeks to make what happens in the NFL understandable via some work that people like David Romer, Richard Thaler, and Daniel Kahneman have done.  But because all pop science books make their point through narrative, I spend a lot of time looking at why football coaches are so old, but other game players like chess players and poker players are so young (Tom Coughlin is 65 and yet the #1 ranked chess player in the world is 21, the world’s best poker players are 25-ish).

The link for the book is here, if this topic sounds interesting to you. I’ll only note in passing  that while physics prodigies are common, biologists seem to hit their stride in their 60s.  Some areas of knowledge do not easily lend themselves to the teen aged super genius.

In my last post, I introduced ways to determine the risk of NFL trades, using Pro Football Reference’s average AV per draft slot metric to assess the relative risk of the trade. I wish to continue the work done in the first post, by also taking a look  at the Eli Manning trades, and also the Robert Griffin III trades.

Eli's debt will be paid off in 2 more years of his current level of play.
RG3 will have to have a Sonny Jurgenson-like career, all in Washington, to pay off the value of the picks used to select him.

In the Eli trade, the New York Giants assumed a ‘AV debt’ comparable to that of Michael Vick, and a relative risk approximately the same as Michael Vick or Julio Jones. Looking, Eli has  rolled up perhaps 87 AV at this point, netting 12-15 AV a year. So, in two years, in purely AV terms, this trade would be even. Please note  that NFL championships appear to not net any AV, so if the value of the trade is measured in championships instead, I’d assume the New York Giants would consider themselves the outright winners of this trade.

The appropriate comparison with the Robert Griffin III trade is actually the Earl Campbell trade. The risk ratio is about the same, assuming the Washington Redskins go 8-8 and 9-7 the next two years, and end up with the #16 pick in 2013, and  the #20 pick in 2014. The lowest  value the AV Given column could total is 106, if the Washington Redskins ended up with the #31 pick twice. In any event, the Redskins are betting that RG3 will have a Sonny Jurgenson-esque career, and not just his Washington Redskin career, but his Eagles and Redskins career, in order to pay back the ‘AV debt’ that has been accrued by this trade.

There were eight trades in the first day involving the first round of the 2012 NFL draft. Most of them involved small shifts in the primary pick, with third day picks added as additional compensation. The one outlying trade was that of the St Louis Rams and the Dallas Cowboys, which involved a substantial shift in  the #1 pick (from 6 to 14) and the secondary compensation was substantial. This high secondary compensation has led to criticism of the trade, most notably by Dan Graziano, whose argument, boiled to its essence, is that Dallas paid a 2 pick price for Morris Claiborne.

Counting  picks is a lousy method to judge trades. After all, Dallas paid a 4 pick price for Tony Dorsett. Was that trade twice as bad a trade as the Morris Claiborne trade?  The Fletcher Cox trade saw Philadelphia give up 3 picks for Fletcher Cox. Was that trade 50% worse than the Morris Claiborne trade?

In order to deal with the issues raised above, I will introduce a new analytic metric for analyzing trade risk, the risk ratio, which is the sum of the AV values of  the picks given, divided by the sum of the AV values of the picks received. For trades with a ratio of 1.0 or less, there is no risk at all. For trades with ratios approaching 2 or so, there is substantial risk. We are now aided in this kind of analysis by Pro Football Reference’s new average AV per draft pick chart. This is a superior tool to their old logarithmic fit, because while the data may be noisy, they avoid systematically overestimating the value of first round picks.

The eight first round trades of 2012, interpreted in terms of AV risk ratios.

The first thing to note about the 8  trades is that the risk ratio of 6 of them is approximately the same. There really is no difference, practically speaking, in the relative risk of the Trent Richardson  trade, or the Morris Claiborne trade,  or the Fletcher Cox trade. Of the two remaining trades, the Justin Blackmon trade was relatively risk free. Jacksonville assumed an extra value burden of 10% for moving up to draft the wide receiver. The other outlier, Harrison Smith, can be explained largely by the noisy data set and an unexpectedly high value of AV for draft pick 98. If you compensate by using 13 instead of 23 for pick #98, you get a risk ratio of approximately 1.48, more in line with the rest of the data sets.

Armed with this information, and picking on Morris Claiborne, how good does he  have to be for this trade to be break even? Well, if his career nets 54 AV, then the trade breaks even. If he has a HOF career (AV > 100), then Dallas wins big. The same applies to Trent Richardson. For the trade to break even, Trent has to net at least 64 AV throughout his career. Figuring out how much AV Doug Martin has to average is a little more complicated, since there were multiple picks on both sides, but Doug would carry his own weight if he gets 21*1.34 ≈ 28 AV.

Four historic trades and their associated risk ratios.

By historic measures, none of the 2012 first round trades were particularly risky. Looking at some trades that have played out in  the past, and one  that is still playing out, the diagram above shows the picks traded for Julio Jones, for Michael Vick, for Tony Dorsett, and also for Earl Campbell.

The Julio Jones trade has yet to play out, but Atlanta, more or less, assumed as much risk (93 AV) as they did for Michael Vick (94 AV), except for a #4 pick and a wide receiver. And although Michael is over 90 AV now, counting AV earned in Atlanta and Philadelphia, he didn’t earn the 90+ AV necessary to balance out the trade while in Atlanta.

Tony Dorsett, with his HOF career, paid off the 96 AV burden created by trading a 1st and three 2nd round choices for the #2 pick. Once again, the risk was high, the burden was considerable, but it gave value to Dallas in the end.

Perhaps the most interesting comparison is the assessment of the Earl Campbell trade. Just by the numbers, it was a bust. Jimmie Giles, the tight end that was part of the trade,  had a long and respectable career with Tampa Bay. That, along with the draft picks, set a bar so high that only the Ray Lewis’s of the world could possibly reach. And while Campbell was a top performer, his period of peak performance was short, perhaps 4   years. That said, I still wonder if Houston would still make the trade, if somehow someone could go back in to the past, with the understanding of what would happen into the relative future. Campbell’s peak was pretty phenomenal, and not entirely encompassed by a mere AV score.

These are numbers that have been published before, but not presented as artfully as this. PFR has the average draft value of a draft pick on a per draft slot basis. They then find a representative player with that AV. Also, they’ve listed the best picks in the slot as well. Looking to pick a world beater?

The url for this page suggests that  the page is temporary. Hopefully it will become a permanent part of Pro Football Reference.

Which of these players was drafted at a premium?

Sebastian Vollmer, drafted in the seond round in 2009.
Wikimedia image.

Derrick Burgess
Second round choice by Philadelphia in 2001.
Wikimedia image by BrokenSphere.

In my  mind, the answer is “both of them”.

One of the meatier passages in War Room comes in chapter 14, where Bill Belichick discusses the thought processes behind his selection of Sebastian Vollmer in 2009:

“Sebastian Vollmer is a good example”, he says of the Patriots’ starting right tackle, one of the team’s four second-rounders in ’09. “There’s no way he was really a second-round pick. Based on film or really based on the player he was at the end of the ’08 season. You know, East-West game and all  that. We knew there would be an undertow of Vollmer. And it was just a question of, ‘When’s this guy going to  go?’

“He should have been a fourth or fifth-round pick, by the film, by his performance. But  you saw him as an ascending player and he had rare size, and  there were a lot of things that you had to fix and all that. But it was clear the league liked him. Now,  the question is always, “How much do  they like him and where are they willing to buy?’ I’m sure for some teams it was the fourth round. For some teams it was the third round. But we just said, ‘Look we really want this guy. This is too high to pick him, but if we wait  we might not get him, so we’re going to  step up and take him.’

“And sometimes when you do that  you’re right and sometimes when you do  that  you’re wrong and everybody looks at you like, ‘Damn, you could have had him in the fourth.’

The Patriots aren’t the only team that practices the overdraft or the premium draft. If the Eagles really like someone, they tend to take them a round ahead of where he is commonly valued. Odd that teams that maintain plenty of draft picks practice this.  Offhand, I can recall the Eagles doing this for Derrick Burgess (generally viewed as a fourth rounder). The Steelers have done this as well;  they drafted Casey Hampton a full round above his common valuation.

In the 2012 draft class, players who appear to be attracting premium attention (we’re a day before the draft, mind you) are Ryan Tannehill (late first by talent, thought to be going to Miami at #8), Stephon Gilmore (drafted #3 in a mock draft by Greg Cosell), Fletcher Cox (mid first talent, seen as high as #5 in respectable mocks), Kevin Zeitner (mid second round talent, often in mocks with Pittsburgh in the first round), Chandler Jones (appearing in the first in some mocks), and Mark Barron (some people claim he’s the #7 now, often ranked as a mid first rounder).

If you feel you need the player, sometimes you have to just go out and get him.

I picked up this book after Greg Cosell gave it a big thumbs up on Rob Rang and Doug Farrar‘s radio show for KJR in Seattle, curious what it might actually say about the NFL draft.

Turns out this book is an update and rewrite of his earlier book, Patriot Reign, and for 11 of the chapters of this book, really has almost nothing to do about the draft, other than teasers spiked throughout the work. One interesting comment about the draft ranking system implemented by Belichick goes:

One of the things that made the system different was that it absolutely required a scout to know his college area or region of coverage in addition to each member of the Patriots’ fifth-three man roster. All reports, without exception, were comparative, and were based on what a given prospect could do vs. any current Patriot playing his position.

As a book, it initially has no sense of overarching storyline, content to wander about the narrative landscape the way a 60 year old grandfather would, telling one story in deep depth and then switching abruptly to another. It follows a variety of points of view. They all do not make much sense until you get to the end, where Michael actually starts talking more in depth about the draft in chapter 12. It finally becomes clear that he has three points of view, all intertwined, that of Belichick, that of Thomas Dimitroff, GM of the Atlanta Falcons, and that of Scott Pioli, GM of the Kansas City Chiefs. But to get there, to the three chapters of new material, he has you read about 11 chapters that I suspect were mostly all told in Patriot Reign.

Disturbing is the often myopic point of view of the book. Most notable in this regard is the coverage of Spygate, which can be summarized as (A) It was all Eric Mangini’s fault (B) Everybody does it and (C) People are picking on us needlessly and hurtfully. It’s in these segments where the book descends even from rambling history and becomes a fanboy lament. When you have to complain, in Poor Poor Pitiful Me fashion, about Gregg Easterbrook talking you down – in football terms, a comic, mind you – then you really do need to step out of the narrative a while and reexamine the facts. Tom, of the blog Residual Prolixity, puts it this way:

There are also a couple things Holley doesn’t seem to get, either from a Boston-centric viewpoint or they’re not obvious and nobody actually bothered to explain them to him, with the foremost example in my mind that Spygate (covered only briefly) exacerbated an existing anti-Boston sentiment arising from a belief that the Patriots were willing to push to the edge of the rules and beyond, if they could get away with it, which they could (see increase in illegal contact penalties, 2004, post Colts-Patriots).

All that said, once you get to Chapter 12, there are three chapters of useful insider stuff on how three teams conduct their draft. The background info on Dimitroff and Pioli are good enough to be useful to fans of the Falcons and Chiefs. Just, the new stuff isn’t substantial enough to be a book on its own – more like a long extended article in the New Yorker or the Washington Post. But, book sales being what they are, the new stuff was tacked onto the old stuff and sold as an entirely new product.

Up to you, whether you should read it. It can be interesting given the limitations of the material. Scaled in the measure of a draft pick, this is day two material.

It’s hard, sometimes, getting past the hopes, dreams, assumed truths, the outright lies, the hype and blather of the draft season and dig down to real assessments, real evaluations, and for that matter, the real news of the day. I don’t consider much of what Pro Football Talk reports as real news. I tend more to fellas like Calvin Watkins of ESPN Dallas, Dan Graziano of ESPN NFC East, and these days, Les Bowen of the Philadelphia Daily News.

"All we want are the facts ma'am"
Image from Wikimedia.

Les, yesterday, came out with a series of tweets with regard to Eagles GM Howie Roseman where he was content to play the straight man. Sometimes, we need that and it is appreciated. I am happy enough to let the Florios and Skip Baylesses of the world be my cynics. Sometimes I need a guy who doesn’t look up and horse laugh when a news figure delivers a dubious line. Les’s followup article is here. Dan Graziano’s take on Howie is here.

That said, Les isn’t without his wits. The best NFCE draft related tweet of yesterday was this one:

Roseman talks danger of taking guy high who is “the best of a bad bunch” rather than a really good player. I think this means no Mark Barron

About as soon as Dallas lost their last game of the season, a veritable consensus formed in fan circles about what Dallas should do: almost any fan mocker worth his salt had Dallas picking up Carl Nicks in free agency  and drafting  David DeCastro in the 14th. That this quinella might be hard to pull off didn’t faze the crowd, and arguing with any of these guys an amazing waste of time. I felt as  if I was looking at the daily barrage of “Patrick Peterson falls to the Boys” exuberance all over again.

As @FO_MTanier has noted on Twitter,  this interest in DeCastro spills over into the media as well.

It’s taken perhaps a month, but those fans that claim “insider” connections, and are respected in general for, perhaps, actually having those connections are saying now that the Boys are more looking for a center in free agency and will let the guards they have develop.  Costa is regarded as the weak link, not the collection of  talent at guard. Further, Steven Jones has said  that the defense needs work, and media/fan draft interest is beginning to shift to others, people such as Melvin Ingram, Luke Kuechly and Dontari Poe.

Is this man a future Redskin? Image from Ceasarscott of Wikimedia.

The recent combine workout, including a 4.38 40 times of Robert Griffin III has changed the fan status of RG3 in  the eyes of Redskins fans to something approaching blowtorch heat. It hasn’t been mellowed by Saint Louis openly shopping the second pick. The first pick is pretty much assumed to be Oliver Luck, but a fella with a cannon arm, clear intelligence and Vick-like speed leads people more and more to think that Mr. Griffin will be, at worst, a poor man’s Vick. And if he’s more judicious with his throws than Michael, learns the game more intimately, well then all the better. There is now the smell of potential top 10 QB around RG3.

Four teams are thought to be interested in Robert Griffin: Cleveland, Washington, Miami, and Seattle. How much of that  is real, how much of that  is assumed,  I can’t tell presently. Talk radio has Cleveland in the driver’s seat for a trade, as it has 2 number 1 picks, and the #4 pick as well. The Skins, by contrast, have only the #6 pick.

A #2 pick is worth 2600 points on the JJ chart. Cleveland’s #4 is worth 1800, and their #22 pick is worth 780, about even in value. That said, Peter King is making comparisons to the trade for Ryan Leaf, which netted 2 firsts, a second, and Eric Metcalf. Already, in Redskins fan circles, people are saying they wouldn’t pay 3 #1s for RG3, but if the Leaf trade sets the benchmark, I’d suggest that the equivalent of three #1 choices is the going rate for a potential top 10 QB.

The Redskin’s first round choice is worth 1600 points. How do they make up 1000 points without at least giving up another #1? Beyond  that, what sweetener could they give that would make their trade better than Cleveland’s two #1s?

Useful links:

NFP’s take on Dallas team needs.

NFP’s take on Redskin team needs.

NFP’s  take on Eagles team needs.

NFP’s take on Giants’ team needs.

Next Page »

Follow

Get every new post delivered to your Inbox.

Join 244 other followers