Code and Football

Amazon.com, if you guys don’t know, is a  terrific source of penny books, books that third parties sell through Amazon for a penny. I’ve picked up a slew of books that way, and I’m reading quite a few of them. Of them, the most promising is “Education of a Coach” by David Halberstam, but I’m not going to talk about it until I finish some film study of the 1990 Super Bowl. David Halberstam has said some things about that game that absolutely deserve video study, so I’m in the middle of doing that. In  the meantime, I’ll mention a book I picked up for sentimental reasons, my father’s only football book when I was young, the book “Championship Football”, by Dana X. Bible.

Originally copyrighted in 1947, and a fifth printing from 1949, the book is old. The book is musty, and it’s a library copy from a junior high near Texarkana Texas. For all I know, this book was held by H Ross Perot some time in the past.

My first impression was how relevant the material was. Not the bits on strategy, but other things, like plays, like tips on playing, tips on blocking technique, tips on tackling a quarterback higher, so that you pin his arms to his body. Photographs, such as Bobby Lane showing people how to pass the ball, and players in 2, 3, and 4 point stances, are useful. Someone needs to resurrect this book, at least in PDF form, if nothing else.

The late 40s were a period when the T formation was coming into common use, but the older formations, such as the single and double wing, long and short punt formations, were still around. Dana talks knowledgeably about spinning backs, or spinners.

Dr Z: Vick as back 4, Duckett as 3, Dunn as 1, Koslowski as 2.

Spinners? You are a Dr Z fan, aren’t you? Can’t you remember Dr Z, Paul Zimmerman, trying to convince Dan Reeves to use Michael Vick as a single wing tailback, and growing sentimental about spinners?

In my dream, I lined up Vick at the run-pass tailback spot. T.J. Duckett was my spinning fullback. Warrick Dunn was the wingback, and Brian Kozlowski, normally the second tight end, was the blocking back. I just couldn’t shake the vision. Finally I called Dan Reeves, who was the Falcons’ coach at the time. Of all the loony calls he’s ever gotten, this must have ranked right up there. Anyway, I laid it all out for him, ending with Duckett as the old Michigan-style spinning fullback.

“What’s a spinning fullback?” Reeves said, and I realized that I was either real old or just dopier than usual. The idea never got any further, but I still think something like that would light up the sky.

Well, Dana has a whole page (or more) on spinners, and not just spinners, he offers a variety of formations on those pages.

Zone defenses, 1940s style. The 5-4 is what we'd call a 34 today.

Curious about the origins of scouting? Wonder how it was done before Steve Belichick came out with his scouting book? I found it interesting to compare pages between the older and the newer book. Seem familiar?

Scouting tips from Steve's book

Scouting tips from Dana's book.

Of course, strategy in those days was incredibly conservative, as offenses were not really well developed in  those days.This is a strategy chart from Dana’s book.

That chart did more to throw me for a loop than anything else, though when teams would quick kick here and there, I was at least ready for the tactic.

Upshot? Terrific book. It probably needs to be preserved in a PDF version, one that won’t age over time. These library copies won’t be around forever.

Update: Google Books has a scan of this book.

I’ve gotten some interesting feedback with regard to my first noise simulation study (see here and here), and wanted to touch base on some ideas, and  then get to a point I actually consider important. I’ve been talking about draft noise, or scouting error, or draft error, without exactly defining the phrase. I’m probably not going to satisfy the definition hounds here either, but I’ll approach the notion of value and draft value and then say what I’m calling draft noise might also be fruitfully called draft variability, since really it’s the variability of the value of a potential draftee that we’re after.

We know this exists. We know draft value, in this context, changes. This value could be related to athletic potential, but that’s another abstract quantity that isn’t measurable, and misses the point that the draft is a market. And as it’s a market, the currency is the draft slot used to ‘purchase’ a player. Pretty simple, huh? Just as you might pay 50 cents to purchase an apple, Reggie Bush is worth the #2 draft slot. That’s Reggie Bush’s value, in the context of the draft.

One approach for getting at the value and variability of  players, we discussed here in the context of Monte Carlo draft simulations.  Simply collect a group of scouts and let them all rank the players, take the mean and calculate the standard deviations of their estimates and you’ve got a normally distributed estimate of the draft variability of the player.  Another was the ‘model  the envelope of the apparent noise’ approach of the first simulation study. Rick Reilly uses a third approach in his recent re-draft article, which I believe really isn’t valuing the draft as a marketplace. He’s assessing the player’s performance after they were drafted. Mel Kiper’s language in this article is not only true, but utterly on the mark.

 the obvious complaint we always hear is, “You can’t really grade a draft for a few years.” Not true. You can’t assess the performance of the players drafted for a few years, but you can assess the degree to which teams maximized value while filling needs.

To flip back to an analogy, I buy an apple. I pay 50 cents for the apple. I bite into the apple and get bruised flesh in my bite, and I say, “That was a lousy apple for the price.” That assessment doesn’t change the price. The price was 50 cents.   Since the price exists, the kinds of analysis that Mel does is legitimate. He’s not analyzing the post draft performance of a player, he’s analyzing the market, and too much “draft” analysis forgets what the draft is.

One critique of my  previous noise model is  that there wasn’t a unique valuation for each individual team. Well, this is the deal. That’s per-player variability built into the model, and misses the point of the original study, which was to get a rough measure of the variability. However, the critique points out something else, and that draft noise, no matter how accurately  teams scout, is never completely going away. In a word, it’s irreducible.

Let’s pick a player out of the blue, call him,  oh Von Miller, and  say that scout A from team A and scout B from team B each rank Von Miller as the 4th best athlete of 2011. But the scout from team A says, “We look for large linemen and large linebackers and so Von Miller isn’t big enough. On  our scale, he’s worth a 15th pick.” Scout B from team B says, “We play a Miami 4-3, and we optimize our teams for pursuit and speed, and Von Miller is faster than greased lightning. We rank him 3rd, because he is a perfect fit to our requirements.” And therein lies a source of variability that will never go away.

Teams have physical and athletic requirements based on the offenses they play and the kinds of players they know can play in them. Scouts are taught to seek and value players based on those requirements. This creates variability in the market, hence noise. As we’ve said previously, that noise can be exploited. And that noise is never going away, no matter how accurately scouts slot players to their system.

There is a sense of embarrassment that pervades Frank’s book, one that could perhaps be explained by the fact  that David Halberstam was planning on writing a book about the 1958 NFL championship game. But it seems deeper than that. He talks about the salaries the pros made in the 1950s, the failures on the field, the sense of embarrassment that he couldn’t win for his dad, his peripatetic childhood. As a focused study of the game, well, it isn’t. It’s an older man’s book, broad in scope, a little rambling and talkative.

And in that is the strength of the story, which captures a snapshot in time that doesn’t exist anymore. No, I haven’t read extensively about 1950s football, and for someone who hasn’t, it can be a fascinating glimpse at their lives, the character of 1950s New York City. Further, Frank talks candidly about the failures in leadership of the period, strips away common myths about the way the champion Giants worked, and in doing so, exposes the growing character of two towering football figures, Vince Lombardi and Tom Landry.

Cowboys fans might find this bit of text fascinating:

For my first two years , I played defense more than offense, which meant I was playing with Landry, who was even then a player-coach. So I knew how rigid, strict, and unyielding he was as a coach.

Actually,  in  one game against the Redskins, I made an interception and lateraled the ball to Tom, who ran it in for a touchdown. On the following Tuesday, we watched the film.

“Gifford, was that the coverage?”

“I know, Tom, but they were in a Brown right, L-split,” I started to explain, “and-”

“There are no ‘buts'”

“But what if –”

“There are no what-ifs”

If you didn’t play the defense Tom’s way, end of conversation.

“He had a computer mind”, is how Huff remembers Landry. “He studied the opposition’s offensive frequencies in various situations, and he taught them, and you studied them. He’d always say, ‘You have to believe, You  gotta believe. I’ll put you in position to make the play, trust me.’ If you weren’t in position, and making the moves he’d given you, he’d give you ‘The Look’. He didn’t have to say anything: you could read his mind, and what he was saying was ‘You dumb-ass.'”

Vince Lombardi? Gifford expresses a great deal of skepticism about Lombardi’s portrayal in Maraniss’s book, because Frank never saw Lombardi as dictator.  Lombardi was, Gifford claimed, a much more approachable man when Vince was their offensive coach.

And so it goes. The book is peppered with those kinds of details. As an example, Lenny Moore always kept a miniature bible in his thigh pad. Perhaps the most evocative writing is a description of the 1950s New York City night life, dominated by saloons, and the search for places where someone could pick up some or all of the player’s tab. After such a fine bit of work on the times, the setting, the game itself tends to fade into the background. Perhaps, this game has been so intensely covered that most of us in the hard core fan category could recite the ebb and flow of the game by heart.

Please note there is a coauthor, Peter Richmond. It’s a tribute to Peter that the book sounds as if Frank is narrating the text.

This book, by Carroll, Palmer, and Thorn, can be regarded as Deep Stats 1.0, a serious attempt to get past raw numbers and generate a Theory of Everything. Well, football Everything.

For a statistically minded crew,  it’s an absolute must read, because they completely destroy the NFL’s passer rating formula. They had thought a lot about the formula, and their critique is penetrating and incisive. It can also be treated as a critique of any goof who stands up and claims that today’s passers are superior because their ratings are better than the players of  yesteryear, because, yes, Carroll et al have taken that whole argument and flayed it open on the written page as well.

That it is an older theory can be seen by  the units the authors choose to use. They reduce everything to yards. Yards? Any self respecting creator of a theory of Football Everything knows that the unit du jour is wins. This has been true ever since Bill James’s Win Shares, at least, and as stats like WARP (i.e. wins above replacement player) have become common. This need to express everything in terms of wins, or better yet, playoff wins, is part of what is fueling the current micro-revolution in football stats (see, for example, this recent Fifth Down Blog article by Brian Burke). We don’t need no steenkin’ points, no yards. How does taking the head off the secondary receiver and separating him from the ball translate into wins, padre? What things does my team need to do to win games, win playoff games, and win championships? That’s what any self respecting data geek wants to know.

Any other issues? I note that they have a rather unique description, in their “how the game evolved” pages, of Earle Neale’s Eagle defense and Steve Owens’s umbrella defense, differing from the descriptions given by Dr Z in Thinking Man’s or Jean Bramel in the Fifth Down blog. And no, I don’t think the Eagle was a 6-2 or that Steve Owen’s “Umbrella” was a 7-diamond. I think Dr Z and Jean are correct and this otherwise fine book wrong.

That said, they go over all aspects of the game, analyze them in terms of yards.. yes, they even convert scoring to .. yards, and then present their version of football Everything to the reader. It’s actually a fine first attempt, and were it not for the trends of the day, to think and eat and breathe in terms of wins, we might still be rating offenses by how many yards they “score”, and defenses by how many “yards” they prevent.

Once you have the concept of a drafting error in hand, and a fairly large one at that, you can ask questions  that have almost Murphy’s Law implications for hoary old theories such as BPA. Consider this scenario: you have three players in the middle rounds you are considering, whose “true career value” is about the same. We’ll assume drafting is an efficient market for now, so our estimation of the value of these picks is a normally distributed estimate whose mean is based off their true career value. Which one of these men do we draft? We draft the player whose value we have overestimated the most. Consequently, we draft the player most likely to underachieve our expectations.

Since in most drafts there are very few times a true BPA falls into the lap of teams (i.e. players where one is wildly superior to all other candidates), it would seem that BPA is a way of optimizing how heartbroken a team will be over the draft choices it actually picks. Though this approach would appear to gather the best athletes, in a draft with a large error, and multiple situations where you’re picking from nearly equivalent athletes, perhaps all BPA will get you is maximally suffering from buyer’s remourse.

Update: Brian Burke, on the blog Advanced NFL Stats, talks about exactly the same issue, except with free agents. This issue has a name, the “Winner’s Curse” and is a well documented problem with auction style transactions.

Super short summary: more accurate drafting is more effective drafting.

Summary for Statheads: Improving the draft accuracy of a single team improves the quality of draft choices picked across the entire draft. Simulations at draft error levels of 0.8 and 0.6 rounds respectively show that the effect is on the order of 7 and 5 picks. In other words, someone picking 12th at a noise level of 0.8, that picks twice as accurately as the norm, has picks equivalent to a team slotted into the 5th position. At a noise level of 0.6, their picks would be equivalent to someone picking in the 7th position. The implications of these findings based on PFR’s approximate value stat and draft round are discussed.

Recently, we posted data showing that the draft error of NFL teams can be estimated based on the kinds of reaches observed in the draft, and our estimated range of error was from 0.5 to 1.0 rounds of error per draft pick. Taking these ideas further, I wanted to examine what would happen to a team that picked twice as accurately as its peers. By accurately, the error of its scouts are half  that of all the other teams. What advantages would they gain?

Figure 1. Pick improvement as a function of round at draft error = 0.8 round

Figure 2. Pick improvement as a function of round at draft error = 0.6 round.

The charts above plot “effective draft position” (i.e. improvements in the value of draft picks, as ranked by the draft position, or slot, they should have been picked) as a function of round, for teams with improved drafting ability. This term can be converted, using Pro Football Reference’s formula for estimated approximate value per slot, into a difference in estimated approximate value for such a choice, and those plots are given below.

Figure 3. AV improvement as a function of round at draft error = 0.8 round.

Figure 4. AV improvement as a function of round at draft error = 0.6 round.

That these results are not unique to these particular error levels is also true, as we calculated estimated AV improvements for a team picking 10th and one picking 20th at error levels of 0.4 as well. 0.4 is so low, in my opinion, as to be unbelievable, but even  then, you can see advantages to the team that drafts well.

One last point. Notice the jump in advantage from the 3rd to 4th round using our model of drafting? That jump is a function of less intense drafting of those players whose first ranking is less than 8.0, and therefore a product of a specific feature in the model. The notion that good teams improve as scouting resources become more scarce is not.

Good teams should  be expected to do markedly better the fewer scouting resources are applied to each player. Where that happens in the real NFL is beyond the scope of this study, but that it almost certainly does happen seems evident. Teams that are expert at drafting will show their expertise more and more as the draft goes on. Or, said another way, anyone with a copy of USA Today or an ESPN Insider subscription can draft a first rounder. It takes really good teams to take best advantage of late round draft choices.

Super short summary: Head scratching moments in the NFL draft are useful clues to the average error in the draft.

Summary for statheads: A simple, efficient market model of drafting can account for commonly observed reaches in the first round if the average error per draft pick is between 0.8 and 1.0 round. The model yields asymmetric deviances from optimal drafting even when the error is itself described by a normal distribution. This model cannot account for busts or finds; players such as Terrell Davis, Tony Romo, or Tom Brady are not accounted for by this model. I conclude that drafting in the most general sense is not efficient even though substantial components of apparent drafting behavior can be analyzed by this model.

Introduction

 There are 4 typical ways to describe a draft choice. The first is by the number of the choice (Joe Smith is the 39th player chosen in the draft). Second, by a scale, usually topping at 10, and going down one point for every round of change. In such a situation  the ideal player is a 10.0, a very promising player a 9.0, a first of the third round a 8.0, and so forth. Ourlads uses a similar device to rank players as draft candidates.  The third way to rank a draft candidate is by the market value of the slot taken, and the best known representative of that kind of methodology is Jimmy Johnson’s trade value chart. The fourth way to rank a draft choice is by the historically derived value of players drafted at that position, and Pro Football Reference has done that here. Note: another interesting attempt at an AV value chart is here.

Jimmy Johnson's trade value chart

Every draft has a moment where you see a player drafted, and you wonder what drove a team to take this player. In the 2011 draft I can recall off the top of my head at least three four head scratching moments: the draft by San Francisco of Aldon Smith (Ourlads  8.99, but rising), by Tennessee of Jake Locker (Ourlads 9.15, considered by many to be late first, second round) , by Seattle of James Carpenter (Ourlads 7.05),  and the draft by New England of Ras-I-Dowling (Ourlads 7.82, but perhaps scheme related). All four left me wondering. Perhaps the same do to you, perhaps they don’t. But what I’m getting at is the number of these moments defines an error level by its recognizable tails, and using that, we can back track to an estimate of the actual error involved in selecting players.

If, say, the first round of 2011 was typical of all rounds of the NFL draft, and there was at least one truly puzzling reach in every round of the draft, and let’s say the puzzlers involved a reach of at least a round or more of value in the draft, then any noise model of the NFL draft has to be at least that noisy, else it is unrealistic. If, for the sake of argument we’ll assume the baseline draft model is efficient, then we add the assumptions that there are no systematic errors in drafting, and that drafting errors are normally distributed. So, if there is 1 error per round of 1.0 rounds or more, then there should be 7 in the whole draft, and 7000 in 1000 simulated drafts. We set out to build a simulator and test  these principles.

(more…)

You would think that when Dom Capers and Rex Ryan clash, and play in a 9-0 defensive slugfest, that the NFL network could find interesting film for both coaches. In this particular game, they largely showed highlights of the frustrations of the Jets offense. No matter, Dom Capers can make very subtle changes to very ordinary looking defenses. This shot for example: looks like a 3 man front on first glance. But is it that? For that matter, is  it even a psycho (1 man in 3 point stance) or just a very well disguised cloud?

This is typical of the fronts you can find throughout the video of this contest. This nickel front is very ordinary by Dom’s standards.

This front, I’m hard pressed to classify. Look at the defensive line splits; almost purely geared for an outside pass rush.

We’ll end with this one, a far more standard psycho front.

For those less familiar with the various kinds of modern fronts, I can suggest this article on Rob Ryan from this blog, and perhaps checking out the very excellent Blitzology sometime.

I’ll note that the Sports Reference blog Statheads now has this blog on their sidebar, something I’m quite  grateful for. Not that you don’t have to fight for readership as an amateur in football  blogging, because you do. But to be spoken of in the same breath as sites like Football Outsiders or Advanced Football Stats is, well,  heady stuff. So to Neil Paine, thank you.

What I’m going to say to readers that are largely team (e.g. Dallas, Atlanta, Green Bay, Pittsburgh, Chicago) fans, you’ll get the best bang for your buck by looking at the tag cloud on the right of the blog and clicking ones that interest you. If you’re one of the football coaches that have drifted here because of Coach Hoover’s recommendations, I’d suggest your best usage of this site would be to follow the  tags “46 defense” and “defensive fronts”. For those for whom algebra isn’t an issue, just read the  general flow of this board. I’m going to try and keep the look visual and fold most of the numerical results behind “more”  tags. The guy who wants to see Rob Ryan or Dom Capers get after the quarterback doesn’t need screen shots of program output, and the guy who does can click on the “more”.

The big splash of the draft, from my point of view, was the trade from 27th to 6th by the Falcons. They netted Julio Jones with the trade, giving up two firsts, one second, and 2 4th round choices in the process. This is because the Falcons felt they weren’t explosive enough, and so had to improve on an offense ranked in the top ten in the league. That their defense was mediocre and the salient feature of the last Super Bowl was that the #1 and #2 ranked defenses met, seemed to be bypassed in the quest for game changing explosiveness.  I suspect trying to become a 21st century Air Coryell certainly has fan and box office appeal, but is it wise over the longer term? I’m reminded of the nursery rhyme:

For want of a nail the shoe was lost.
For want of a shoe the horse was lost.
For want of a horse the rider was lost.
For want of a rider the battle was lost.
For want of a battle the kingdom was lost.
And all for the want of a horseshoe nail.

I’m not convinced there was that much more air based explosiveness to get out of the Falcons offense. Perhaps on the ground, where some rest for Michael Turner might save him from regression to the mean.

JMO, but the focus of the Falcons is classic Parcells style ball control, where yards per carry are far less important than time of possession (this, incidentally, is why Curtis Martin will always be underrated by YPC-heads – Parcells just never cared about his ball carriers YPC). In such an offense, the most important component of  the offense are first downs, not pretty stats. Given how light the Falcons defensive line tends to be, keeping them  off the field as much as possible has to be a serious design consideration for the whole team.

(more…)