Bears GM Candidates by the Numbers

Last week, I started out my article by bemoaning the complete lack of statistics to measure GM performance. This whine, however, was followed by an article almost completely lacking in any numbers of its own. To remedy this situation, I have spent the last couple of days converting the draft performances of seven different potential GMs (and one Mr. Jerry Angelo) into my very own GM measurement statistic: the Draft Value Rating System. Follow me below the fold to get the full breakdown of how the many potential GMs did in their drafts from 2001-2007, complete with charts!

There are more statistics in football than you could shake your copy of Football Outsiders Almanac at, so why come up with one more? Simply put, I wanted to create a system that would attempt to objectively measure how well a GM did in his drafts. The system I came up with is pretty straightforward, assuming you find any math straightforward. If you don't, feel free to skip the explanation and go straight to the chart.

Draft Value is designed to be a measurement of, well, the value gained out of a draft pick. I was operating under some basic assumptions:

• Top-round draft picks should go on to become long-term starters in the NFL
• MIddle-round draft picks should be able to make an NFL roster and stick around for a couple of years
• Bottom-round draft picks are valuable if they are able to make an NFL roster, and even more valuable if they become starters.

With these three assumptions in mind, I set about finding a good way to compare player longevity, starter-level quality, and draft position into a basic measure of draft quality. Draft Value (DV) operates on the premise that a starter is twice as valuable as a backup, and a Pro Bowler is three times as valuable as a backup. As such, every player a GM drafts earns one point of DV for every season he was on an NFL roster, another point for every season he was a primary starter in his position, and an extra point on top of that for every Pro Bowl season he had. From here, I applied a curve based on the round in which the player was picked. Since the average career length of a first round pick is nine years, I subtracted eight points off their total score: this curve assesses the value of a first-round pick based on their seasons as a starter and a Pro Bowler, as a first-round pick who is only holding onto a roster spot is not much of a pick at all. This curve gets progressively smaller in the later rounds, until rounds six and seven, where there is no curve at all: any late-round pick who can make a roster is a positive pick according to DV. To give an example of how DV is calculated:

Reggie Wayne has been in the league 11 seasons, has been a primary starter for all of those 11 seasons, and has made five Pro Bowl appearances. 11 seasons + 11 years as a starter + 5 Pro Bowls = 26. Then the curve for him being a first-round pick is applied, and 26 - 8 = 19, his final (and very high) Draft Value score.

The curve is mainly there to give a greater value to later-round picks who end up being strong starting players: Lance Briggs' 22 DV is even higher than Reggie Wayne's, as a third-round pick with multiple Pro Bowls to his name is a "better" pick than a first-rounder with a similarly excellent career: the first-rounder is expected to be a great player by DV. The other reason a curve is applied to higher round picks is to allow for negative scores when top round picks end up going on to have bad careers. For example, Bears' 2003 first-round pick Michael Haynes tied for the worst Draft Value I measured, -5, by only playing three seasons in the NFL and having zero seasons as a primary starter: 3 Season + 0 as a starter + 0 Pro Bowls - 8 for his draft position = -5, an appropriately bad score for squandering a first-round pick on him.

A post containing all the data I looked at would be absurdly long even by my standards, so I put the full data set into a Google spreadsheet that you can view here. I conveniently highlighted offensive linemen, DBs, and WRs in shades of Bears' blue and orange so you can compare how each of the GMs did filling these positions of need for the Bears. I also put a star (*) next to the name of each player who is still active in the league, as these players' scores are "in progress" and can continue to improve as they continue to play. The potential for DV improvement also explains why the averages presented below seem to drop off year by year: more recent drafts have yet to reach their "true" scores.

So, without further ado, here are the average DVs for our "old" GM candidates:

	Jerry Angelo	Bill Polian	Tim Ruskell	Ted Sundquist
'01	3.17	8.86	6*	7*
'02	5.44	6.0	1.5*	4.13*
'03	5.91	5.5	4.17*	0.33
'04	4	5.4	6.57*	1.56
'05	5.5	2.8	2.44	3.67
'06	3	6.29	3	6.14
'07	1.44	2.11	4.14	2.5

(A * indicates the candidate in question was not the GM at the time, but was still given a score for the team they were working for.)

As you can see by looking at that data yourself, these top-line averages tell only part of the story. For example, while Bill Polian's below average 2005 draft is largely the result of a big miss in the first round, Tim Ruskell's equally mediocre 2005 draft average is the result of him having a couple of extra lower round picks that drag down his higher-value early picks. With this in mind, what conclusions can we draw from these averages and the numbers behind them?

• It's clear that Bill Polian was the best drafter of the bunch as a GM: he was consistent in finding starting-caliber players in the first round and also earned big DV points by finding some diamonds in the rough of the late rounds. While I am still worried about the whole dynamic of him and his son and their ugly departure from Indy, the man clearly has a good eye for the draft. While his more recent drafts don't look all that good at first glance, he does have a clear history of first-round success that the Bears have lacked in the Angelo era.
  
• Although Tim Ruskell was part of the excellent Rich McKay office from 2001-2004 before he went to Seattle, he managed to never draft a player with a double-digit DV during the years I measured. While that is partially the result of more recent drafts having generally lower scores - players who are still in the league continue to earn DV points - it is also the result of some plain ol' bad drafting. To his credit, though, he has made some very good O-line picks over his career and could help the Bears find more Carimi-like players if he were to take over.
  
• This system gives surprisingly decent marks to Jerry Angelo. His median DV score is pretty low based on my eyeball estimate, but his his averages look good on the basis of some great mid- and lower-round picks: Mike Gandy, Alex Brown, Lance Briggs, Bernard Berrian, Kyle Orton, and Chris Harris all became long-time starters after being picked by Jerry in the third round or later. While he often whiffed in the first round, the weighting applied to starters found in the later rounds gives Angelo higher marks than perhaps he deserves. Also, by not looking at more recent seasons, Angelo is given a pass on his most suspect drafts of the last couple of years.
  

• Ted Sundquist, who I was promoting as a strong candidate the last time around, doesn't look so good after the numbers were crunched. He missed on many of his early-round picks and didn't make up much ground in the later rounds like Angelo did. That said, he had one of the best draft classes I looked at in 2006, the year he drafted Pro Bowlers Jay Cutler, Brandon Marshall, and Elvis Dumervil and three other still-active players with his seven picks. He also had a pretty good success rate with his offensive line picks, many of which ended up scoring double-digit DVs by sticking to a roster despite being lower-round picks. Plus, I still like Sundquist as a potential GM if only because he is the candidate most likely to try and bring Brandon Marshall to Chicago.

Now, let's take a look at the new guys:

	Phil Emery	Jason Licht	Jimmy Raye	Marc Ross
'01	3.17	5.40	8.00	6.50
'02	5.44	6.17	5.13	8.25
'03	5.91	8.43	5.50	3.83
'04	4.00	9.71	9.22	2.70
'05	5.00	2.00	5.00	3.33
'06	2.33	4.00	4.25	2.78
'07	3.00	3.20	3.50	3.86

Before I jump into my analysis of this latest batch of interviewees, a word of caution. Unlike the actual game of football, where anyone with a TV can see how each player contributes to the team's performance, it's impossible to know exactly how each member of a scouting team contributed to the overall success (or failure) of a draft. Think of it as the GM's version of the "Bill Belichick effect": just because the overall group did well doesn't necessarily mean that a promoted underling will bring that success with them to their next job. With that big asterisk in place, here's my breakdown of this set of numbers:

• Phil Emery? In a word, meh. He doesn't benefit from having his scores tied to Jerry Angelo's for the first four seasons I looked at, although our '03 draft class wasn't half bad after the first round. His draft classes in Atlanta were equally as mediocre as one would expect out of a Jerry Angelo disciple: after finding two quality starters in the first two rounds of the 2005 draft, the Falcons had two straight years of misses or near-misses in the first two rounds. His luck with finding offensive linesmen is summarized by the fact that his "best" O-line pick with the Falcons according to DV was False Start Frank. The numbers agree with my smell test that Emery would likely have a tough time outdoing his former boss, both getting it right with his first couple of picks and in improving our offensive line.
  
• Jason Licht is a possibility I am pretty favorable towards. While history has shown it is wise to be skeptical of former Patriots - ask Josh McDaniels for details - his success in New England traveled with him to Arizona. Admittedly, he was the beneficiary of the Cardinals' bad play on the field: it's a bit easier to get top-quality players when you have top ten picks. Still, other than the Matt Leinart experience, his top picks turned out to be decent enough. I also like that when his team needed a new starting center, he was willing to invest two mid-round picks on the position: clearly he understands just how important getting a functional offensive line is to having a winning football team. His skill at finding wide receivers is equally as good: Larry Fitzgerald was an easy pick in the top ten, but he was also able to find two other quality starters (Anquan Boldin and Steve Breaston) in the later rounds.
  
• Jimmy Raye has the best draft success of the bunch, and even beats out Polian for the title of best drafter according to DV. His drafting of offensive linesmen was shaky in the earlier years I looked at, but his drafting in this area steadily improved over the years. Simply put, the guy can draft. I do worry about the Chargers management staff outside of their drafts - they seem to have found themselves on the wrong side of some big-time trades over the last ten years - but you have to admire a team that can draft its way into the luxury of trading away starting-caliber franchise players when they feel like it.
• Marc Ross looks to be the victim of the Belicheck effect. In Philadelphia, he had some good looking drafts, but once he went up the road to Buffalo, his DV took a nose dive. Part of this may be because the Bills are simply a bad team, but the numbers bear out that most of his draft picks ended up being not all that good. His drafting was especially bad in regards to offensive linesmen in the years I looked at. That said, his more recent drafts with the Giants have worked out much better so far, so perhaps the general badness of the Bills is more to blame than some major flaw in Ross' eye for talent. Not my first choice, or even my second.

While my measurement system is beautiful in its simplicity, it doesn't do a good job of taking into account some of the many other things as GM is responsible for. I'm sure there is a way I could include the DV of picks traded for players, but that would require some mathematical gymnastics beyond what I wanted to do here. Also, by giving double points for seasons spent as a starter, GMs are penalized for already having a strong roster and being able to plan ahead: the "Aaron Rodgers effect," if you will. For example, Bill Polian gets a low score for taking Anthony Gonzalez in the first round of the 2007 draft: while Gonzalez has stayed in the league and has been a decent enough WR, he has never been a consistent starter because he plays behind other Polian draft picks who are far better. Also, the use of seasons as a starter and Pro Bowl seasons as the only measurements of a player's value is a somewhat crude method to account for a player's overall value to a team. A more fine-grained approach would use an advanced statistic like pro-football-reference.com's AV to account for the quality of play out of a draftee. Speaking of PFR, I have to give them infinite thanks for their Draft Finder, which made this whole project much more straightforward by providing me with all the data in one convenient location.

Please, take a look at my lovely Bears-colored draft charts and let me know who you think would be the best GM of the bunch would be down in the comments. It looks like we're in for the long haul on this search, but there are still some good-looking candidates in the prospect pool. Let's just hope the Bears can reel one in.

UPDATE: I just finished applying a new curve to the data to recenter the numbers on zero. The updated spreadsheet can be viewed here, and I will update the charts on this page when I get home from work at about 6:30 tonight.