Bears GM Candidates: the Draft Value System Revisited

Last week, I set out to create an objective way to measure the draft success of the Bears' GM candidates. Thanks to the helpful feedback from WCGers, I have made some refinements to my system: the numbers now make a lot more sense than they did the last time around. For your consideration, here is an updated version of the WCG-exclusive Draft Value system, once again complete with charts!

Once again, if you don't want to delve into the underside of the Draft Value system, feel free to skip ahead to the new charts and see the top-line numbers.

For the trial run, the Draft Value system simply assigned every player a value based on three simple stats: years in the league + years as a starter + Pro Bowl seasons = Draft Value. There was a curve applied to each player, but this curve was based only on my estimates of how long a draft pick should stay in the league. Since the whole point of creating DV was to have an objective measurement of draft success, my "take a good guess and hope the numbers look good" system of curving the numbers was a major flaw in the system. To correct for this, I cross-tabulated the data from every draft pick I looked at, finding the median values for each of the three metrics in question. (These calculations can be seen here, if you're really curious.) I used the median values for each round (or set of two rounds, in the case of fourth & fifth/sixth & seventh rounders) to make sure that the best players from each round - the ones who were way above the average - wouldn't skew the numbers and give an "average" draft pick a negative value. The net effect of this new curve, though, is a much clearly system of measurment: now, negative draft values mean that a pick was a below-average pick, and positive values mean the pick was above-average. The new cross-tabs for my overall averages is here.

It was interesting to note that, even for first-round picks, the median amount of Pro Bowl seasons was still zero: even among players considering the best in their draft year, it's still tough to win that popularity/skill contest. Since Pro Bowl appearances are such a rare thing, one commenter - Sweetness Lives On - suggested that Pro Bowl berths shouldn't simply be added into a player's DV score, but instead should give the player's score a multiplier effect. Since simply multiplying the amount of Pro Bowl appearances by a player's seasons played + seasons started would give absurdly high numbers to perennial Pro Bowlers, Sweetness Lives On also suggested that the multiplier should be the square root of Pro Bowl appearances. For example, if Johnny Quarterback played in the league for seven seasons, started for five, and made two Pro Bowls, his uncurved score would now be (7+5)(1.4)=16.92, not (7+5+2)=12. For players with more than four Pro Bowls to their name, the multiplier will remain at 2: otherwise, the numbers would get really big really fast for the select few in this club. Also, if a player only makes one Pro Bowl, he will now get two points tacked on to his score, as a multiplier of 1 wouldn't change his DV but adding only one undervalues having a Pro Bowl appearance. With this addition in place, Draft Value now assumes that Pro Bowl players are up to twice as valuable as non-Pro-Bowlers.

It is very important to keep in mind as you look at the numbers that this is not a system of measuring player skill or performance. It is called "Draft Value" for a reason: the only thing I am interested in finding out is which GMs/teams are the best at drafting players who are able to stick around in the league, become starters, and/or make it to a Pro Bowl. This disconnect between what DV is designed to measure and our own common sense become apparent in what I will dub the "Frank Omiyale Effect."* I have corrected for the somewhat absurd outcome that Frank Omiyale had as many DV points as Roddy White, a result that suggests a flaw in the system. The first, the application of a Pro Bowl multiplier, was mentioned above. Between this change and the steeper curve on lower-round picks, Omiyale now only has a 3 DV while Roddy White has a 6.8, the highest of any first-rounder in 2005. Problem solved, and thanks to tomas21 for pointing out this weirdness.

(*Not to be confused with the other Frank Omiyale Effect, the one in which any team unfortunate enough to play him finds its quarterback face-down on the turf and is called for an endless number of false starts.)

As before, these numbers have to be read with a degree of caution. The biggest problem that I am unsure of how to correct for is the "Aaron Rodgers effect," as DV system does not account for a team whose draft needs are long-term. Ultimately, though, this effect would be ironed out by the end of a player's career: while an Aaron Rodgers might have sat on a bench for a couple of seasons, he will still earn lots of DV points for the seasons he is a starter or a Pro Bowler. This "time cures all things" problem is also why DV values from more recent years are lower. Simply put, I'm not in the prediction business. I don't want to account for the potential future DV of still-active players because it would require pulling a number out of thin air. I could apply a corrective based on the assumption that still-active players would stay in the league for an average length of time (i.e. just add on years to get the player's number to equal the average career length for that draft round), but since many of the still-active players are beyond this average already, this wouldn't really fix the problem either. Long story short, more recent numbers are still "in-progress scores" that can't really be accurately compared with earlier years.

With all that out of the way, let's turn to the actual numbers. Once again, remember that in this new and improved system, positive averages represent strong drafting and negative averages represent mediocre or poor drafting. The full cross-tabs can be viewed here, still color-coded to help you spot a candidate's WR, DB, and O-line picks and with a * by players who are still active in the league. The top-line averages for our candidates:

What do these numbers mean, exactly? The easiest way to think about DV, especially in terms of its overall average, is how many seasons or starting seasons of "extra" value the GM got out of his draft class. This is even more true under the new curve, which moved the overall numbers clloser to zero. I won't re-hash my points from the last article, but my observations based on the averages from the old system hold up pretty well against these new averages. To put these new numbers into order, though, my ranking of our options would be:

1. Jason Licht
2. Jimmy Raye
3. Bill Polian
4. Marc Ross
5. Phil Emery
   
6. Jerry Angelo
7. Tim Ruskell
8. Ted Sundquist

The most alarming conclusion of this analysis is that "house candidate" Tim Ruskell would actually represent a down-grade from our former mediocrity. The good news is that all the serious candidates other than Ruskell have a history of being better than Jerry Angelo. And that's real good news given my general pessimism about our GM search after the best two candidates were taken off the board by Oakland and Baltimore.

Since I want to squeeze some more information out of all these numbers, let's look at another chart: the median values for each draft year. While taking the average has the effect of providing extra weight to really good picks - Jerry Angelo gets a really good 2003 draft score because he got a steal on Lance Briggs in the third round - the median gives a better sense of how good the "middle-of-the-road" player taken by each GM/team was.

As you can see, the cream really rises to the top when we consider median. Essentially, these results show that only Jason Licht and Jimmy Raye have been able to find players that are consistently above average. While the other candidates may have been able to strike gold once or twice in each draft and get their averages up, this accounting shows that their picks quite literally tended towards the median. Our top two options, though, consistently added above-average players to their rosters. I just hope that it is in fact Licht and Raye who are in some way responsible for these sterling numbers, as I don't want the Bears to become the latest example of why you shouldn't promote the underlings who haven't done anything of note away from the nest. Given that Licht has been successful at two different teams, I would give him a slight edge in this department, but I think Raye would also prove to be a good GM choice for the Bears.

I would still like to do some more analysis of these numbers. What I am most interested in is finding out which has a bigger correlation to Draft Value: a team's record from the season before, or it's record from the season(s) after? If it turned out to be the team's past record, it would suggest that having higher draft picks leads to better overall drafting. If future records had a stronger correlation, however, it would be a nice feather in my statistical cap: that would show that these numbers have the potential to be statistically valid, i.e. a team's Draft Value for a season would predict how well they did in the years after. There are also plenty of improvements that could be made, the biggest being applying an extra weighting for position. Since career lengths differ wildly - running backs' careers are short, kickers careers are long, etc. - it would only make sense to measure each player against draft round and positional averages. This double-weighting would make DV an even more robust system, but I would need a more robust data set to get positional averages: while I have around 400 players in my data set so far, some positions would only be represented by a handful players Anyone feel like typing a hundred or so drafts into a spreadsheet and sorting them by position and round for me?

I hope you have enjoyed following me on my quest through Mathmagic Land. The numbers show that we are definitely on the right track with our search and things are looking up even more than I thought. Let's send anti-Ruskell thoughts in Ted Phillips' general direction and hope for the best. For now, though, please continue to offer your feedback on my system and discuss our GM prospects down in the comments.