FanPost

Draft charts and potential draft grading mechanisms for feedback

I've been playing with some data from Pro Football Reference and created some charts that I think are interesting, so I'm sharing them. I'm not spending a bunch of time on commentary, but wanted to explain the charts and why I created them. Forgive any awkward language, I didn't spend a lot of time editing.

Below is a fairly famous chart showing the Jimmy Johnson Chart vs the Chase Stuart Chart where the two systems have been normalized to a common scale. Across the bottom are the picks. The left axis just puts the 2 systems on a common scale. The red and green shows you the % difference between the two systems on the right axis. Red is an "overpay" and green "a bargain".

chart1.0.jpg

The conclusion drawn is usually that you should trade back from the first. The NFL overvalues the picks before pick 44 (Round 2-12) and undervalues those after. However, NFL teams continue to pay more for those early picks. To show this, I added the Rich Hill system as well (blue line). People forget that the Jimmy Johnson chart isn't random. It was a chart created by a statistician by the Cowboys based on actual trade values. Rich Hill continues that same method to create a scale showing what the GMs actually pay.

Why do NFL GMs continue to overpay? Have they never seen this chart? I think something else is going on and download some data from 2000-2020 to see if I could explain, "Why are NFL GMs such idiots?"

Chase Stuart created his chart by using Pro Football Reference's AV (approximate value). He took the AV for each player during the 1st five years of their career and fitted a line to that dataset. Chase also postulated that a UDFA "should" generate 2 AV and subtracted this from each value to create a table of "incremental value" for each draft pick.

Two small things always bothered me about this. First, I hated that he used 5 instead of 4 years. Sure, there are option years and tag years that can into play, but those are free agency decisions, not draft decisions. I also suspected that the 5th year might skew the results a bit because almost by definition he's adding value for a lot of players on their 2nd contract. Second, I wondered why he assumed a UDFA was worth 2AV a year.

Below I added a blue line based on the 4 year Average AV for a player drafted from 2000-2020. Because I'm using the average per year I can even include the most recent draft class. I then fitted a logarithmic line to that average (thick blue). I shifted the Stuart Line up by 2 AV based on his assumption and I put the Jimmy Johnson line at a base of the actual UDFA average (which is 0.98). This shifts the inflection point quite a bit. It now says NFL GMs are overpaying In Rounds 1 and 2... it isn't until Round 3-11 that they begin to underpay. So far I'm not doing a very good job of explaining the idiocy.

chart2.0.png

My biggest complaint about the Chase method is that it is based on the average. When GMs are making trades they aren't trying to get a guy who will be average at that draft slot. They are trying to get a specific guy they think will be good.

So, I plotted the max AV for each draft slot (red dots) and fitted a line to that (red). As you can see, I think this explains a lot of the variance in the early rounds. The GMs are paying to take a guy they think might be the best at that slot. The inflection point above the "max AV" and the Jimmy Johnson chart is now pick 10, not the 2nd or 3rd round.

chart3.0.png

But, part of me realized, isn't it still kind of stupid for a GM to pay the price for a guy 'as-if' he's the best drafted in that round? What does it look like if I plot the average of the Next 5 highest AV by draft slot? The thin green line shows the results and the thick green line is again a fit against that dataset.

chart4.0.png

It shows the GMs are again idiots in the first, but once again not as severe idiots. I think I know the reason for the additional value they are paying in the first. I strongly suspect it relates to the AV generated after four years, but I'm still working on that math.

However, once I had the 3 lines above, I realized I had a kind of nice way to grade draft picks. Those above the red could be an A. Those between Red and Green a B... etc... and I wondered... where do the Bear draft picks fall on these charts?

chart5.0.png

The orange circles are Pace's picks. Blue = Emery. Grey = Angelo. The left axis is the average AV for the player during their 1st 4 years. The horizontal axis is their draft slot.

I kind of like this grading method. It grades harder for earlier picks and can distinguish between a Roquon (B) and an Eddie Jackson (A) because of the draft round. Because it uses the average annual AV draft classes can also be graded each year. The final grade won't be known until the 4 years are over, but after 2 or 3 the average within 4 won't change that much.

I can provide a data table of how each pick grades. I can also apply this mechanism to any and all teams. I do think there should also be an "F" grade for those that are below a certain annual AV. I'm leaning towards 1 per year. If a UDFA will average one per year it seems like the bare minimum threshold that should be required (and also makes the Kevin White pick an F).

There are some things I also don't like about this method. J'Marcus Webb might have been an "A pick", but he isn't an "A Player". So, I also think there is a grading mechanism where the draft pick value doesn't really come into play. What is changing above isn't really the top of each rounds, it is the frequency of different performance bands.

Below shows the frequency of total AV over the 4 year period (a 40 below would equate to a 10 above).

chart6.0.png

I think there might be a better draft trade evaluation model that takes these probabilities into account. Rather than judging the trade vs historical trades (Jimmy Johnson/Rich Hill) or the average player performance (Chase Stuart), how do trades stack up in terms of the probability of identifying someone who is Elite (40) a Star (30), a Quality Contributor (20), JAG (10), a contributor (4) or a bust (0's). If someone wants to see these probabilities by round, just ask.

BUT, before I go too much further I wanted to get some feedback. I also tried to minimize my commentary about what I think I learned from some of these charts... for now I wanted to share them to see what you can glean and your thoughts about how to grade drafts (plus, I'm curious about the reaction to some of the names on the Bears chart above).

This Fanpost was written by a Windy City Gridiron member and does not necessarily reflect the ideas or opinions of its staff or community.