The Draft Research Project: Performance-based Tiers

Background

So, after a looking at a traditional round-by-round analysis of the draft, considering the variation between draft classes, and discussing the role that position plays in draft outcomes, there are some implications for how draft picks actually sort out on a tier-by-tier basis. Simply put, it makes no sense to expect the same outcome from the fifth pick in the draft as from the twenty-fifth, but the variations between the 200^th pick and the 250^th pick are much less noticeable.

In developing the tiers for this installment, bands of picks 5-, 10-, and 15-strong were originally tested. Ultimately, however, bands of 11 picks at a time proved to be the most consistent in finding tiers of selections that were more like one another than they were the next group. These became the basis of the initial tiers, but each tier was open to restructuring based on whether or not the players in each band showed similar outcomes.

Note that already a casual view would observe that these tiers do not fade in performance as consistently and evenly as a traditional round-by-round reporting might suggest. The variations in the tiers remains when looking at these bands of players in terms of their honors and accomplishments, as well.

This next table uses the categories introduced prior in the series. A player was deemed a starter if he started in at least 40 games across the first 80, and in order to be an impact player the draft pick in question needed to have at least one Pro Bowl distinction while playing in 40 games. A player needed to earn multiple Pro Bowls or first-team All-Pros to be deemed a star. Failures track players who failed to play at least four years in the league, and the 5^th-year column tracks whether or not a player lasted at least five years in the league on any team, not just the team that drafted him.

The Tiers

Tier 1: This tier averages more than one Pro Bowl selection per player, and it is where the stars of the NFL are found. The majority of players drafted this highly will earn a Pro Bowl, and a team with multiple selections in this range essentially needs to find a Pro Bowler with at least one of these picks. Truly missing on a pick in the Top 11 is all but unheard of. Instead, what is more likely to set a franchise back is for a player to underperform.

In 2016, the Chicago Bears drafted Leonard Floyd at #9 overall, trading up from #11 to get him. Despite managing to play in 70 games at a fair level (at the median for players drafted in the top tier), his performance his first five years represented below-average contributions otherwise. Floyd is not a miss by the standards of the draft as a whole, but his results turned what should have been a top-tier selection into simply an acceptable performance.

Tier 2: The second tier produces a very impressive group of players, unless it is compared to the highest level. The same number of total players in this band end up earning half as many Pro Bowls and a third as many first-team All-Pro considerations. While most teams can probably afford to find a solid starter in this tier instead of a true difference-maker, these players must at least provide stability at a position.

Tier 3: The end of the first round sees two important shifts. The first is that the average number of starts declines quickly, and this is not simply the result of one or two dramatic failures—the median number of starts falls from 54 to 45 as well. The second is that despite that trend, the total number of actual first-team All-Pros goes up. Before diving into this specific result, it’s worth considering the next tier, as well, however.

Tier 4: By contrast with the third tier, the fourth tier offers more starts and more games, but it has fewer impact players and stars. Players make it into their fifth year, but they do so unremarkably compared to their first-round brethren. Taken together, there are two obvious (and likely equally applicable) explanations for the way Tier 3 and Tier 4 interact. Players drafted at the end of the first round are more likely to go to teams that are already successful. These players therefore do not face the same pressures to perform as players drafted more highly, and so they might be developmental prospects who take longer to get on the field, and they might even need to earn their way onto the field against an established starter who is already capable. However, these players are more likely to play well once they get on the field either because they were given that time or surrounded by more talent.

Unfortunately, players drafted at the top of the second round face more pressure to play, and their organizations put them on the field even if they are not as promising. Teams drafting this highly probably cannot afford to be as picky about whether or not they play their draft picks, and they have to draft for need instead of potential. Players drafted here also last longer perhaps because their teams have so many needs that their managers and coaches cannot afford another hole.

Tier 5: Finally, things begin to settle down and even out. After more than one round of draft picks wherein the results vary with each step down, what would normally be considered the bulk of the second round all acts largely the same.

Tier 6: In general, Tier 6 represents the role-players found in the draft. These are players who stick around and who in fact develop into starters, but who are otherwise not as notable as their peers drafted earlier on. However, for this project, Tier 6 also provides an object lesson on small data sets.

Simply put, the more granular the data, the greater the likelihood of a quirk appearing that can be interpreted as significant even if it is nothing more than an artifact. Is it possible to find two rocks that happen align with two stars? Certainly, as there are many rocks, many stars, and the alignment of those stars relative to the rocks changes over time.

In this case, a seemingly significant alignment happened at pick #69. Four of these seven players made the Pro Bowl and three were first-team All-Pros. They played in an average of 64 games. This one little eddy of players was almost like finding a hidden pocket of Top 11 players. Except there is no causal factor to explain it. Yes, they are all compensatory picks. However, some went to strong teams and others went to weak teams. Some were traded and some went to the teams that earned the picks. Some were players who fell and some were just diamonds in the rough. In short, not every data means something, especially when there are such small numbers involved. This is a nice cautionary note in drawing too many conclusions from too little information and overreading the numbers.

Tier 7: Essentially the start of the fourth round and most of the fifth round, this tier represents the tipping point. Players are more than twice as likely to leave the NFL before their fourth year than they are to start in half of the available games, but the median player still does appear in at least forty games.

Tier 8: A player drafted this late is far more likely to be a depth piece or a placeholder on a roster than a starter, let alone a player who changes the shape of a season for a franchise. Individual plays might stand out to fans, but these are the players who make compelling stories for whether or not they even make the roster.

Tier 9: It’s not an exaggeration to say that Tier 9 draft picks have minimal impact on rosters. While there are exceptions, these selections largely serve as “exchange tokens” at the end of the draft, with different GMs using them as a chance to level out a draft trade of higher-value selections or else as one more chance to prospect for a special need at a given position.

Conclusions

Even the staunchest advocate of trading back for value in the draft should notice that when there’s a less than 4% chance of finding a starter after #199, and when the median player selected at the end of the draft only appears in 12 total games, it is arguable how much good it does to stockpile 6^th- and 7^th-round selections. After all, each of these players needs playing time and a chance to develop as part of a team. Thus, there’s an upper limit to how useful it is to take three wide receivers at the end of the draft when each one has only a 4-9% chance of developing into a starter.

By contrast, the second round is perhaps slightly undervalued by most draft charts, given how much of a falloff in performance there is starting in the third round. After the first 100 selections, however, the ability to find a player who will provide more than depth is substantially reduced.

A savvy GM would be willing to give up later picks in order to maximize the chances of success earlier on, and trading an established player of almost any kind for less than a 5^th-round selection is unlikely to result in anything like an equivalent exchange of value.

Ultimately, unless they get disproportionately lucky, teams must hit on their opportunities in the first two rounds in order to improve through the draft. Even simply settling for “adequate” with high-end picks can undermine a franchise’s chances to compete if it happens too consistently. Chicago, for example, had four selections in the top tier from 2015-2018 and thus far has not come away with a single “star” player.