Fact Check Update: Windy City Drafts

Almost two years ago, I wrote a piece checking on whether or not Ryan Pace was actually delivering in the draft. It was not a comprehensive piece by any stretch of the imagination, but it did start to frame the question of whether or not some of the Bears’ woes at the time (which have continued) are linked to how well the GM does at drafting players. Since then, as part of another project, I have done a much deeper dive into understanding what is and is not reasonable to expect when it comes to drafted players.

Background

Although a longer piece will dive into more detail and will not be Bears-centric, for now it’s worth framing the data. From the 2012-2017, there were 6 draft classes, and as of this summer each of those classes had at least 4 years to prove itself. All of these players were drafted during the modern era of the rookie salary scale, which has changed how teams are motivated to play and retain their players. The group of players (1,525) is large enough to feel a little robust. Broadly speaking, the results of players drafted in the top ten stand out from the rest of the first round, the players drafted in the “top half” of the second round score better than the players drafted in the bottom half, and after that while there are differences within rounds and breaking points that don’t line up perfectly with the rounds themselves, using the round limits as a shortcut for tiers isn’t wrong, it’s just not as granular as it could be.

Overall, the results of the draft from 2012-2017 tell us that 77% have careers of 3+ years, which is about the same number as players who appear in at least 15 games (in raw numbers, it’s 1181 of the former, 1177 of the latter), However, only 36% of drafted players are starters for at least two years. Only 11% of drafted players make the Pro Bowl. At least since the rookie wage scale was implemented, drafted players typically play. More than three-fifths (63%) of all players drafted appear in at least 30 games.

This is not a hundred percent predictive, obviously, but it’s interesting to me that 55% of Top 10 picks in this group have made the Pro Bowl at least once and that 90% have been starters for at least two years, while for players taken in the third round those same rates drop to 12% and 43%.

Note that this data is descriptive. What I mean by that is it does not offer answers, it simply observes what is happening. Is the reason that all the overwhelming majority of Top 10 picks were starters for two years that they were good, or is it that the teams who drafted them kept playing them no matter what to get something out of their investment? The charts don’t say.

Despite the disdain of some fans, “Pro Powls” are in some ways the most neutral, because being elected to the Pro Bowl is a decision outside of the control of the drafting team. Of course, there’s some problems with distinguishing between primary selections and alternates, too. All of those caveats aside, this data pool can provide a reference point for fans independent of their own guesswork and personal observation.

TL;DR: Three-quarters of drafted players “make it,” and a third of them end up as starters for at least two years. About one in ten draft picks make it to the Pro Bowl, but nearly two-thirds of draftees play in at least 30 games.

Belief #1: Ryan Pace is Bad/Good at Drafting Itself

From 2015-2019, Ryan Pace has made 32 selections, and by now each of those players has had at least three seasons to register results. The only questionable draft is 2019, which included 5 players. When any of those players were involved, the players in question were given “credit” for finishing this season as positively as possible. All this really means as it turns out is that David Montgomery is registered as having another season as a starter.

Based on the selections he has actually made (not picks given up, just how those picks on net fared against average), Pace is remarkably average. He has made four picks in the Top 10, three picks in the top half of Round 2, two picks in the bottom half of Round 2, three Round 3 selections, eight Round 4 selections, and so on. Taken all together, if Pace just got “average” results for the picks that he actually made from 2015-2019, here is what his scorecard should look like:

He should have 25.3 three players whose careers lasted 3+ years and 12.6 players who spent at least two years as a starter. He should have found 21 players who played in at least 30 games and 25.3 players who played in at least 15 games. He also should have had 4.4 players who appeared in at least one Pro Bowl.

He actually does more or less as expected. He has 26 players whose careers have lasted 3+ years and 14 players who spent at least two years as a starter. However, he only found 20 players who played in at least 30 games, and only 25 players have played in at least 15 games. As a slightly positive note, he has found 5 players who have earned a Pro Bowl distinction.

He is within a rounding error on career players and a little ahead on starters, but he is a little behind on total players making it into games. He is ahead by about half of a Pro Bowler. Still, he is getting what would basically be considered “average” results, and while the extra starter is nice, it would be nice to have players in games. It is worth noting that he has a slight edge on players that people outside of the Bears are impressed by (i.e. the Pro Bowl), but within the vagaries of the draft this is very consistent.

Verdict: Inconclusive. Pace is pretty middle-of-the-road, and enthusiasts and detractors alike can hang on to at least one outcome to make their cases.

Belief #2: Ryan Pace is Bad Early and Great Later On

One belief that persists is that Ryan Pace is exceptionally good in late rounds. This perception is likely influenced by the fact that he makes many late-round picks (8 selections in Round 4 in 5 years, for example). Are Eddie Jackson and Bilal Nichols finds? Sure. What about Jordan Morgan and Riley Ridley? There’s some survivor bias in how fans look at things. He’s actually just a bit better than average in late-round starters, but there’s a major asterisk there. Cutting out the first three rounds, his numbers look like this:

He should have 14 players whose careers lasted at least three years and he found exactly 14 players. He should have 4.4 players who earned at least 2 years as a starter and he has actually found 6 players (that’s notable, but deceptive). He should have 10.6 players who played in at least 30 games and he actually has 10. He should have 13.7 players who played in at least 15 games and he has 13. Finally, he should have had 3.1 players who appeared in at least one Pro Bowl and he actually has 3.

Verdict: Plausible. Besides starters, these are again very average results. Remember that he is ahead by 1 starter overall, and his +1.6 players versus expectations in the later rounds is where he picks up that extra player. In every other category, while he’s a little behind, it’s only by the thinnest of margins. While he does not have any special talent at finding “gems” (as acknowledged by the rest of the league in the Pro Bowl), he does make up the ground he loses early in the draft later on.

Belief #3: The Bears’ Roster is “Behind” Because of Pace’s Trades

So, this survey has only looked at the draft picks Ryan Pace actually made. He has at least an extra starter, but he has fewer role-players on net despite that. Otherwise, he’s almost perfectly average. Note that none of this has taken into consideration the draft picks he gave up in order to be able to make his “average” selections (or, for that matter, the picks he gave up to get Khalil Mack). This is just the picks he made, where he made them, compared to the average results across 32 teams and 6 years.

Remember, however, that because there are basically 254 picks every year (plus or minus a couple), the average NFL team should really get to make 8 picks a year (7.94). That means that Ryan Pace should have had 39.7 selections in the draft, and he actually only made 32. Through 2019, he is behind by 7.7 picks, or almost exactly one full draft.

What does this translate into in terms of outcome?

A perfectly average team’s actual “expected return” on a single year’s draft, regardless of draft position, would be 6.15 players with 3+ year careers, 2.9 players with at least 2+ years as a starter, 5 players with at least 30 games and 6.1 players with at least 15 games. A team should average 0.9 Pro Bowlers a draft.

To be fair, the Khalil’s Mack trade needs to be considered, as well. Add in Khalil Mack as a player across the board on all of those numbers, and the team gains an additional Pro Bowl player, starter, and so on. However, it’s also worth noting that the 2020 draft (which is not even considered in this analysis) is short by 1 first-round pick and 1 third-round pick due to the Mack trade. Thus, the real results relative to Chicago’s draft position should be 33 players with 3+ year careers, 5.4 Pro Bowlers, 16.3 starters for at least 2 years, and 27.3 players with appearances in at least 30 games.

What Pace actually found with all of that draft capital (counting Mack) is 27 players with 3+ year careers, 6 Pro Bowlers, 14 starters for at least 2 years, and 20 players with appearances in at least 30 games. The Bears are therefore short by 6 players with 3+-year careers, they lack 2.3 starters (or at least players who have earned starter roles for at least two years), and they are outright missing 7.3 players who should have made appearances in at least 30 games. In exchange, they are ahead by half of a Pro Bowler.

Verdict: Confirmed. The Bears are essentially behind by one full draft class (more in terms of contributors, less in terms of Pro Bowlers) despite having had prime draft position for four of the first five years of Pace’s tenure. This is because Pace has given up draft picks often to get only average results and because other aspects of team management have not resulted in compensatory picks compared to the rest of the league.

Conclusion

By this point, people have probably made up their minds for better or worse on Ryan Pace. However, for those who are curious, these are what the numbers themselves suggest about how Pace’s drafting compares to typical results from the rest of the league.