Questioning the Predictions for the Draft

The draft is almost here, and there are plenty of experts willing to tell you which team is going to take which player. There is an entire industry devoted to making these predictions, and we're no strangers to that game ourselves here at Windy City Gridiron.

For Bears fans this season, one big issue is wondering who might to slip to #11. Alternately, it might mean wondering who we want to slip to #11.

Is there really any way of figuring it out, or is everybody just guessing? To me, it seems like there are three distinct questions at play.

First, how much should anyone worry about the combine?

For the last couple of weeks, I have been promising to talk about the overlap between combine results and performance. However, except in very special circumstances, there isn't one. Don't take my word for it, however. Experts of all kinds agree. Running backs who are already considered very good might be better off when they have a faster 40 time, but the same isn't really true of wide receivers. There is very little evidence to suggest that combine performance means much of anything at all. Part of this is because the tests are redundant [warning: paywall on this one], but another part of it is that raw athleticism is such a small part of the game once you are talking about the top 1% of the human population in terms of athleticism. If you can get combine results or the feedback of an informed individual who knows how to view tape, the tape is the way you should go.

However, because GMs are fighting for their careers, they will often attach value to the combine in a desperate effort to sort the wheat from the really, really good wheat (I can't bring myself to call All-Americans who are only in the top 1.1% of the population, instead of the top 0.9% of the population, chaff). There might be certain minimum thresholds that a GM has for particular positions, but if there's a player a team likes and the tape is there, they are better off going on that than they are over-analyzing bench press reps.

Second, do the experts really know where players are going to be drafted?

To get a sense of how much the experts might be able to tell us, I looked at the first round of the draft for the last three years and I pulled the "closest to the draft" mocks I could find from each of three sources: SB Nation's Dan Kadar, ESPN's Mel Kiper, and NFL's Mike Mayock. Each of these men had 96 chances to be right, and together they represent three different ‘sources' of expert analysis. In addition to looking at whether or not they got the pick right, I also looked at whether or not they got the position right and whether or not the player they chose to go at that spot was selected within 5 spots and/or within 10 spots of where the expert in question predicted.

The results are far from overwhelming, and they likely confirm what a lot of us suspect--there's some obvious choices and a lot of guesswork. For example, Kiper and Mayock tied at getting 20 of the 96 picks exactly right--the player went exactly when the expert said, usually to the team the expert projected (Sammy Watkins was #4 on two of the three boards, which is when he went, if not where he went). Mayock should probably get credit for three more times he matched player and team, only at different picks (for example, he had the Raiders trading up to 28 to take Derek Carr, when instead they waited until 36).

Kadar, on the other hand, did the best when it came to projecting which position would be selected (he was right 34 times out of 96), but the others were close (Kiper trailed at 26 correct positional hunches). However, all of the experts did much better when it came to projecting about when in the draft players would go. All of the mocks in question predicted within five slots the eventual draft position of more than half of the players (52 for Kadar, 54 for Mayock, and 57 for Kiper). Likewise, even the least-accurate of the experts (Kadar, in this case) predicted two-thirds of the players' draft positions within ten slots.

As one might expect, toward the top of the draft things became a little more scripted. However, even this was far from a sure thing. Mayock (again in the lead) only correctly predicted the top ten picks in each draft a total 14 of 30 possible times. It's better than I could have done, but even the experts are wrong more often than they are right.

Kiper was the best at predicting ‘about' where a top player would go, calling an average of 90% of his top ten picks each year within five positions. However, that range of "within five positions" is a huge qualifier, especially for a fanbase whose team is sitting at #11. Five positions could be the difference between having a chance at a player projected to go #7 or having to settle for the relative scraps. This leads to the final question.

Third, what are the chances of a "clear" top ten prospect slipping down the draft?

For Bears fans, what probably matters most is how many of the players projected to go in the top ten were gone, each year, before #11 rolled around? The answer is that not even the experts really know who is a top ten pick. The closest I looked at was Dan Kadar, who predicted 9 of the top 10 in 2014 (he missed on Manziel, who slipped to #22). However, overall Kadar was still only right on 23 out of 30 tries when it came to predicting who would be taken before #11. Kiper was also at 23 out of 30. Mayock was a hair better at 24 out of 30.

It's worth looking at who dropped. Most often, it was the players they disagreed about who slipped. If they all agreed on a prospect, there was a pretty good chance he was gone before the eleventh spot. In 2013, only one player projected to go in the top ten on at least two of the three boards dropped, and he dropped a fairly long way. Sharrif Floyd was projected to go #3 on all three boards, but he fell to #23, below two other DTs. I have no ready explanation for this drop, and my guesswork has to be inferior to the analysis of our resident draft expert, so I reached out to EJ Snyder, who suggested the following:

"Floyd was a case of real groupthink/over-projection. People started thinking he was Warren Sapp because he was a good athlete, at a great size who had shown flashes. I never got the way-up-high love for him, so I thought he got drafted in a more natural range. By the way, if you want history to repeat itself, watch Butler this year. Same pre-draft pattern except Butler is from a small(er) school."

Next, in 2014, Johnny Manziel and Taylor Lewan were each on two of the three boards, and both fell out of the top ten. The issues with Manziel could fill a book, and the reality is that the fact that he was placed so high in the first place is a clear sign of the "quarterback grade inflation" that is rampant through the league. Lewan fell from either #6 or #9 (depending on if you asked Mayock or Kiper) to land with the Titans at #11; this beat Kadar's projection of going to the Giants at #12 by one slot. Given his off-field issues, a small drop is probably not too tough to explain, especially with the other players--likeJake Matthews--available.

Finally, in 2015, Bud Dupree was projected to go in the top ten by all three analysts. He was supposed to go 6th or 8th and instead fell to 22nd. Again, he seems like a good player, and there were chances for him to be taken. Again, I reached out for an answer:

"Dupree was similar to Floyd, but different. He had the same profile as Floyd (good flashes, decent testing, not quite the production you'd expect) and his play style only suited certain defenses. The D's that took rush DE's ahead of him (JAX, ATL) wanted speed ATHLETES and Dupree was the 3rd best one... so they took Fowler and Beasley. The last thing that made people waffle on Dupree was Za'Darius Smith (Ravens, R4). Right at the end people started to wonder if he wasn't better than they'd thought originally, and possibly creating/facilitating more of Dupree's "wins". It was just enough doubt to cause him to slip a bit (not far). The draft is a very, very fickle beast."

That fickle beast is almost upon us once again, and it's tough to escape the conclusion that every year someone who was favored dropped down to what would be the range where the #11 pick would have been able to take him. Whether or not the Bears will be interested in these falling talents remains to be seen.

Fortunately, the suspense will be over soon, for at least one more year.