Will the Bears win the draft by losing the season?

Suck for Luck. Fail for Cardale. Take some games off for Goff. Whatever name it has at the moment, there is a persistent belief that a mediocre football team needs to tank a season in order to be in a good enough position to draft a top quarterback. Sometimes, of course, no effort is involved for a team to fail miserably—it happens naturally. It might be that kind of year for the Chicago Bears. Still, after the last three games, it's worth wondering if a year of misery is worth it. Is there really that big of a difference between the quarterbacks taken with the top five picks and those taken later in the first round?

I decided to find out by looking at ten years' worth of recent top draft picks. I only looked at the first round, even though I'm sure that there have been some diamonds in the rough that might cloud the data. I do need to admit one big assumption I made before I explain my methodology: I do not believe it is possible to "overdraft" a starting quarterback. I don't care about value at position. If any draft pick gets you a starting QB, then you got value. The position is simply too important to bargain shop. Some readers might disagree, but this assumption still informed my process.

With that assumption in place, we need a few metrics for success. The first measure is easy—quarterback passer rating. It's an old standby, and it's a pretty good yardstick of a quarterback's performance. Second, we can look at wins versus losses. I actually really dislike using wins and losses for a team sport, but I know a lot of people do look at things this way, so let's include it. Finally, we can look at playoff appearances and playoff wins. Supposedly, a good quarterback will get you to the playoffs and then will push the team over the edge once you get there.

There are no modern advanced metrics being used. I love ANY/A, DVOA is awesome, and PFF has their own thing. However, let's keep it simple for now. I pulled numbers from the end of 2014 season, so these might be a little off, but they should still be in the ballpark (numbers come from Pro Football Reference and Sports Reference LLC).

Since the Bears are trying to replace Jay Cutler, it makes sense to use him as our benchmark. Some like him, some hate him, and a lot of people are sick of talking about him, but he's the comparison point because he's not the long-term answer at quarterback, no matter what people think about the short term. As of the start of this season, he had a passer rating of 85.2, a win-loss record of 61-58, and two playoff appearances with one playoff victory. So, in order for a draft to be considered a success, the quarterback has to turn out to be at least as good as Jay Cutler.

Between 2003 and 2012, 30 quarterbacks have been selected (one of them #6 himself) in the first round. I'm excluding anyone from 2013 or later because I want to give those careers a chance to develop before we judge them. Of the 29 quarterbacks who are relevant, thirteen were drafted with a top five pick. Of these thirteen, six have a passer rating of 85 of better (46%): Carson Palmer, Cam Newton, Andrew Luck, Robert Griffin III, Matt Ryan, and Philip Rivers.

Seven of these top picks have more wins than losses (54%), but only some of the names overlap. In addition to Luck, Ryan, and Rivers, winning records belong to Eli Manning, Alex Smith, Vince Young, and Mark Sanchez. Most impressively, eleven of the thirteen have made it to the playoffs—everyone but Jamarcus Russell and Sam Bradford (85%). Half of those who made it to the playoffs managed a win, and the names aren't too surprising: Smith, Newton, Luck, Ryan, Rivers, and Sanchez.

Not bad. Given all of the variables and all of the guesswork that goes into a draft pick, these are some strong results. Basically, it's basically a coin toss to see if you do as well as our benchmark, Smokin' Jay.
Now, what about those fine men drafted in positions six through thirty-two?

Of the 16 of them, only three have a passer rating at 85 or better: Ben Roethlisberger, Joe Flacco, and Aaron Rodgers. Actually, Joe Flacco only had a passer rating of 84.8 as of the end of last season, but it's close enough for a guy with a Super Bowl ring. Even with this extra entry, we're at just 19% of the field. It's not much better when we look at winning records, with Rex Grossman and Tim Tebow bringing our tally up to 5 (or 31%). Byron Leftwich gets added if we include competitors who saw the playoffs (38%), but he is left off again once we require a win in the playoffs.

So, while the sample size is small, there are some pretty divergent results. The quarterbacks available with a pick in the top five had a much better chance of recording a good passer rating, of winning more than half of their games, and of making it to the playoffs.

What's interesting is that conventional wisdom would tell us that the teams that the quarterbacks drafted so highly would be going to substandard teams and that, as a result, they would have worse win-loss records than their peers who landed on more established teams. This apparently isn't the case. In fact, for the whole group, the 13 top-five picks turned in 577 wins and 505 losses (53% winning record) while the rest of the first rounders recorded 446 wins and 411 losses (52% winning record). For the ones who fell out of the top five, 176 of those wins belong to two men: Ben Roethlisberger—who was part of a QB draft class that has three ‘successful' picks in our group—and Aaron Rodgers.

So, with the exception of a rare stacked draft class or one true outlier, it really does seem like there is an advantage to a season of misery, but only if it's miserable enough.