How Not to Scout a Quarterback, Part 1

Presume, for the moment, that Ryan Pace elects not to engineer a trade of some kind and therefore ends up looking for Jay Cutler’s successor in the draft. For that matter, simply presume that you are a bored football fan trying to crack the code and figure out what to look for in a college prospect. What’s the best way to sort through the haze? How can we spot what a good quarterback looks like coming out of college?

Ignore stats and watch film. This might sound strange coming from someone who typically writes articles laden with stats, attempting to quantify a game that has so many moving parts. However, the fact remains that there is no clear and simple cutoff for a “good” prospect’s college stat line.

Of course, as a wonderful paradox, I’m going to need some stats to make my case about the futility of using stats. Somewhere, my high school math teacher is laughing at me.

First, in order to figure out what a good quarterback looks like coming out of college, we need to have a definition of a good quarterback in the NFL. This is, inherently, a heated subject. However, we have a few ways of looking at it. In the last 10 years, there have been 60 quarterbacks drafted who have at least 100 pass attempts. Almost by definition, these are “better” quarterbacks or at least “highly drafted” quarterbacks, because it’s not easy to earn three starts in the NFL at QB, and that’s the minimum for the group. However, their performance varies a lot.

Lacking anything more concrete to use, I sorted each quarterback by completion percentage, yards per attempt, and touchdown-to-interception ratio. A quarterback in the top third of any category received a +1, while a quarterback in the bottom third of any category received a –1. This left me 18 quarterbacks with at least a +2 score (to get a +2, a player could not have been bottom-third in any group and had to be top-third in at least two categories). There are two notable exclusions—Derek Carr missed the cut because his completion percentage was bottom-third, and Jimmy G missed the cut because he has only attempted 94 passes.

Meanwhile, this same approach left me with 17 quarterbacks with a –2 or worse (to get a –2, a player could not been top-third in any group and had to be bottom-third in at least two categories). I did not do anything more sophisticated than this—this is a very rough cut, but a look at the names tells me that these are reasonable lists.

For those of you who are curious, if I were to insert Jay Cutler into the mix, he would be in the “good” camp. His scores against this same pool of 60 quarterbacks would have him as the 14^th-best in completion percentage, the 15^th-best in yards per attempt, and the 26^th-best in touchdowns to interceptions. Let the attacks on methodology begin, as anything calling Jay Cutler “good” must be flawed in the eyes of some.

Back to the subject at hand. What did the quarterbacks on each of these lists look like coming out of college?

They looked like one another.

The average completion percentage of the “good” group coming out of college was 63%, with five of these eighteen men (almost a third) below the magical “60%” cutoff. Meanwhile, the “bad” group had a completion percentage in college of around 62% and five of those seventeen men similarly posted sub-60% completion rates.

The average adjusted yards per attempt, frequently used as a solid indicator of performance, was 8.6 for the first group and 8.1 for the second group. Before that half-yard split looks too promising, consider that the worst AY/A from the good quarterbacks (Matt Ryan) is 6.5, while the best among the bad quarterbacks is 10.6 (Bryce Petty). Likewise, passer rating varies wildly and even measuring the total number of attempts, games played, or attempts per game ends up in a mess.

Why?

There are lots of reasons, but one big reason is that the people who find and select NFL players for a living are not—despite fan protest—total idiots. If there really were a formula to determine the minimum level of performance necessary for success, then more GMs would have found it. Instead, JaMarcus Russell’s 147.9 college passer rating just barely edges out Russell Wilson’s 147.2. Across 45 games apiece (with both playing for Michigan State), Drew Stanton’s 64.2% completion percentage is just a hair more impressive than Kirk Cousins’ 64.1%.

Even Football Outsiders, which tries very hard to get to objective analysis of football, includes a scout’s rating in their own QBASE formula before determining how well a college prospect is likely to perform in the NFL.

Thus, when evaluating a prospect, it is tempting to get enamored of a stat line. However, when evaluating a quarterback prospect, the most numbers can do is serve as a cross check. They can provide a warning flag, sometimes.

For these reasons, it’s good for even casual fans to get some video and to actually watch the prospect play (for extra fun, watch alternating college and pro video to really get a sense of how different the game is at the two levels).

Next week, I want to try to look at whether or not there are any legitimate warning flags about top contenders. However, for now, just remember that when someone quotes a stat line from college that those numbers are unlikely to have much connection to the athlete’s pro career...

Unless otherwise noted, all stats come from Pro Football Reference