The Disconnect Between Evaluating Talent and Drafting Talent.

Matt Waldman from Football Outsiders, a scout that does a lot of rookie film evaluation studies, who I have much respect for, has a very insightful article about evaluating and drafting. In his article he evaluates Tulane quarterback Ryan Griffin and also speaks of an evaluation he performed back in 2006 between Matt Leinart (USC), Vince Young (Texas) and Jay Cutler (Vanderbilt). Matt also produces a rookie scouting report that is awesome. All credit goes to Matt Waldman. All I have tried to do is condense it and paraphrase for brevity on occasion without losing the essence of his work.

Link to full article:

Matt states that quarterbacks with losing records aren’t at the top of most draft boards. Which seems obvious but, there is more to a quarterback's worth than their team's win/loss record. One of the problems with quarterback development is because of the way reps are given to NFL quarterbacks in many organizations, the lower the draft pick, the less likely that prospect sees meaningful time to develop his game.

Positional need and a prospect’s attributes away from the field appeal to draft-day decision makers just enough that skill and talent aren’t the only factors involved in a player’s evaluation. Sometimes these other factors are important, but they often mislead decision makers.

There is difference between evaluating talent and drafting talent. One is about identifying who is worth picking. The other is about knowing when to pull the trigger. Matt also states; "There is no greater example of the disconnect that can exist between scouting and drafting talent than at quarterback." He also refers to an Alex Marvez report that NFL scouts and management are divided on the importance a quarterback prospect’s win/loss record. Marvez recounts the Broncos’ inner debate of the merits of Jay Cutler, Matt Leinart, and Vince Young.

Matt evaluated all three players. Young was the most physically talented, but he played in an offense that did not require him to develop the craft of quarterbacking from the pocket. Leinart had a good start on the craft needed for the NFL game, but he lacked the physical talent. Cutler had the best combination of athleticism, passing skill, and mental toughness. Matt had Cuter as his top quarterback, Young No.2, and Leinart as an overrated prospect at No.3. The combination of physical talent, positional skill, and on-field behavior mirrors the takes of many NFL scouts. The 2006 NFL draft order for the three quarterbacks was Young first, Leinart second, and Cutler third.

Young’s team won the national championship, Leinart’s team won a national championship the year before and then faced Young’s team in the 2005-2006 title game, and Cutler was on a bottom-dwelling SEC team. Scouts are worker bees; they aren’t the major voices in most NFL war rooms. The general managers and executives are the ones who tend to place the greatest emphasis on win-loss record.

Matt refers to a quote from former Colts GM Bill Polian; "In many respects, you’re going to be asking him to carry your team in the NFL. If he can’t carry his team at the collegiate level, which is quite a bit lower in terms of the level of competition, what makes you think he can do it at the NFL level?"

Matt admits that he may be parsing the words of the former Colts GM too finely; Polian may believe that "carrying a team" is more than just a winning record. At the same time, Matt thinks Polian's response and Marvez’s report illustrates that win-loss records carries too heavy a weight in the war rooms of NFL teams.

Matt thinks if NFL executives do a better job of defining how a quarterback carries a team, they will do a better job of integrating talent evaluation into the draft-day process. Returning to the 2006 NFL Draft of Young-Leinart-Cutler; Young, who many would say "carried" the Longhorns to a BCS title, lacked the maturity and work ethic to be a talented pro prospect and become a consistent, productive pro player. Leinart, had many in the media drawing parallels to Tom Brady’s game, but he also had difficulties crossing the same divide as Young. Leinart and Young were on college rosters with a lot of future NFL players. While Cutler was the college quarterback who consistently showed he could carry his team against opponents that outmatched his teammates.

Vince Young's Texas team had 17 other player from the class of 2006 to 2008 that were drafted by the NFL. Matt Leinart played on a USC team that had 25 players from the same period drafted. The same can’t be said of Cutler. Vanderbilt’s notable NFL players from the Cutler era include Jovan Haye, Earl Bennett, and Jonathan Goff. Texas and USC’s list is staggering by comparison.

Matt recalls the story of Titans executives, coaching staff, and scouts each having different favorites. Owner Bud Adams clearly wanted Young. The scouts wanted Cutler. The coaches were split. Norm Chow, Leinart’s former offensive coordinator and the Titans coordinator at the time, wanted his former pupil. To the best of Matt's knowledge, Fisher’s favorite has never been made public. However, it was divulged on draft day that then-Broncos coach Mike Shanahan called Fisher the night before and asked him about Jay Cutler. Shanahan told the media that Fisher believed Cutler had everything you wanted from a quarterback. The Broncos traded up for Cutler and while he has his flaws, he has been far and away the most successful of the 2006 class and still has potential for a better career ahead.

When Matt watched Young, Leinart, and Cutler, the player he thought who did the best job of "carrying" his team was Cutler, no contest. The reason is that he defines the concept of carrying a team as putting players in position to succeed regardless of the level of competition or the data in the box score, including the scoreboard. Leinart had surrounding talent who routinely put the USC quarterback in position to succeed more than the other way around.

One of the big reasons Matt had Steve Smith as his No.3 receiver prospect in 2007′s draft class was that he demonstrated NFL-caliber athleticism, technique, and awareness in situations that his quarterback Leinart created when his execution was not NFL caliber. Smith carried Leinart in the passing game more than Leinart carried Smith.

Despite great surrounding talent, there was no question that Young carried his offense at Texas. However, Young was thrust into a pro game that expected him to acquire and refine skills that were not the strength of his game. Matt's buddy Sigmund Bloom has wondered how Young may have fared if his introduction to the NFL game was through a spread/pistol scheme that Robert Griffin has in Washington.

Drafting to Win vs. Drafting Not to Lose:

Matt states; that the NFL's decision change in mentality is still slow because present decisions seemed to be reinforced by past history. There’s urgency for teams in need of a quarterback to select one in the first round despite the fact that the failure rate remains high.

The fear of not finding a quarterback certainly comes into play for most teams. Coaches without a strong quarterback are probably going to be looking for work sooner than later. Matt has a mathematical background so he has a different way of looking at this problem than a lot of coaches. His research into drafting quarterbacks reveals a second-round prospect’s chances of becoming a solid starter is around 20 percent, maybe a little higher for earlier picks in the round. The likelihood of a second-round pick at another position becoming a true difference-maker is 50-50 at best.

Most coaches would tell you if they had the mathematical background. A 20 percent hit rate in the second round isn’t very good, but it continues to get worse and worse as the draft goes on. The Bradys, and even the Hasselbecks, come few and far late in the draft. The math makes sense to the degree that it explains the results of the current decisions that NFL teams are making. However, that math doesn’t tell why a second-around prospect has less of a chance being a difference maker than the first-rounder or why the Bradys and Hasselbecks are rare. The standard explanation is that better talent tends to be drafted earlier than lesser talent.

The problem Matt has with this explanation is that the teams experiencing success often have game changers who were exceptions to the rule: Tom Brady, Kurt Warner, and Russell Wilson are three examples. The greater the exception, the more dramatic the advantage. Playing the percentages may keep a team from making draft-day mistakes that compound with each pick, but it can also keep a team from winning big.

Organizations often use the data "not to lose." The problem with making these decisions based on this data is that when a team fails on these "not-to-lose," first-round quarterbacks it has committed to a three or four year process of giving a player a chance and/or a huge sum of money. Missing on a high-round quarterback and playing out the string of "appropriate development time" is a huge setback.

Based on what Matt believes about Matt Barkley and Mike Glennon’s game, selecting them in the first two rounds of the 2013 NFL Draft will be a decision two teams will make "not to lose." They fit all the safe bullet points in terms of physical potential, system, and basic skills at the position. Neither possesses the slam-dunk, early-round skills, in-game performance, and potential to put their players in position to win on a consistent number of snaps. The problem with making these decision based on this data is that when a team fails on these "not to lose" early-round passers is that it has committed to a three or four year process for a lot more money than taking a lower-round talent with equal or greater potential, but less marquee value.

Considering need is fine, but Matt believes a team should always build on talent. If the talent fits the need – great, but the most important skill that evaluators and executives may need to develop is how to resist the pressure of succumbing to need over talent and fooling themselves into thinking they haven’t.

Whether he's right or wrong about Barkley or Glennon is not the point. The disconnect between scouting and drafting is apparent and it will continue even if Russell Wilson’s selection and open opportunity to earn the job in Seattle is a potential glimmer of change. However, Wilson was a winner at N.C. State and Wisconsin. Marvez’s piece shows that there is a belief if the quarterback isn’t succeeding when it comes to the bottom line then it’s a red flag for his NFL potential.

On the one hand football is the ultimate team sport, but no individual is more celebrated and coveted in any sport than the quarterback. NFL personnel believe that you can’t win without one, yet there are plenty of superstar passers who lacked the surrounding talent to get the job done. It’s not a one or the other proposition.

Since Emery has a scouting background, maybe the Bears will have a better approach to the draft than others.

In the remainder of Matt's article he does a film breakdown on Tulane's Ryan Griffin. If the Bears could get him in the 4th round or later, they may have their future quarterback.

<em>This FanPost was written by a Windy City Gridiron member, and does not necessarily reflect the ideas or opinions of its staff or community.</em>

Log In Sign Up

Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Windy City Gridiron

You must be a member of Windy City Gridiron to participate.

We have our own Community Guidelines at Windy City Gridiron. You should read them.

Join Windy City Gridiron

You must be a member of Windy City Gridiron to participate.

We have our own Community Guidelines at Windy City Gridiron. You should read them.




Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.