When Advanced Stats Don’t Pass the Eye Test: The Curious Case of the 2019 Chicago Bears Offensive Line

Isaiah J. Downing-USA TODAY Sports

This Fanpost was written by a Windy City Gridiron member, and does not necessarily reflect the ideas or opinions of its staff or community.

NFL stats have come a long way in the last 5 years. But some advanced stats need some work before they tell the full story. I think that applies to several stats, but most clearly with the new advance stats measuring o-line play.

The advanced stat most people point to measure offensive line in the passing game is Pass Block Win Rate (PBWR), which measures the rate that the line holds their blocks for 2.5 seconds or more. If the ball is thrown before 2.5 seconds and the line has held the block, that’s a win. So what does that stat tell you? In my opinion, it tells you that you need more information.

My problem with PBWR is it makes a fatal assumption. And it is an assumption over something with huge variability.

PBWR assumes that all pass plays need the same amount of time. In the eyes of this metric a snap and throw WR screen and a seven step drop with a play fake both require 2.5 seconds of pass protection to be successful. That is insane. And that is the fatal flaw of PBWR.

Let’s see how this flaw plays out using the 2019 Chicago Bears

The 2019 season had huge expectations followed by a huge letdown, specifically from the offense. I think all Bears fans can agree with me so far. If not, maybe stop reading now. (If you have seen enough and decide to stop reading, I only ask that you not comment. I welcome a healthy discussion to anyone who is interested, but only after you read to the end.

In the majority of games, just about everything looked terrible for the Bears offense. They couldn’t run, they couldn’t pass. They couldn’t run block, they couldn’t pass block. At least that was what the eye test said. But advanced metrics disagreed. PBWR said the Bears were above average pass blocking. They ranked 13th in the NFL with a 61% PBWR. But the eye test showed miscommunications, unblocked rushers, and a crumbling pocket.

This is where those WR screens come into play. In 2019, the huge number of WR screens the Bears called was the second most common complaint on Bears twitter in 2019 (unofficial). All of those screens go as wins for the offensive line.

After an embarrassing opener, the Bears went to a short/quick passing game week 2 against Denver. Because the ball is out before the defense has time to get pressure, these short passes go as wins in PBWR. This made a bad o-line look better statistically. The irony is that the o-line’s inability to keep a clean pocket long enough to throw down field was a key driver of why Nagy went to a quick passing game. Put another way, the o-line was so bad that Nagy adjusted his game plan to cover for them, and their PBWR stats looked better as a result. That is a problematic stat.

So how do we fix PBWR?

Being a bit atypical of internet writing, I am not just going to complain about a problem, I will propose a solution. For this metric to tell us anything, we need to bucket plays by how much time they need. How do we know how much time a play needs? First, we can see how many seconds the play takes before an on schedule throw, assuming the play results in one. But beyond that, we need to look at the QBs drop and receiver route depth. NFL Next Gen stats track every receiver’s route on every play. By looking at route data across the league we can get an average of time needed for an on schedule throw of each route and route combo. We also use player tracking to look at the depth of the QB’s drop. But we have to account for play action fakes and plays where the pocket collapses before the top of the drop. This may seem too complicated to accomplish, but remember, PBWR is based on player movement tracking, so this would simply add to that.

We bucket each drop back based on these three factors into quick, intermediate, and slow. And instead of applying 2.5 seconds to every play to determine a pass block win or loss, we use the average seconds needed for the type of play. But still adding up all the wins can be misleading, so you would look at the win rate in each bucket to truly evaluate the offensive line. I don’t actually have the route data or drop data to calculate this, but NFL next gen stats does, so if you can implement my proposed Adjusted PBWR, that would be great.

My hypothesis is the Bears would rank significantly lower than 13th. I would imagine they would have a pretty high percentage of throws that fell into the quick bucket compared to the rest of the league. And I would expect their Adjusted PBWR in the intermediate and slow buckets would be well below league average. Further, I would expect their numbers were above their season average in games where the offense shined like week 3 vs Washington, week 13 vs Lions, and week 14 vs Cowboys, plus most of 2018. Because that’s pretty much what the eye test showed.

I’ve heard some advanced stat guys call doubters dinosaurs. If you rely on the eye test, you’re an extinct animal that couldn’t adapt. I’ve worked in financial markets for 20 years using and building data tools. In my experience, when your numbers don’t match the eye test, best practice is to evaluate your formula for potential weaknesses. Sticking with the science thing, maybe this version of PBWR is the Okapi, the Giraffe’s short necked cousin on the evolutionary tree, and Adjusted PBWR gets us one step closer to the Giraffe.

What about Trubisky?

Is this whole article an elaborate excuse for Trubisky? Or do I despise Mitch so much that I could even bring myself to mention him until now?

Neither. I’ve seen a lot of angry Bears fans argue that the o-line was completely at fault for the offenses regression, which I don’t think is accurate. And I’ve seen other fans argue that Trubisky was to blame and that the o-line was not that bad, pointing to PBWR. My point is, when you draw conclusion based on flawed data, you end up with flawed conclusions.

My answer to the Mitch Trubisky question is that we need more data, just like we do with PBWR. Some of the criticism that people use to disqualify him as an NFL QB are that he can’t read the field and that he has no pocket presence. I think he definitely needs to improve his pocket presence, but he has shown success when asked to avoid one pass rusher, but was often asked to avoid two this year and he struggled as a result. I think he needs to work on stepping up in the pocket instead of escaping out. I expect that is something he works on this offseason. Some young QBs come into the league with a veteran pocket presence. Others don’t. But history has shown us that young QBs can get better at avoiding pressure in the pocket as they mature.

Second, I’m not sure how much of an impact the offensive line had on his comfort in reading the field. When I look at clips of his worst decisions and worst throws, they tend to be plays were the o-line crumbled. Again, Trubisky has to play better under pressure. And sometimes the pressure on one play made him feel pressure that wasn’t there on another play. That needs to be fixed. I think his best plays this season were where he threw the ball with touch. Under pressure, he always threw that fastball instead of the change up, and he needs to work on that. But I’ve seen him read the field (see 2018) so I know he is capable of doing so. And that is not typically a skill that QBs lose.

Maybe you’re a respected analyst on this site that has a great twitter feed and better YouTube channel and you say, but his yards per attempt are among the lowest in the league. Is that also a function of the quick passing game that Nagy went to after a letdown from the offense week 1? Maybe. A QB’s stats are a function of what the QB does, what the coaching staff asks him to do, and what the other 10 teammates on the field do. If the coach asks a QB to throw short quick passes, I don’t think you can use a low yards per attempt stat as evidence that the QB is not good.

To be clear, I’m not saying that Trubisky is definitely the future of the franchise. He has things he needs to work on, I am just not writing him off yet. Plenty of talented QBs washed out of the league if they couldn’t develop pocket presence. Trubisky could become another one. Or he can turn it around. Anything is in play. My point is that his career fate has not yet been written. Knowing that QBs have improved with experience in the areas that he needs to improve, I would say I have not seen anything that says he won’t be a quality NFL QB. I wish he was already good at those things too, but that has little bearing on the facts as they are.

If the line improves and he doesn’t, I have no issues with the Bears moving on. I am a fan of the team, not one player. I would be ecstatic with rookie Anthony Gordon or some FA QB they sign coming in and saving next season if needed. I just don’t know how possible that is if the line plays like it did last year.

I’ve seen a lot of respected analysts point to PBWR as a way to say that Trubisky’s 2019 struggles were not driven by the offensive line. To that, I say, I think we need more data.

Numbers never lie. But sometimes they don’t tell you what you think they do.

This Fanpost was written by a Windy City Gridiron member and does not necessarily reflect the ideas or opinions of its staff or community.