There’s a new approach to NFL QB projections — and the 2018 draft class is in trouble

[oc_jet]

A long article analyzing the 2018 QBs. Same results as most other analysis is telling us...Mayfield's history of successful performance has the highest probability of being repeated and Allen is likely to repeat his mediocrity...or worse. Note: bolding is my highlighting key points. As much as most of the pundits keep talking up the other guys and talking down Mayfield, all the stat analysis, actual performance and history says Mayfield is the guy. How the top of this draft unfolds will be FASCINATING!!! And hopefully in a good way for us....

 

https://www.sbnation.com/nfl/2018/4/5/17046116/2018-nfl-draft-quarterbacks-josh-allen-sam-darnold-projections-hype

There’s a new approach to NFL QB projections — and the 2018 draft class is in trouble

Looking for a quarterback’s NFL ceiling? Look no further than college stats.

By Bill Connelly@SBN_BillC  Apr 5, 2018, 9:00am EDT

According to Football Outsiders’ DYAR ratings — defense-adjusted yards above replacement — the top five NFL quarterbacks in 2017 were New England’s Tom Brady, the Los Angeles Chargers’ Philip Rivers, New Orleans’ Drew Brees, Minnesota’s Case Keenum, and Pittsburgh’s Ben Roethlisberger.

• Brady was a moderately successful two-year starter at Michigan and a sixth-round draft pick.
• Rivers was a record-setting four-year starter at NC State and a top-five pick.
• Brees was an undersized but prolific three-year starter out of an early version of the spread offense. He was picked in the second round.
• Keenum was an undrafted journeyman and former two-star recruit who threw for nearly 20,000 yards out of the spread in college but just bounced to his fourth team in six seasons in the NFL.
• Roethlisberger was a small-school star, a three-year college starter at Miami (Ohio), and a top-15 pick.

Three power-conference players and two mid-major stars. Two top picks, a second-rounder, and two draft afterthoughts. Two former blue-chippers and three two- or three-star guys. Three players with perfect QB size and two undersized gunslingers. And lest you think experience was too much of a predictor here (since four of these five are up there in years), the No. 6 (Jared Goff) and 8 (Carson Wentz) QBs on the DYAR list were second-year guys, so that only goes so far as well.

The top of this list of passers was low on indicators and high on symbolism. The simple truth: It’s really, really hard to identify good quarterbacks before they become good quarterbacks.

Heading into the NFL Draft at the end of April, so much analysis always focuses on the idea of projection, on what a guy might be able to do, not necessarily what he’s done.

This makes sense, of course, to a point — so much of success at any level is based on situation, scheme, and circumstance. The right coach, teammates, or system can make an immense difference, plus these guys haven’t faced NFL talent, with NFL coaching, before. Plenty of QBs with great college stats have bombed out in the pros, and plenty with merely good stats have thrived.

NFL GMs can be forgiven for thinking that, once we get a kid in our system, it’s all gonna work out just fine. We can fix his flaws and maximize his talent. Stats will only tell you so much. They are, dare we say, for losers.

If we look at the right stats, however, and do so from the right perspective, we can still get further down the road than we would get just relying on basic stats or the eye test.

For instance, we definitively know a prospect’s ceiling: His college stats.

It makes sense, right? Just as we don’t expect a blue-chip running back to average 12 yards per carry in college like he perhaps did in high school, a college back who averaged seven yards per carry in college probably isn’t going to do so in the NFL. And the odds of a quarterback matching his college stats at the next level are almost null.

That is, he won’t match his rate stats. Keenum, for instance, isn’t throwing 50 times per game in the NFL like he did in his junior year at Houston, so his per-game yardage totals will be different. But things like completion rate, interception rate, etc., can be more telling. And success rate can be extremely telling.

Success rate and IsoPPP (isolated points per play) have been go-to stats for a lot of my college analysis in recent years.

• Success rate is a common Football Outsiders tool used to measure efficiency by determining whether every play of a given game was successful or not (the terms: 50 percent of necessary yardage on first down, 70 percent on second down, and 100 percent on third and fourth down).
• IsoPPP, meanwhile, looks at the magnitude of the successful plays in terms of expected points.

As it turns out, the correlation between one’s success rate in college and in his first four years in the pros is around 0.272, better than other rate stats I experimented with.

There are 38 quarterbacks who a) were drafted from an FBS school between 2010-17 and have thrown at least 300 passes in the NFL. This is not a huge group, and it overlooks players who either just entered the league, have battled injuries, or, of course, weren’t good enough to throw 300 passes in the league. Looking at college-to-pro results will always have limitations like that.

We can still pretty clearly learn something from these 38 QBs. For starters, none of them exceeded their college success rate in their first four years in the pros*.

* Why first four years? Because that’s generally how long a rookie contract lasts. If you pick a guy who’s going to need two or three years of grooming, you might lose him as soon as he becomes a viable player.

Your college success rate is your ceiling.

Almost the only guys who came the close to matching their college success rates were near the bottom:

• Brock Osweiler (45.4 percent success rate in college, 43.9 percent in first four NFL years) mostly sat the bench for three seasons, then parlayed a fourth-year surge into a lofty contract (and promptly fell apart).
• Jake Locker (40.0 percent / 38.3 percent) was the least efficient college QB in the sample and one of the least efficient in the pros.
• New Viking Kirk Cousins (48.5 percent / 46.1 percent), a fourth-round pick in 2012, also perhaps benefited from early-career bench time before thriving. We’ll see if his sparkly new contract ends up a better investment than Osweiler’s.

Osweiler and Cousins got to sit and learn for a while before being sent into action. A lot of top draft picks, however, were given early playing time, both because of their potential and the fact that teams had invested top draft picks in them. For players like Jameis Winston (45.8 percent success rate in the pros), Cam Newton (42.6 percent in his first four years), and Marcus Mariota(43.4 percent), this has worked out pretty well.

For others, like Gabbert (32.9 percent), Tim Tebow (35.8 percent), and, thus far, Mitchell Trubisky(34.0 percent), it has meant early exposure of all their flaws to opponents. Gabbert and Tebow never really recovered, but Goff (29.6 percent in his rookie year, 44.3 percent in his second) did.

Jared Goff’s first season in the pros was a nightmare. His second ended in the Pro Bowl.

 Aaron Doster-USA TODAY Sports

We don’t learn as much about the guys with great college efficiency as the ones with statistical deficiencies. Your success rate is going to sink as the degree of difficulty improves, and while the most efficient college quarterbacks have the best odds of pro efficiency, the variance is pretty high.

But your ceiling is your ceiling, and even if this doesn’t say much about guys with obscene college stats, it says a ton about the Lockers and/or Blaine Gabberts of the world, the guys with mediocre stats and standout physical traits, the guys about whom scouts will say “Yeah, his stats aren’t that good, but I can fix him. Just look at that arm!”

So what does this tell us about this year’s draft prospects? Less than amazing things.

I’ve been very confused by the chatter about this being an amazing QB draft class. The buzz began before the 2017 season and continued despite Wyoming’s Josh Allen regressing drastically from a statistical perspective, UCLA’s Josh Rosen continuing to struggle with injuries, and USC’s Sam Darnold dealing with some turnoveritis.

I have long suspected that this QB buzz has come in part because most of the truly best players in the draft play positions that don’t tend to warrant the top pick — running back (Saquon Barkley), offensive guard (Quenton Nelson), safety (Minkah Fitzpatrick, Derwin James), inside linebacker (Roquan Smith).

That is to some degree understandable. But you have to add a lot of favorable context to these stats to convince yourself that this is even an above-average QB crop.

Since the highest four-year pro success rate from any QB in this sample is 46.1 percent (from both Cousins and, thus far, Dak Prescott), and since 35 of 38 quarterbacks in our draft sample were at least three percentage points lower in the NFL than in college (most were much further away than that), let’s set an artificial bar at 49.1 percent. Those at or above that mark are the ones with Prescott-level early-career efficiency potential.

Where do this year’s 13 primary QB prospects land?

2018 QB prospects with a career success rate of 49.1 percent or higher:

• Baker Mayfield, Oklahoma (54.8 percent)
• Sam Darnold, USC (52.0 percent)
• Mason Rudolph, Oklahoma State (50.0 percent)
• Logan Woodside*, Toledo (49.5 percent)

Of the 38 players in the NFL sample, only Winston (57.1 percent) and Bradford (55.4) had higher career success rates in college than Mayfield. Winston’s first three seasons in the NFL have generated a 45.8 percent success rate; Bradford battled injury and a porous offensive line, generating a 37.2 percent success rate in his first two years before rising to 42.3 percent, near the league average (which is usually between 42.5 and 43 percent), in his next two.

Again, since both were drafted so high, there was no sitting — they threw a combined 1,181 passes in their respective rookie seasons. Mayfield and Darnold will potentially be thrust into action just as quickly. Rudolph and Woodside, perhaps less so.

* Note: these are raw stats, unadjusted for opponent, and we don’t have enough of a sample of QBs from mid-major schools to know how much of a difference to expect from that jump. But the guys in the sample — Colin Kaepernick (47.3 percent in college, 42.1 in the NFL), Andy Dalton (49.5, 42.6), Derek Carr (47.1, 41.0), and Blake Bortles (51.6, 39.6) — have made the statistical transition about the same as the power-conference guys.

Baker Mayfield’s college stats were otherworldly.

 Mark D. Smith-USA TODAY Sports

Here are the 2018 QBs who came relatively close to that 49 percent mark:

2018 QB prospects with a career success rate within two percentage points of 49.1 percent:

• Riley Ferguson, Memphis (49.0 percent)
• Nick Stevens, Colorado State (48.9 percent)
• J.T. Barrett, Ohio State (48.6 percent)
• Luke Falk, Washington State (48.0 percent)
• Lamar Jackson, Louisville (47.4 percent)
• Mike White, WKU (47.4 percent)

You can’t really get a conclusive read on prospects within this range, especially someone like Lamar Jackson, whose rushing ability is incredible. (I limited this look to just pass proficiency.)

You can, however, draw some pretty stark, alarming conclusions about prospects in this range:

2018 QB prospects with a career success rate lower than 47.1 percent:

• Rosen (46.6 percent)
• Litton (45.2 percent)
• Allen (43.3 percent)

Of the 38 QBs in our pro sample, only one (Osweiller) managed a league-average passing success rate in the NFL over his first four years after producing a college success rate this low. The only two QBs in the lower-efficiency range who were drafted in the first round, as Rosen and Allen will be: Gabbert and Locker. Not the greatest of role models.

Of course, Rosen’s career numbers were dragged down by the simple fact that he played as a true freshman. Allen didn’t, nor did Gabbert. Rosen’s success rate improved over his three seasons, from 44.8 percent as a freshman in 2015, to 46.3 percent in 2016, to a perfectly solid 48.7 percent last year. So maybe he’s in the clear.

Allen, however? If you’re likely to finish, at best, two to three percent below your college success rate, that means his ceiling is around 40.5 to 41 percent. That’s Ryan Mallett territory (40.8 percent). As a ceiling. Are we sure we’re willing to spend a top-five pick on a guy who might, with some good breaks, become Ryan Mallett?

Maybe he goes on to become the outlier of outliers, as insisted on by every draft scout who watches him throw in shorts against no defenders. But what an incredible gamble it will be for whatever team inevitably picks him in the top 10.

Josh Allen’s Wyoming stats were ... lacking.

 Brian Losness-USA TODAY Sports

To add further context to these numbers, let’s run some basic projections. To do so, though, let’s talk a moment about explosiveness.

Back in January, I began playing with what I call marginal efficiency and marginal explosiveness.

Marginal Efficiency: the difference between a player’s success rate (passing, rushing, or receiving) or success rate allowed (for an individual defender) and the expected success rate of each play based on down, distance, and yard line.

Marginal Explosiveness: the difference between a player’s IsoPPP (passing, rushing, or receiving) or IsoPPP allowed (for an individual defender) and the expected IsoPPP value of each play based on down, distance, and yard line.

For offensive players, the larger the positive value, the better. For defensive players, it’s the opposite — the more negative, the better.

In my 2018 college football preview series, I have been integrating marginal efficiency and marginal explosiveness into my player analysis, and it works pretty well. You can never truly isolate one player’s performance from others’ using play-by-play stats — there’s always extra context to address — but this can perhaps take us further down the road. In this case, it basically tells us that big plays don’t carry over to the pros.

While the correlation between one’s marginal efficiency in college and the pros is about the same as success rate, the correlation for marginal explosiveness was much lower (0.099). That is to say, there’s almost no relationship.

This was what I expected to see, both because of the inherent randomness of big plays and the fact that, because of fewer lopsided matchups and/or crippling errors, there are fewer big plays in the pros. But it means that we have to stick mostly to efficiency when attempting to make college-to-pro projections. That’s a differentiation we can’t make with more standard stats like yards per attempt.

I ran a simple regression to see how these players’ college stats might translate to the pros. I included explosiveness below as a way to figure out who we might be getting a false impression of as much as anything.

Players like Mayfield, Rudolph, Woodside, Ferguson, and Allen likely benefited more from a level of explosiveness that won’t carry over to the pros. Mayfield nailed the efficiency element of the routine as well, however. Allen, on the other hand...

Since a common retort from Allen advocates has been that his supporting cast was terrible in 2017, and that this should negate his mostly awful stats from last fall, I included a projection based on his 2016-only stats as well.

Using 2016 upgrades him from DeShone Kizer to Tim Tebow.

Stats will never tell you everything about what a player can do. In this case, though, it tells you what certain players probably can’t.

And in the case of Josh Allen, it would take a spectacular outlier performance — one that hasn’t happened this decade — to live up to the expectations of the top-five or top-10 pick it appears he will become.

[rangerous]

hmmm.  another statistical projection that may or may not have some meaning.  imo one of the biggest issues is how well or fast the college qb adapts to the nfl game and not just the x'x and o's.  there's just the whole aura surrounding the nfl.  most of these guys have dreamed from an early age of being the next montana or namath or even brady and then they get thrust into the spotlight.  some will wilt and some will thrive.  that can't be gaged by any of the statistical analysis out there.  and that's why a guy like allen, with his high wonderlic, and actually turn out to be pretty good because he will be able to not be awed by the pro game.  we'll see what happens later this month.

[T0mShane]

10 minutes ago, oc_jet said:

f we look at the right stats, however, and do so from the right perspective, we can still get further down the road than we would get just relying on basic stats or the eye test.

The dude seems to be doing a sh*t-ton of equivocating throughout this analysis, which leads me to think he doesn’t believe much of it himself. 

[greenwave81]

The only problem with regression analysis is it assumes everyone is operating from the same, or relatively (to make comparisons relevant) the same position to begin with...which in football is impossible of course.

What would Mayfield’s stats look like if he was at Wyoming? What would Allen’s look like at Oklahoma? No one can say.

Then you throw in the different offenses these guys operated out of and need to consider that too.

There’s only so much statistics can tell you...sometimes you need to look at an individual and make a judgement outside of ‘pure statistics’ is all I’m saying. That’s what a good scout does.

[Larz]

If it came down to stats, there would never be busts and geno Smith would be a pro bowler 

Fans don't like it but the art of drafting is basically predicting the future based on circumstances you have never seen the candidates face like million dollar contracts and poisonous tabloid media 

Don't forget, leaf is a better prospect than Manning because he is an alpha male 

Handing in the card isn't even the beginning of the process. 

Coaching, development strategies, supporting cast and sometimes just dumb luck play a role 

My biggest concern really isn't who they pick its how they develop him and if this city and fanbase can be patient 

[oc_jet]

16 minutes ago, rangerous said:

hmmm.  another statistical projection that may or may not have some meaning.  imo one of the biggest issues is how well or fast the college qb adapts to the nfl game and not just the x'x and o's.  there's just the whole aura surrounding the nfl.  most of these guys have dreamed from an early age of being the next montana or namath or even brady and then they get thrust into the spotlight.  some will wilt and some will thrive.  that can't be gauged by any of the statistical analysis out there.  and that's why a guy like allen, with his high wonderlic, and actually turn out to be pretty good because he will be able to not be awed by the pro game.  we'll see what happens later this month.

Yes, some will wilt and some will thrive. That is why the exercise of choosing a QB in the draft is such a crap shoot. The whole point of this analysis is to actually look at what these guys have actually done under specific circumstances to see how well they actually did. Yes, they all play in different conferences and under different systems. But they were all asked to do the same things within the confines of the game. This assessment is measuring how well they actually did that. While past is not always prologue, it is a good indicator of what someone is capable of. 

The troubling thing for me about Josh Allen is the comments made by Dan Orlovsky in that King podcast. Dan Orlovsky was a professional QB in the NFL for 12 years. Yes, he may have been a career backup, but he did not last that long without having some decent skills and without having learned something. He also played in a number of different systems behind some decent QBs. So, when he looks at a QB perform, he knows what he is looking at and what he is looking for. That he felt that Allen did not have " a plan" when he came to the line of scrimmage to deal with what the defense would throw at him is very telling. His wonderlic scores indicate that he is very smart but that may only be book smart. He doesnt appear to be "street smart" with the ability to figure things out quickly on the fly. Which is the exact skill needed to be a successful QB in the NFL.

[Villain The Foe]

You like the QB's you like and you dislike the QB's you dislike.

This entire offseason has been filled with folks dismissing stats in order to criticize QBs as well as making excuses for QBs who dont have those same stats. Its all subjective. 

None of us know. What is safe to say is that If the Jets do not become a stable organization with a competent coaching staff, it wont matter who we pick. 

 

[RoadFan]

And how do any of these projections incorporate supporting cast and opposition into the equation?

All of this number-crunching stuff is useful to a point.  Just as the eyeball test is useful to a point. 

There is no exact science to scouting... especially at the QB position.  Too many variables.  All of the 2018 QB prospects are just that.  Prospects.  1 of them might be great.  1 might be very good or good.  1 or 2 serviceable starters.  A couple of busts...give or take... The usual.

 

[UntouchableCrew]

Any metric that thinks JT Barrett is going to be more successful in the NFL than Josh Rosen is completely and totally useless.

[j4jets]

2 hours ago, T0mShane said:

The dude seems to be doing a sh*t-ton of equivocating throughout this analysis, which leads me to think he doesn’t believe he has no life  

Fixed it for you. 

[Patriot Killa]

Rosen’s comp is Weeden? 

Boy oh boy.

[Untouchable]

And here is a perfect example of why analytics and football don’t go together.

That sh*t may work in Baseball, but there are way too many variables when projecting these guys to the NFL.

The analytical data said that guys like Colt Brennan and Kellen Moore should’ve been smashing successes at the pro level.

Now these dicks are using a bunch of numbers and pie charts to say that J.T. Barrett (who may not even be drafted) is going to be the best of the top QB class since ‘04...lol.

[dbatesman]

4 hours ago, Untouchable said:

The analytical data said that guys like Colt Brennan and Kellen Moore should’ve been smashing successes at the pro level.

It did?

[NoBowles]

Math is literally never, ever wrong, it’s dumb ass people who use it wrong that are the problem.

[Obrien2Toon]

What the hell did I just read

and JT Barrett WTF

[Grandy]

Logan Woodside for Jets QB!

Anybody?

no?

okay.

[Dcat]

Bill Connelly's mentor

[20andOut]

Yards above replacement? IsoPPP? Who wrote this sh*t, Paul DePodesta?  Sounds like sabermetrics.

I f--king hate sabermetrics!!!!

[Patriot Killa]

20 minutes ago, 20andOut said:

Yards above replacement? IsoPPP? Who wrote this sh*t, Paul DePodesta?  Sounds like sabermetrics.

I f--king hate sabermetrics!!!!

You know it was super foggy whether or not you disliked sabermetrics until THANKFULLY! you capped it off with 

“I f*cking hate sabermetrics.” 

I’m glad you didn’t leave us in the dark here. I was very confused at first.

[20andOut]

9 hours ago, Patriot Killa said:

You know it was super foggy whether or not you disliked sabermetrics until THANKFULLY! you capped it off with 

“I f*cking hate sabermetrics.” 

I’m glad you didn’t leave us in the dark here. I was very confused at first.

Your welcome.

[k-met57]

April should be known as the "there is nothing actual happening, so we will just die of analysis paralysis month"....i would venture to say, April is where a lot of bad draft picks originate...

[ASH1962]

That's exactly why we need to take a kicker, safety, or CB at 3 and forget this QB nonsense. What do need a QB for anyway  ?

[Ohio State NY Jets fan]

I thought it was a really good read thanks to #oc_jet for posting

As a Buckeye fan I was surprised to see Barrett that high on the list but remembered that the goal of this analysis is an attempt to predict a quarterback’s ceiling in the NFL based on his college stats, it does not attempt to predict the likelihood of reaching that ceiling (and could not since hacks were not included, needed 300 passes in the NFL to be included)

I thought the marginal explosiveness might help adjust for that but also remember a bunch of explosive RB plays during JT's career (Zeke says hello) and the Buckeyes were on the right side of more lopsided matchups than many other teams that greatly helped his college stats.That said, Barrett has skills - we saw that vs PSU last year, he was just not consistent (would be another interesting stat to add)

The author Bill Connelly started the article with so much of success at any level is based on situation, scheme, and circumstance. The right coach, teammates, or system can make an immense difference, plus these guys haven’t faced NFL talent, with NFL coaching, before. 

He also mentioned the following with Rosen to help explain his ranking 

Rosen’s career numbers were dragged down by the simple fact that he played as a true freshman. Allen didn’t, nor did Gabbert. Rosen’s success rate improved over his three seasons, from 44.8 percent as a freshman in 2015, to 46.3 percent in 2016, to a perfectly solid 48.7 percent last year. (above Barrett)

This was never meant to be the answer or one solution to the puzzle, it is  just another piece of information to help a very difficult decision and for that purpose, I think it's good - would be interested to see how Watson scored...

Cheers Bill Connelly (@SBN_BillC) this Bloody Mary is for you, my hangover feels better already

 

[derp]

It’s kind of funny that the article notes Tom Brady very early and then goes on base an entire analysis on the premise that your ceiling as an NFL quarterback is your college stats.

Saying the only quarterbacks better than Mayfield were Bradford and Winston was funny too since Bradford’s bounced around the league (partially to do injuries but you can’t really cite him as a success, at best he’s an unknown) and Winston has lost a lot of his glow.

This guy’s also looking at ten years of quarterback stats when we really don’t know a whole lot about good quarterbacks drafted in the last ten years. Most of the current good ones were drafted before that or are too young to really be established.

The issue with analytics in football is there’s a ton of variance and sample sizes are very small.