There are a lot of problems with it:
- Almost nobody knows how to calculate it. Wikipedia has the formulas here. Hence, nobody knows what a good number is. The NFL average is typically in the low to mid-80s. A rule of thumb is that 100 is excellent.
- Because it includes touchdown passes v. interceptions, which are inherently small sample sizes, it's not terribly stable from year to year. For example, according to passer rating, the sixth best single season of all time was Milt Plum's in 1960 when he threw 21 touchdowns v. only 5 interceptions. Plum was a good quarterback, but that year's passer rating was anomalous. Luck plays a sizable role in a single season's touchdowns and interceptions. Nonetheless, when you aggregate over multiple seasons or across multiple players, the larger sample sizes make the emphasis on touchdown passes and interceptions more reliable. After all, there is a pretty high correlation between them and winning.
- Passer Ratings is a "Pedro Martinez statistic" in that it doesn't give credit for durability. A brilliant but fragile pitcher like Pedro Martinez looks very good in sophisticated statistics like ERA but not quite so great in simple counting statistics like career wins. Similarly, guys who played forever like Dan Marino, Brett Farve, and John Elway don't look quite as good as the guys with shorter careers like Steve Young and Jeff Garcia. (Garcia is an interesting case in that he was a career journeyman — his statistics might be inflated because he would get plugged in when the situation was ripe for his particular talents and benched when they weren't.) Similarly, Daunte Culpepper, is ranked 11th all-time in career passer rating, but has only managed to start about 9 games per season on average.
On the other hand, passer rating has advantages over simpler qb statistics like yards passing. If you rush for 150 yards per game, you are probably better than somebody who rushes for 100 yards per game, because rushing is debilitating, so there are diminishing marginal returns. On the other hand if you pass for 300 yards per game, you aren't necessarily better than somebody who passes for 200 yards per game. Throwing a football isn't that tiring. Also, although it's less true in today's passing-oriented offenses, but in the past, the big yardages were typically wracked up by quarterbacks who fell behind early and had to mount desperate comeback attempts, often unsuccessful.
Then there are the problems with all current quarterback statistics:
- Quarterbacks are much more dependent on their receivers, running backs, and offensive lines than, say, baseball pitchers are on their teammates, so San Francisco quarterbacks throwing to Jerry Rice, for example, will rate better than guys throwing to people who weren't Jerry Rice.
- There's no home field adjustment like in modern baseball statistics. This goes to the heart of the long-running Peyton Manning vs. Tom Brady dispute. Manning usually has better passer ratings, but he plays his eight home games per season indoors, while Brady plays outdoors in suburban Boston.
Nonetheless, I think history has largely vindicated passer rating. Why? Notice that 12 of the top 15 quarterbacks of all time in career passer rating are currently active. This suggests that it measures pretty accurately the direction that improvements in offensive play are taking football in order to win games.
Other statistics have been invented. For example, Adjusted Net Yards per Pass Attempt looks like a pretty good single number stat based on yards per attempt: It dispenses with completion percentage and augments yards per attempt by adding 20 yards per touchdown pass and subtracting 45 yards per interception. It also subtracts yards lost when sacked.
This has the advantage of making the statistical differences more comprehensible than passer rating. For example, over his career, Peyton Manning's team has averaged a 7.13 yard gain everytime he passed (or was sacked), while his younger brother Eli Manning only averages 5.39 yards. So, if both brothers drop back to throw 30 passes per game, Peyton's team will gain 214 yards and Eli's team only 162 yards. I would guess that Peyton's extra 52 yards is worth somewhere between a field goal and a touchdown more points per game, plus it keeps the other team's offense off the field a little more.
Yet, the results for ANY/A are pretty much the same as for passer rating, though. Steve Young drops from #1 to #3 for career, and Peyton Manning moves up to #1 with 7.13 yards per attempt. (Young doesn't get credit for his rushing yardage, which seems a shame, but that's a tricky thing to account for because a lot of quarterbacks rush mostly on 3rd and 1 quarterback sneaks. If they average 2 yards per carry, they're doing fine, but including that would lower their overall yards per play average. I mean, you don't want to penalize the QBs who are good at sneaks worse than the ones who are no good at sneaks and thus never attempt them. On the other hand, it would be nice to find a way to credit quarterbacks who were genuine rushing threats like Young and Michael Vick.)
The top of the ANY/A list continues to be dominated by active players, with Young, Joe Montana, and Dan Marino the only old-timers in the Top 10.
Among the 29 active quarterbacks on the all-time Adjusted Net Yards per Pass Attempt career list, Culpepper is the highest ranking black quarterback at 12th with an average of 5.97 yards per attempt, Donovan McNabb is at 15th, David Garrard at 16th, Byron Leftwich at 20th, Jason Campbell at 21st, and Michael Vick at 26th (4.91). Some of these guys were good to very good runners, especially when they were younger, so they were probably a little more effective overall than this ranking suggests.
Overall, not too bad, but not at all "the future of football," as was widely hyped as recently as a few years ago.