You've already seen Malcolm Gladwell's letter, with his ad hominem attack on me as a crimethinker. I'd half-assumed that the NYT would cut that part out in the interests of saving space, but they left it in.
From the NYT:
As a commenter pointed out, this debate over NFL quarterbacks is really a stalking horse for the debate over IQ and race, which, in turn, influences practically every other concept about how the world works.Steven Pinker replies:
What Malcolm Gladwell calls a ”lonely ice floe” is what psychologists call ”the mainstream.” In a 1997 editorial in the journal Intelligence, 52 signatories wrote, ”I.Q. is strongly related, probably more so than any other single measurable human trait, to many important educational, occupational, economic and social outcomes.” Similar conclusions were affirmed in a unanimous blue-ribbon report by the American Psychological Association, and in recent studies (some focusing on outliers) by Dean Simonton, David Lubinski and others.
Gladwell is right, of course, to privilege peer-reviewed articles over blogs. But sports is a topic in which any academic must answer to an army of statistics-savvy amateurs, and in this instance, I judged, the bloggers were correct. They noted, among other things, that Berri and Simmons weakened their ”weak correlation” (Gladwell described it in the New Yorker essay reprinted in ”What the Dog Saw” as ”no connection”) by omitting the lower-drafted quarterbacks who, unsurprisingly, turned out not to merit many plays. In any case, the relevance to teacher selection (the focus of the essay) remains tenuous.
For example, if NFL experts can't predict better than random which college quarterback will outperform which in the NFL, then why should we believe that, say, the SAT is any good at predicting who will benefit most from college?
The correlations between draft position and NFL success (0.33 to 0.52) are quite similar to the correlations between, say, SAT score and freshman year in college GPA. Both sets of correlations would be much, much higher if it weren't for restriction of range — e.g., pro quarterbacks are chosen only from college quarterbacks, and Harvard students are people who got into Harvard.
IQ-denialism is the "rotten core" (to use Stephen Jay Gould's phrase in a more accurate context) of the modern conventional wisdom. He who says A must say B, as Lenin liked to say. And Malcolm is naive enough to illustrate that. Gould, for example, wasn't dumb enough to follow his logic in Mismeasure of Man to its conclusions (e.g., he taught at Harvard, which uses IQ-like tests to select Gould's students), but Malcolm, in contrast, is a true believer.
Gladwell's basic problem is that he doesn't understand normal probability distributions.
The NFL quarterback problem is, roughly, this. There are about two million males who turn 22 each year. At, say, four standard deviations above the mean in current quarterbacking ability, there are 63 individuals, which is about the number of starting quarterbacks who run out of eligibility each year from Division I or the better lower division colleges. What NFL teams are doing is looking for the one individual who will turn out to be five standard deviations above the mean — the best NFL quarterback of his age cohort.
Using one single-number measure — Pro-Fooball.Reference.com's Career Approximate Value number — for all the quarterbacks drafted from 1980 through 1999, we see that the first quarterback chosen proved to have the highest Career Approximate Value out of his draft class nine times out of 20. (And the "mistakes" include picking John Elway over Dan Marino; three times the first quarterback chosen proved to have the second best career of his draft cohort.) On average, almost 14 quarterbacks were chosen each year, so being right 45% of the time is a lot better than random.
Moreover, the second quarterback drafted turned out to be the best quarterback of his year five out of 20 times.
To some extent, that measure is biased upward by higher draft picks being handed more playing time. If we use a higher measure of excellence, Pro Bowl selections, then the first quarterback picked wound up with more Pro Bowl honors than anybody else in seven of the 20 drafts, and tied for the most twice (Elway and Marino with 9 each, and in 1980 none of the 17 quarterbacks drafted ever went to a Pro Bowl).
Also, quarterbacks who are undrafted and thus don't show up in the draft database would bias this correlation upward somewhat. To estimate the impact, I checked the careers of four undrafted QBs who are inspiring NFL success stories — Kurt Warner, Jeff Garcia, Jake Delhomme, and Jon Kitna — and their inclusion wouldn't change these results much even if they had been drafted, since they all went undrafted in years in which the first quarterback drafted wasn't the best.
Gladwell's innumeracy shouldn't be such a fatal problem for the articles published under the lucrative Malcolm Gladwell brand name. Many successful authors have research assistants who help the face of the organization concentrate on doing what he does best. For example, I once met the research assistant to the octogenarian crime novelist Elmore Leonard. The assistant's job was to put in the shoe leather work scouting locations, studying old newspapers, interviewing people who have jobs that will feature in the book and so forth, so that Leonard's novels can have very realistic, very detailed senses of time and place.
Similarly, Malcolm could well afford to hire a young research assistant who understands quantitative analysis.
Why doesn't he?
