Education Statistics: What Have We Learned?
March 26, 2010, 10:34 PM
A+
|
a-
Print Friendly and PDF
I've been following education statistics since the summer before my freshman year in high school when I started preparing for the 1972-73 debate topic:
Resolved: That governmental financial support for all public and secondary education in the United States be provided exclusively by the federal government.
What has been learned over those 38 years?

By way of contrast, I'd like to cite what Bill James learned about baseball in his first dozen years of statistical research:

A Bill James Primer Extracted from The Bill James Baseball Abstract 1988

"What I wanted to write about... is a very basic question. Of all the studies I have done over the last 12 years, what have I learned? What is the relevance of sabermetric knowledge to the decision making process of a team? If I were employed by a major-league team, what are the basic things that I know from the research I have done which would be of use to me in helping that team?"

1. Minor league batting statistics will predict major league batting performance with essentially the same reliability as previous major league statistics. 2. Talent in baseball is not normally distributed. It is a pyramid. For every player who is 10 percent above the average player, there are probably twenty players who are 10 pecent below average. 3. What a player hits in one ballpark may be radically different from what he would hit in another. 4. Ballplayers, as a group, reach their peak value much earlier and decline much more rapidly than people believe. 5. Players taken in the June draft coming out of college (or with at least two years of college) perform dramatically better than players drafted out of high school. 6. The chance of getting a good player with a high draft pick is substantial enough that it is clearly a disastrous strategy to give up a first round draft choice to sign a mediocre free agent. (see note #1) 7. A power pitcher has a dramatically higher expectation for future wins than does a finesse pitcher of the same age and ability. 8. Single season won-lost records have almost no value as an indicator of a pitcher's contribution to a team. 9. The largest variable determining how many runs a team will score is how many times they get their leadoff man on base. 10. A great deal of what is perceived as being pitching is in fact defense. 11. True shortage of talent almost never occurs at the left end of the defensive spectrum. (see note #2) 12. Rightward shifts along the defensive spectrum almost never work. (see note #2) 13. Our idea of what makes a team good on artificial turf is not supported by any research. 14. When a team improves sharply one season they will almost always decline in the next. 15. The platoon differential is real and virtually universal

Notes:

1. Major league teams still must surrender choices in the amateur draft in exchange for signing free agents. 2. The defensive spectrum looks like this: [ - - 1B - LF - RF - 3B - CF - 2B - SS - C - - ] with the basic premise being that positions at the right end of the spectrum are more difficult than the positions at the left end of the spectrum. Players can generally move from right to left along the spectrum successfully during their careers.

It took James another decade and a half to get that baseball job he was advertising for here, but this was pretty decent start.

So, what have we learned from school statistics?

The overwhelming finding, going back to James S. Coleman's 1966 report funded by the Civil Rights Act of 1964, is that race matters in school performance. Persistently large racial gaps are the single most obvious fact about educational performance.

But making the racial gaps go away is also the highest priority of educational research, which debilitates the research. Wishful thinking is preferred.

But, once we adjust for race, what have we learned over the years about what works in education? Can we make up a list James made up for baseball?