Gladwellian Implicit Association Testing Is Just Another Example of Psychology's Replication Crisis
Print Friendly and PDF

Over the last decade, the Establishment has celebrated the Implicit Association Test as a way to scientifically uncover those who are hiding Implicit Bias so they can be put through the re-education process. But does it really work?

From the Scandinavian Journal of Psychology:

A closer look at the discrimination outcomes in the IAT literature

Rickard Carlsson, Jens Agerström First published: 24 April 2016


To what extent the IAT (Implicit Association Test, Greenwald et al., 1998) predicts racial and ethnic discrimination is a heavily debated issue. The latest meta-analysis by Oswald et al. (2013) suggests a very weak association. In the present meta-analysis, we switched the focus from the predictor to the criterion, by taking a closer look at the discrimination outcomes. We discovered that many of these outcomes were not actually operationalizations of discrimination, but rather of other related, but distinct, concepts, such as brain activity and voting preferences. When we meta-analyzed the main effects of discrimination among the remaining discrimination outcomes, the overall effect was close to zero and highly inconsistent across studies. Taken together, it is doubtful whether the amalgamation of these outcomes is relevant criteria for assessing the IAT’s predictive validity of discrimination. Accordingly, there is also little evidence that the IAT can meaningfully predict discrimination, and we thus strongly caution against any practical applications of the IAT that rest on this assumption. However, provided that the application is thoroughly informed by the current state of the literature, we believe the IAT can still be a useful tool for researchers, educators, managers, and students who are interested in attitudes, prejudices, stereotypes, and discrimination.

Also, the Implicit Bias test turns out to be explicitly biased to find more bias than actually exists. From the Journal of Applied Psychology in 2015:
Toward a Meaningful Metric of Implicit Prejudice

Hart Blanton, University of Connecticut, James Jaccard, New York University, Erin Strauts, University of Connecticut, Gregory Mitchell, University of Virginia, Philip E. Tetlock, University of Pennsylvania

The modal distribution of the Implicit Association Test (IAT) is commonly interpreted as showing high levels of implicit prejudice among Americans. These interpretations have fueled calls for changes in organizational and legal practices, but such applications are problematic because the IAT is scored on an arbitrary psychological metric. The present research was designed to make the IAT metric less arbitrary by determining the scores on IAT measures that are associated with observable racial or ethnic bias. By reexamining data from published studies, we found evidence that the IAT metric is “right biased,” such that individuals who are behaviorally neutral tend to have positive IAT scores. Current scoring conventions fail to take into account these dynamics and can lead to faulty inferences about the prevalence of implicit prejudice.

Since passage of landmark civil rights legislation in the 1960s,survey researchers have observed steady declines in overt racism and other forms of prejudice (Carmines, Sniderman, & Easter,2011; Schuman, Steeh, Bobo, & Krysan, 1997). Many researchers have been skeptical of these trends and have pointed to the pervasiveness of inequality in American society as evidence that hidden forms of prejudice must be alive and well (e.g., Dovidio & Gaertner, 2010). Recently, psychologists have drawn on the distinction between explicit and implicit attitudes to develop methods of capturing sentiments that self-report-based surveys allegedly fail to detect, with several creative measures of implicit attitudes having been developed over the past 15 years (see Petty, Fazio, & Briñol, 2008; Wittenbrink & Schwarz, 2007). These measures yield response patterns that are widely interpreted as showing pervasive implicit biases (a term that we use to encapsulate the hidden forms of implicit prejudicial associations that ostensibly aredetected by these measures).

By far, the most influential of these new measures is the Implicit Association Test (IAT). Modal distributions of IAT scores— especially from data collected from millions of administrations of the test over public web-sites—suggest that survey researchers have substantially under-estimated contemporary levels of prejudice (e.g., Nosek et al.,2007), which has led to calls for legal and organizational reforms to address the high levels of implicit racial biases revealed by this measurement advance (e.g., Ayres, 2001; Bennett, 2010; Levinson & Smith, 2012; Saujani, 2003; see Banaji & Greenwald, 2013).Such applications of IAT test scores are problematic, however,because the IAT relies on an arbitrary metric: … Our analyses offer evidence that the zero point on the IAT fails to map onto behavioral racial neutrality and that researchers’ tendency to treat the zero point as valid likely leads to significant overestimation of the degree of implicit bias in populations.

It’s kind of like how Scientologists’ E-meters tend to overestimate Thetans.

[Comment at]
Print Friendly and PDF