To the Editor:
The recent essay by Michael Jindra and Arthur Sakamoto (“When Ideology Drives Social Science,” The Chronicle Review, March 6), was remarkably naïve and ill-informed. Excellent and responsible critiques of the use of statistical procedures in the social sciences have been available at least since the 1960s. Examples in economics include work by Edward Leamer (“Let’s Take the Con Out of Econometrics”), David Hendrey (“Econometrics: Alchemy or Science?”), and Guido Imbens (“Better LATE than Nothing”). Examples in sociology include work by Stanley Lieberson (“Making it Count”), David Freedman (“Statistical Models and Shoe Leather”), and myself (“Regression Analysis: A Constructive Critique”). Examples in psychology include work by Valen Johnson and colleagues (“On the Reproducibility of Psychological Science”), Phillip Stark (“Before Reproducibility Must Come Preproducibility”), and Richard Darlington (“Multiple Regression in Psychological Research and Practice”). There is no evidence that Jindra and Sakamoto have done their homework. Parenthetically, critiques of statistical practice in the natural sciences easily can be found, such as by Blake McShane and Abraham Wyner (“A Statistical Analysis of Multiple Temperature Proxies: Are Reconstructions of Surface Temperatures over the Last 1000 Years Reliable?”).
There can be at least three explanations for statistical malpractice: evil, incompetence, and a healthy evolution in statistical methods. Jindra and Sakamoto apparently focus on evil, especially evil driven by ideology and self-serving professional advancement. But as they remind us over and over, causal inference can be very challenging. Yet, we are provided with a clumsy causal narrative delivered with striking gusto. Narratives making causal claims require close scrutiny too.
Perhaps more important, Jindra and Sakamoto seem confused about the practice of statistics. There is almost always a need for judgment and subject-matter expertise. There is also room for talented and informed researchers to make different discretionary decisions. As long as those decisions are well documented and easily accessible, the resulting science or policy can properly proceed with a healthy give and take. Problems arise when researchers hide the ball. But here too, is it evil, incompetence, or an evolution of statistical methods?
Finally, commentaries such as “When Ideology Drives Social Science” can easily be misunderstood as a critique of all science at a time when politically motivated individuals are prepared to pounce. The Chronicle needs to do a better job weeding out irresponsible and poorly informed contributions.
Richard Berk
Emeritus Professor of Criminology and Statistics
University of Pennsylvania
