Making It All Up

The cover story in the October 19, 2015 issue of The Weekly Standard is titled "Making It All Up." It is about the state of science in the behavioral sciences. It's not pretty, as you can guess from the title of the article.

The article discusses an August report in the widely-respected journal Science titled "Estimating the reproducibility of psychological science" (link). Scientists attempted to reproduce the results of 100 studies published in three psychology journals:

The Reproducibility Project: Psychology was a collaborative, crowdsourced effort of 270 authors and 86 additional volunteers. Across multiple criteria, we successfully reproduced fewer than half of the 100 original findings investigated. (link)

Fewer than half! This is bad science.

Why does this matter? One reason is that bad science contributes to mistaken ideas about the human condition. Another reason is that bad science undermines public support for and trust in science itself.

The problem is not just these 100 relatively current studies. The article explores the weaknesses of several landmark studies in psychology including the famous Stanley Milgram electric shock experiments conducted at Yale in the 1960s.

Another famous psychology study, not mentioned in The Weekly Standard article, was the Stanford Prison Experiment conducted by Philip Zimbardo in the 1970s and the subject of a 2015 movie by that name. For a good discussion of the weaknesses of that study see: Why Zimbardo's Prison Experiment Isn't in My Textbook.

Beware the findings of social scientists!

The gold standard of what it means to "know something" in science is still Feynman's 1974 commencement address at Caltech: Cargo Cult Science. I have seen numerous references to this theme lately in widely different fields. Search for "Feynman integrity" on Google to see some examples.