快猫短视频

Social science and experimental psychology are still shaking from the 鈥渞eplication crisis鈥�, an embarrassing decade that revealed that the results of many classic studies couldn鈥檛 be reproduced when the experiments were repeated. Now research adds to this embarrassment by suggesting a simple solution to the problem: common sense.

There have been many eye-catching, high-profile studies that haven鈥檛 been replicated, and we now know that eating from small plates doesn鈥檛 really make you eat less, and that children who can resist marshmallows don鈥檛 reliably become successful adults.

To understand how 鈥渙bvious鈥� it might have been that studies like these may not stand up to further testing, Alexandra Sarafoglu at the University of Amsterdam, the Netherlands, and her colleagues recruited 233 volunteers. About half of them were first-year psychology students, but none were experts in the field. After reading a quick explanation of what replicating an experiment means, participants were shown 27 short descriptions of well-known findings from social science and psychology.

Of these, 14 had recently been replicated successfully, including a 2008 study that found that students think they are more likely to be questioned in class if they arrive unprepared. Attempts to replicate the other 13 studies hadn鈥檛 worked, including a 2013 study that suggested people are better at recognising emotions after reading a passage of literary fiction.

After reading about each study, participants had to predict whether repeating it was likely to validate its findings. When shown just the description, participants guessed correctly 58 per cent of the time. When they were also shown a simple statistical measure of the strength of the study鈥檚 evidence, this rose to 67 per cent.

The wisdom of crowds

These individual predictions are better than you would expect by chance but pooling them all was even better. When most of the group confidently guessed the same, their prediction was almost always right. Among the 10 most confident predictions that a study wouldn鈥檛 replicate and the 10 most confident that it would, the group was wrong just twice.

Previous studies have shown that psychologists can predict replications. The RepliCATS project at the University of Melbourne in Australia is now gathering expert predictions on 3000 claims from across the social sciences. However this research suggests that, to some extent, a group of laypeople is all the expertise required.

The finding implies that scientists, journals and the media may benefit from not letting eye-catching and surprising findings override their common sense. 鈥淲e are in this crisis for a reason,鈥� Sarafoglu says. 鈥淭here is a strong incentive in science in general to publish sexy findings. So implicitly, people get pushed towards finding effects that are counter-intuitive.鈥�

鈥淚 find it very interesting that laypeople are pretty successful in predicting replication outcomes,鈥� says Anna Dreber Almenberg at the Stockholm School of Economics in Sweden, whose research has found that expert betting markets can also predict replications.

Before conducting their study, Sarafoglu and her colleagues pre-registered it 鈥� an approach that prevents researchers from changing their plan to find positive results in whatever data they collect. Although she accepts that the picture might change a little with a different selection of studies, or with differently written descriptions, she confidently predicts that people will be able to replicate this research.

However, the team hasn鈥檛 yet asked a group of laypeople whether they agree. What do you reckon?

Read more: Work the crowd: How ordinary people can predict the future