Recently, people have wondered why researchers seem to have a special interest in replicating studies that demonstrated unexpected or surprising results. In this blog post, I will explain why, statistically speaking, this makes sense. When we evaluate the likelihood that findings reflect real effects, we need to take the prior likelihood that the null-hypothesis is true into account.

I recently read a meta-analysis on precognition studies by Bem, Tressoldi, Rabeyron, and Duggan (available on SSRN). The authors conclude in the abstract: 'We can now report a metaanalysis of 90 experiments from 33 laboratories in 14 different countries which yielded an overall positive effect in excess of 6 sigma with an effect size (Hedges’ g) of 0.09, combined z = 6.33, p = 1.2 ×10^-10). A Bayesian analysis yielded a Bayes Factor of 1.24 ×

It’s important to communicate any remaining uncertainty in the results of an experiment.

‘The last couple of years it has been repeatedly pointed out that it is essential to perform replication studies.’ This is the first line of a Dutch article by Annie van Bergen, written in 1963 for the Nederlands Tijdschrift voor de Psychologie en Haar Grensgebieden (Dutch Journal for Psychology and her Border Areas). I first heard about it one month ago, when a group of researchers were giving short presentations about the value of replication