Speakers
Description
Effect inflation and non-reproducibility of exploratory effects plague empirical sciences in general and Psychological Science in particular. It has been argued, that the resulting biases should lead to a larger prevalence of declining effects over time compared to increasing effects, regardless of the investigated research question. These expectations are supported by evidence about systematically more prevalent and stronger decline effects than others in intelligence research. Here, we introduce the lightning plot, a novel method for meta-meta-analytically visualizing cross-temporal effect changes (https://the-meta-analysis-project.shinyapps.io/lightning-plot/). In lightning plots, initial study effects of a given meta-analyses are used as reference for annual effect changes. Specifically, relative annual effect changes that are reported in replication studies are cumulatively added thus illustrating overproportional effect declines or increases in the pattern of effect change trajectories. We demonstrate the features of the lightning plot on a large data set of meta-analyses (k = 500+; N = 280,000,000+) that have been published in five flagship journals in Psychological Science.
