Deirdre McCloskey has long emphasized the warping effects of the “statistical significance” hurdle to publication in economics. Alan Gerber and Neil Malhotra survey the top two journals in political science to produce this finding:
There are plenty of publications with findings that are barely statistically significant and a noticeable absence of papers that fall just short of the goalline. Figure 2a is more damning.
The goal of this paper is to raise awareness of publication bias in political science. We have found that many more results are published just over the p=.05 threshold than below it, implying a certain amount of bias in parameter estimates. Our results suggest that as reviewers, editors, and researchers, political scientists appear to be far too conscious of the .05 significance level, and that this might cause important distortions in how knowledge advances in political science.
Full paper here.