Publication bias

In published academic research, publication bias occurs when the outcome of an experiment or research study biases the decision to publish or otherwise distribute it. Publishing only results that show a significant finding disturbs the balance of findings in favor of positive results.^[1] The study of publication bias is an important topic in metascience.

Despite similar quality of execution and design,^[2] papers with statistically significant results are three times more likely to be published than those with null results.^[3] This unduly motivates researchers to manipulate their practices to ensure statistically significant results, such as by data dredging.^[4]

Many factors contribute to publication bias.^[5][6] For instance, once a scientific finding is well established, it may become newsworthy to publish reliable papers that fail to reject the null hypothesis.^[7] Most commonly, investigators simply decline to submit results, leading to non-response bias. Investigators may also assume they made a mistake, find that the null result fails to support a known finding, lose interest in the topic, or anticipate that others will be uninterested in the null results.^[2] The nature of these issues and the resulting problems form the five diseases that threaten science: "significosis, an inordinate focus on statistically significant results; neophilia, an excessive appreciation for novelty; theorrhea, a mania for new theory; arigorium, a deficiency of rigor in theoretical and empirical work; and finally, disjunctivitis, a proclivity to produce many redundant, trivial, and incoherent works."^[8]

Attempts to find unpublished studies often prove difficult or are unsatisfactory.^[5] In an effort to combat this problem, some journals require studies submitted for publication pre-register (before data collection and analysis) with organizations like the Center for Open Science.

Other proposed strategies to detect and control for publication bias^[5] include p-curve analysis^[9] and disfavoring small and non-randomized studies due to high susceptibility to error and bias.^[2]