302G Wyss House
Harvard Business School
Boston, MA 02163
Institutional Affiliation: Harvard University
NBER Working Papers and Publications
|September 2018||Sniff Tests in Economics: Aggregate Distribution of Their Probability Values and Implications for Publication Bias|
with Christopher Snyder: w25058
The increasing demand for rigor in empirical economics has led to the growing use of auxiliary tests (balance, specification, over-identification, placebo, etc.) supporting the credibility of a paper's main results. We dub these "sniff tests" because standards for passing are subjective and rejection is bad news for the author. Sniff tests offer a new window into publication bias since authors prefer them to be insignificant, the reverse of standard statistical tests. Collecting a sample of nearly 30,000 sniff tests across 60 economics journals, we provide the first estimate of their aggregate probability-value (p-value) distribution. For the subsample of balance tests in randomized controlled trials (for which the distribution of p-values is known to be uniform absent publication bias, al...