Why P < 0.05 Became the Standard Threshold
The p < 0.05 threshold is an arbitrary historical convention, not a scientifically derived absolute—it was originally proposed by Ronald Fisher merely as a suggestion for when evidence against the null hypothesis might warrant attention, but has since been misinterpreted as a rigid rule 1.
Historical Context and Arbitrary Nature
The 0.05 threshold originated from Fisher's proposal that any finding occurring by random variation no more than 1 in 20 times could be considered noteworthy. Critically, this was offered only as a suggestion in its historical context, not as a rigid boundary 1. There is no biological, mathematical, or clinical reason why 0.05 is inherently superior to 0.01,0.001, or even 0.20—the choice is fundamentally arbitrary.
Why Other Thresholds Could Be Used
Different thresholds are actually more appropriate depending on the research context:
For exploratory/discovery studies: A more relaxed threshold like p < 0.20 may be appropriate to avoid missing potentially important findings that require further validation 2. Using p < 0.20 reduces "testimation bias" (exaggeration of effect sizes) to approximately +130%, compared to +220% with p < 0.05 2.
For confirmatory studies with adequate power: The traditional p < 0.05 may be acceptable if the study is properly powered (≥80%) 3.
For high-stakes interventions or genomic studies: Stricter thresholds like p < 0.005 or even p < 5×10⁻⁸ (for genome-wide analyses) reduce false-positive rates 1, 4.
The Problem with Rigid Thresholds
Current guidelines explicitly recommend against dichotomous "significant/non-significant" labeling 5. The American Statistical Association and recent medical journal guidelines call for:
- Reporting exact p-values (e.g., p = 0.03) rather than symbols like "p < 0.05" 6, 5
- Never using the terms "significant" and "non-significant" 7
- Emphasizing effect sizes and confidence intervals over isolated p-values 6, 5
- Recognizing that p-values should be interpreted as graded measures of evidence, not binary outcomes 8
Clinical Reality: The False Security of P < 0.05
When you observe p = 0.05, the actual odds favoring a real effect are only about 3:1—far weaker than the 19:1 that might be wrongly inferred 7. To achieve a false positive risk of just 5% with p = 0.05, you would need to be 87% certain of a real effect before conducting the experiment 7.
Even with p = 0.001 in a well-powered study, the false positive risk could still be 8% if the prior probability of a real effect is only 0.1 7.
Common Pitfalls
- Testimation bias: Selecting effects only after achieving p < 0.05 leads to exaggerated effect estimates (up to +220% bias for true effects around 0.7 times the standard error) 2
- Replication failures: At 80% power, two studies will produce "conflicting" results (one significant, one not) in one-third of cases even when a true effect exists 8
- The "absence of evidence ≠ evidence of absence" error: A p-value above your chosen threshold does not prove no effect exists 6
What Should Be Done Instead
Report exact p-values alongside point estimates and confidence intervals, and interpret them as continuous measures of evidence strength rather than pass/fail thresholds 6, 5. The threshold should be pre-specified and context-dependent, adjusted based on study power, the consequences of false positives versus false negatives, and whether the research is exploratory or confirmatory 3, 4.
For prediction modeling studies specifically, guidelines recommend using a higher threshold like p < 0.20 to avoid strongly biased effect estimates 2.