What are the biases in the systematic review and meta‑analysis on shift‑work disorder among nurses?

Biases in Systematic Reviews and Meta-Analyses on Shift-Work Disorder Among Nurses

The primary biases affecting systematic reviews and meta-analyses of shift-work disorder among nurses include publication bias (selective reporting of statistically significant results), heterogeneity in exposure assessment methods, inconsistent outcome definitions across studies, and residual confounding from unmeasured variables.

Key Sources of Bias

Publication and Selective Reporting Bias

Publication bias represents the most serious threat to validity in these reviews. Studies with statistically significant findings (P<0.05) are 81% likely to be published within five years, compared to only 68% of non-significant studies 1, 2, 1. This creates systematic overestimation of effect sizes. In the context of shift-work disorder research, this means:

• Smaller studies showing harmful effects of shift work are more likely to be published than those showing no effect
• Negative or null findings remain unpublished, creating an incomplete evidence base
• The pooled prevalence of 45.5% in nurses 3 may be inflated due to missing negative studies

Selective outcome reporting within published studies compounds this problem. Studies may measure multiple gastrointestinal, musculoskeletal, or sleep outcomes but only report those with significant results 1. Evidence shows that 38% of efficacy outcomes and 50% of safety outcomes per trial are not available for meta-analysis, with statistically significant outcomes having higher odds of being fully reported 1.

Exposure Assessment Heterogeneity

The definition and measurement of "shift work" varies dramatically across studies, introducing substantial bias 4. The IARC evaluation noted that:

• Early studies lacked consistent definitions of shift work exposure
• Control groups had inconsistent and limited exposure assessment to light at night
• Better exposure assessment methods emerged primarily in case-control studies rather than cohort studies 4

This heterogeneity directly affects prevalence estimates. The systematic review on shift-work disorder found that assessment tools themselves introduce bias: scales and questionnaires yielded 54.2% prevalence, while ICSD-3 criteria showed 40.2% and ICSD-2 showed 32.9% 3. This 21-percentage-point difference reflects measurement bias rather than true prevalence variation.

Residual Confounding

Unmeasured or inadequately controlled confounders represent a critical source of bias 5. In shift-work research among nurses, key confounders include:

• Baseline mental health status (anxiety, depression) which both predicts shift work selection and outcomes
• Socioeconomic factors influencing who works night shifts
• Lifestyle behaviors (diet, exercise, substance use) that cluster with shift patterns
• Pre-existing health conditions affecting shift assignment

The IARC evaluation specifically noted "concerns regarding potential residual confounding" in shift work studies 4. Given the heterogeneity of findings, bias could not be excluded with reasonable confidence 4.

Study Design Limitations

Most included studies are cross-sectional, preventing causal inference 3, 6, 7. This introduces:

• Reverse causation bias: Nurses with health problems may self-select out of rotating shifts, making day shift workers appear less healthy
• Healthy worker effect: Nurses unable to tolerate shift work leave the profession entirely, leaving a survivor cohort
• Recall bias: Retrospective reporting of shift patterns and symptoms

The meta-analysis on musculoskeletal disorders found no significant associations (OR=1.15,95% CI: 0.98-1.35 for rotating shifts) 6, while gastrointestinal disorders showed modest associations (OR=1.15,95% CI: 1.02-1.30) 7. These inconsistent findings across similar populations suggest unmeasured confounding and methodological heterogeneity rather than true biological differences.

Statistical Heterogeneity and Its Implications

High statistical heterogeneity (I² values) indicates substantial between-study variation that may reflect bias rather than true effect variation 1, 2, 1. The PRISMA guidelines emphasize that:

• I² >50% represents moderate to substantial heterogeneity requiring investigation
• Data extraction errors commonly cause heterogeneity in continuous outcomes
• Heterogeneity may reflect differences in study quality, populations, or unmeasured biases 1

The shift-work disorder review found substantial heterogeneity across studies 3, while the musculoskeletal pain meta-analysis showed I²=81.1% for fixed night shifts 6. This level of heterogeneity suggests that pooled estimates may be misleading and that bias varies substantially across included studies.

Geographic and Population Selection Bias

Regional variation in prevalence (Africa 51.0%, Asia 46.5%) 3 likely reflects selection bias and differential study quality rather than true geographic differences. Studies from regions with:

• Less rigorous research infrastructure may have higher risk of bias
• Different healthcare systems may select different nurse populations
• Varying cultural attitudes toward shift work affect reporting

Missing Data and Incomplete Reporting

The absence of study protocols prevents assessment of selective outcome reporting 1, 2, 1. Without pre-registered protocols, reviewers cannot determine:

• Which outcomes were measured but not reported
• Whether analyses were pre-specified or data-driven
• If subgroup analyses represent hypothesis testing or data dredging

Critical Methodological Pitfalls

Common pitfalls that amplify bias in these reviews include:

1. Combining studies with incompatible shift definitions (e.g., "three-shift" model showing 61.7% prevalence 3 pooled with other schedules)

2. Failure to stratify by study design quality - cross-sectional studies should not be weighted equally with prospective cohorts 5

3. Inadequate assessment of temporal relationships - most studies cannot distinguish whether shift work causes disorders or disorders lead to shift selection

4. Language and database restrictions creating geographic bias toward English-speaking, Western populations

5. Small study bias - funnel plot asymmetry may reflect genuine bias or simply heterogeneity in smaller studies 2, 8

Implications for Interpretation

Given these substantial biases, the true prevalence and causal effects of shift work on nurse health outcomes remain uncertain. The pooled estimates should be interpreted as upper bounds rather than precise point estimates. The evidence is strongest for:

• Irritable bowel syndrome (OR=1.74,95% CI: 1.27-2.37, I²=0%) 7 - low heterogeneity suggests more robust finding
• Shift-work disorder prevalence being elevated, though exact magnitude uncertain 3

The evidence is weakest and most bias-prone for:

• Musculoskeletal outcomes showing null results despite biological plausibility 6
• Long-term outcomes like cancer where confounding is most problematic 4
• Comparisons across different shift patterns where exposure misclassification is highest

Future research must address these biases through prospective cohort designs, standardized exposure assessment, pre-registered protocols, and individual participant data meta-analyses 5.