How do you score a systematic review using the Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Systematic Reviews?

How to Score a Systematic Review Using the JBI Critical Appraisal Checklist

Use the JBI Critical Appraisal Checklist for Systematic Reviews, which contains 11 questions that assess specific methodological domains, and score each item as "yes" (1 point), "no" (0 points), "unclear" (0 points), or "not applicable" (excluded from scoring). 1

Scoring Methodology

The standardized approach involves:

• Calculate the final score by adding all "yes" responses and dividing by the total number of applicable items (out of 11) 1
• Express the result as a percentage
• Two independent reviewers should conduct the assessment, with disagreements resolved through discussion and consensus 1, 2

Risk of Bias Classification

Apply these cutoff thresholds to categorize study quality:

• Low risk of bias: ≥70% completion 1, 2
• Moderate risk of bias: 50-69% completion 1, 2
• High risk of bias: <50% completion 1, 2

Some variations exist in the literature—one study used slightly different thresholds (≥80% for low risk, 60-80% for moderate, <60% for high) 2, but the 70%/50% cutoffs are more consistently applied across recent guidelines 1.

Key Domains Assessed by the 11-Item Checklist

While the specific questions aren't fully detailed in the evidence, the checklist evaluates:

• Clear research question/objectives
• Appropriate inclusion criteria
• Search strategy comprehensiveness
• Study selection process
• Critical appraisal of included studies (Item 5 is specifically mentioned as frequently problematic) 3
• Data extraction methods
• Methods for combining/synthesizing results
• Assessment of publication bias
• Recommendations for policy/practice
• Appropriate directives for future research
• Disclosure of conflicts of interest

Practical Implementation

Follow this workflow:

1. Assign two independent reviewers to assess the same systematic review 1, 2
2. Each reviewer scores all 11 items independently
3. Calculate individual percentage scores (sum of "yes" responses ÷ number of applicable items × 100)
4. Compare scores between reviewers
5. Discuss any discrepancies until consensus is reached 1
6. If consensus cannot be achieved, involve the broader research team 1
7. Apply the risk of bias classification based on final percentage

Common Pitfalls

Be aware that certain items consistently show poor completion rates across systematic reviews:

• Item 5 (critical appraisal conduct) is frequently inadequately addressed 3
• Protocol registration and pre-established methods (Item 2 in AMSTAR-2 equivalent) often score poorly 4
• List of excluded studies with justifications is commonly missing 4

Recent evidence shows that even published systematic reviews in high-quality journals demonstrate generally low methodological quality, with median scores of only 2-3 items met out of 9 assessed 3. This underscores the importance of rigorous critical appraisal rather than assuming quality based on publication venue.

Reporting the Results

Present the risk of bias assessment results clearly in your systematic review, as this allows end users to make informed decisions about the reliability of the evidence 5. The percentage score and corresponding risk category should be reported for each included systematic review, and these assessments should inform your confidence in the synthesized findings.