Skip to Main Content

Systematic Reviews & Other Syntheses

Introduction

Critical appraisal or risk of bias assessment of eligible studies is an integral component of conducting a systematic review and other types of syntheses. 

Sometimes the terms critical appraisal and risk of bias are used interchangeably as a step in the synthesis process. However, the original conception of critical appraisal (as a step in practicing evidence-based medicine, for example) often goes beyond internal validity into assessment of external validity, generalizability, and even reporting quality (Stone et al., 2023). Global organizations that collaborate to produce high-quality synthesized evidence such as Cochrane, and more recently JBI, use the term “risk of bias” when referring to the assessment of the internal validity of studies included in quantitative syntheses, where bias is considered to be systematic error in the results of quantitative studies (Stone, et al., 2023; Boutron et al., 2022).

Not sure if your review type requires the critical appraisal of studies?
The following table provides a typology of review types that specifically indicates whether or not critical appraisal is generally a component of 14 different reviews types Table 1: Main review types characterized by methods used (Grant & Booth, 2009).

Critical Appraisal and Risk of Bias Tools

Not sure which critical appraisal or risk of bias tool to use? 

Check out the Quality Assessment and Risk of Bias Tool Repository for finding and selecting a risk of bias or quality assessment tool for evidence synthesis projects. This project was created and is maintained by Duke University's Medical Center Library & Archives.

For animal studies: 

Bibliography

Barker, T. H., Stone, J. C., Sears, K., Klugar, M., Tufanaru, C., Leonardi-Bee, J., ... & Munn, Z. (2023). The revised JBI critical appraisal tool for the assessment of risk of bias for randomized controlled trials. JBI Evidence Synthesis, 21(3), 494-506.

Boutron I, Page MJ, Higgins JPT, Altman DG, Lundh A, Hróbjartsson A. Chapter 7: Considering bias and conflicts of interest among the included studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook.

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91-108.

Hooijmans, C. R., Rovers, M. M., de Vries, R., Leenaars, M., Ritskes-Hoitinga, M., & Langendam, M. W. (2014). SYRCLE’s risk of bias tool for animal studies. BMC medical research methodology, 14(1), 1-9.

Stone, J. C., Barker, T. H., Aromataris, E., Ritskes-Hoitinga, M., Sears, K., Klugar, M., ... & Munn, Z. (2023). From critical appraisal to risk of bias assessment: clarifying the terminology for study evaluation in JBI systematic reviews. JBI Evidence Synthesis, 21(3), 472-477.