Skip to main content

Systematic Reviews & Other Syntheses

Introduction

A systematic review attempts to identify, select, synthesize, and appraise all evidence for a given research question in a systematic and transparent manner.

 

Other introductory videos and modules:

Systematic Review Steps

Eight stages of a systematic review and meta-analysis* (Uman, 2011):

  1. Formulate the review question
  2. Define inclusion/exclusion criteria
  3. Develop reproducible search strategies and locate studies
  4. Select studies
  5. Extract data
  6. Assess study quality
  7. Analyze and interpret results
  8. Disseminate findings

*Important: before embarking on a systematic review, make sure that 1) a recent review on the same topic has not already been published, and 2) that a review protocol has not already been registered for the same topic. 

  1. To check for systematic reviews that have already been published on your topic, search databases such as Ovid MEDLINE/PubMed, Embase, PsycINFO, CINAHL, and even Google Scholar to catch any journal content that is not covered in these databases. You can also search Epistemonikos for published systematic reviews, a freely available resource that attempts to identify all of the systematic reviews relevant for health decision-making.
     
  2. To check for registered (i.e. forthcoming) systematic reviews, search PROSPERO, the international prospective register of systematic reviews. You can also search for registered protocols by systematic review collaborations such as Cochrane, Campbell and the Joanna Briggs Institute. When you are confident that your research will not be duplicating efforts, you are ready to begin your own review.

Systematic Review Types

Most often the type of systematic review conducted will depend on the review question:

- Systematic reviews of healthcare interventions are often quantitative reviews that include research studies in the form of randomized controlled trials.
- Systematic reviews of patient experiences are generally qualitative reviews that may include qualitative and mixed methods research studies.
- To address the limitations of single method reviews, which are frequently too narrow in scope, mixed methods reviews are utilized: "By including diverse forms of evidence from different types of research, mixed methods reviews attempt to maximize the findings - and the ability of those findings to inform policy and practice" (Peters et al., 2015, p. 5).

Cochrane describes the five types of Cochrane Reviews:

  1. Intervention reviews assess the benefits and harms of interventions used in healthcare and health policy
  2. Diagnostic test accuracy reviews assess how well a diagnostic test performs in diagnosing and detecting a particular disease
  3. Methodology reviews address issues relevant to how systematic reviews and clinical trials are conducted and reported
  4. Qualitative reviews synthesize qualitative evidence to address questions on aspects other than effectiveness
  5. Prognosis reviews address the probable course or future outcome(s) of people with a health problem

Systematic Review Time Frame

Some workload considerations:

  • Research topic: The review topic will  impact the amount of time required to conduct the review. For example, a systematic review on the effectiveness of a relatively new drug therapy may require less time to complete if the search yields a low number of results to screen and if the review itself only includes a small number of studies.
  • Searching for studies: The amount of time needed to complete this component of the review will depend on: how straight-forward or complex the topic is, the number and type of resources that will be searched (refer to Where to Search), and the searcher's level of expertise. Working with an experienced librarian is a best practice recommendation (refer to Librarian Support).
  • Research team: Estimating how long it will take to complete a systematic review will depend on the size of the research team and how available the team members are to work on the review. Researchers often have competing priorities throughout the duration of the systematic review that can delay the process.

Research evidence:

  • The average length of time for systematic review projects (using the project start date from the registered protocol in PROSPERO to the review's publication date) was 67.3 weeks (Borah, 2017).
     
  • A research study found that the median time spent searching was just under 8 hours, including 1.5 hours spent searching the grey literature, while the average time spent searching was approximately 24 hours, of which 6.5 hours were spent searching the grey literature (Saleh et al., 2014). The average number of resources searched was 9, including grey literature resources. Grant funding influenced the amount of time spent searching and institution type impacted the number of resources searched.
     
  • Greenhalgh and Peacock (2005) found that: "Electronic searching, including developing and refining search strategies and adapting these to different databases, took about two weeks of specialist librarian time..." (p1065).

Bibliography

Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A. (2017). Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ open, 7(2), e012545.
 
Cochrane. (n.d.). About Cochrane reviews. Retrieved May 11, 2015, from http://www.cochranelibrary.com/about/about-cochrane-systematic-reviews.html

Couban, R. (2018). Epistemonikos. Journal of the Canadian Health Libraries Association / Journal De L’Association Des Bibliothèques De La Santé Du Canada, 39(3), 155-157. https://doi.org/10.29173/jchla29390

Crombie I.K. & Davies H.T.O. (2009). What is meta-analysis? London: Hayward Medical Communications.
 

Gough, D., Oliver, S., & Thomas, J. (2018). Systematic reviews and research. Los Angeles: SAGE reference.

Greenhalgh, T. & Peacock, R. (2005). Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ: British Medical Journal, 331(7524), 1064-1065.

Hemingway, P. & Brereton N. (2009). What is a systematic review? 2nd ed. London, UK: Hayward Medical Communications.

Higgins, J.P.T. & Green, S. (Eds.). (2011). Cochrane Handbook for Systematic Reviews of Interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Available from www.handbook.cochrane.org.

Higgins J.P.T, Thomas J., Chandler J., Cumpston M., Li T., Page M.J. & Welch V.A. (Eds.). (2019). Cochrane Handbook for Systematic Reviews of Interventions version 6.0. Cochrane. Available from www.training.cochrane.org/handbook.

Institute of Medicine. (2011). Finding what works in health care: standards for systematic reviews. Washington, DC: The National Academies Press.

Koffel, J. B. (2015). Use of recommended search strategies in systematic reviews and the impact of librarian involvement: A cross-sectional survey of recent authors. PLoS ONE, 10(5), 1–13.

McGowan, J., & Sampson, M. (2005). Systematic reviews need systematic searchers. Journal of the Medical Library Association, 93(1), 74.

Meert, D., Torabi, N., & Costella, J. (2016). Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. Journal of the Medical Library Association : JMLA, 104(4), 267–277. https://doi.org/10.3163/1536-5050.104.4.004

Pearson, A., White, H., Bath-Hextall, F., Apostolo, J., Salmond, S., & Kirkpatrick, P. (2014). Methodology for JBI mixed methods systematic reviews. The Joanna Briggs Institute Reviewers’ Manual

Petticrew, M. (2001). Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ: British Medical Journal, 322(7278), 98.

Rethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C., & Brigham, T. J. (2015). Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. Journal of Clinical Epidemiology, 68(6), 617–626. https://doi.org/10.1016/j.jclinepi.2014.11.025

Saleh, A. A., Ratajeski, M. A., & Bertolet, M. (2014). Grey literature searching for health sciences systematic reviews: a prospective study of time spent and resources utilized. Evidence Based Library and Information Practice, 9(3), 28-50.

Uman, L. S. (2011). Systematic reviews and meta-analyses. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 20(1), 57.

Educational Research

Conducting a systematic review in educational research?

Check out the systematic review library guide created by the Institute of Education.