Systematic Reviews & Other Syntheses

Searching multiple databases for studies may retrieve thousands of search results and will inevitably result in retrieving duplicate references. ​Removing duplicate references before screening will expedite the process.

When you import search results from different databases into Covidence, it will automatically identify and remove duplicate references effectively [1]. 

•  The first stage of screening involves examining the titles and abstracts of records from the search for study eligibility. Irrelevant records are excluded and potentially relevant records move on to the next stage of screening.
• The second stage of screening involves examining the full-text reports (of records that were included in the first stage) for study eligibility.

Best practice is to have more than one reviewer screen studies for eligibility in both stages:

Having more than one reviewer independently assess citations for inclusion is one method of reducing the risk of biased decisions on study inclusion [2].

As the Institute of Medicine explains “[e]ven when the selection criteria are prespecified and explicit, decisions on including particular studies can be subjective" [3; p. 110].

Research evidence:

A research study found that using a complete dual review approach, where two reviewers screen at both stages, identified additional eligible studies at both the title/abstract and the full-text stage [4].

Guidelines:

Cochrane Handbook [5]:

4.6.4 Implementation of the selection process: "Decisions about which studies to include in a review are among the most influential decisions that are made in the review process and they involve judgement. Use (at least) two people working independently to determine whether each study meets the eligibility criteria."

Agency for Healthcare Research and Quality [2]:

"Even with clear, precise inclusion criteria, elements of subjectivity and potential for human error in study selection still exist. For example, inclusion judgments may be influenced by personal knowledge and understanding of the clinical area or study design (or lack thereof)."

"Dual review—having two reviewers independently assess citations for inclusion—is one method of reducing the risk of biased decisions on study inclusion, as is recommended in the Institute of Medicine's “What works in healthcare: standards for systematic reviews” [3].  Some form of dual review should be done at each stage to reduce the potential for random errors and bias. Reviewers compare decisions and resolve differences through discussion, consulting a third party when consensus cannot be reached. The third party should be an experienced senior reviewer."

Syntheses should utilize the PRISMA Flow Diagram to map out the screening process for readers. Different templates are available depending on the type of review (new or updated) and sources used to identify studies.

From: https://www.prisma-statement.org/prisma-2020-flow-diagram.

1. McKeown S, Mir ZM. Considerations for conducting systematic reviews: a follow-up study to evaluate the performance of various automated methods for reference de-duplication. Res Synth Methods. 2024 Nov;15(6):896-904. doi: 10.1002/jrsm.1736. 
2. McDonagh M, Peterson K, Raina P, Chang S & Shekelle P. Avoiding bias in selecting studies. In methods guide for effectiveness and comparative effectiveness reviews [Internet]. 2013. Rockville (MD): Agency for Healthcare Research and Quality (US).
3. Institute of Medicine. Finding what works in health care: standards for systematic reviews (Morton S, Levit L, Berg A & Eden J, eds.). 2011. Washington, DC: The National Academies Press.
4. Stoll CRT, Izadi S, Fowler S, Green P, Suls J & Colditz GA. The value of a second reviewer for study selection in systematic reviews. Research Synthesis Methods. 2019; 10(4), 539–545. https://doi.org/10.1002/jrsm.1369.
5. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ & Welch VA. (Eds). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane.