This page provides some basic considerations for developing comprehensive search strategies. Both inexperienced and experienced searchers are encouraged to consult with a librarian when conducting a systematic review or other synthesis, particularly when the intent is to publish.
Balancing sensitivity and specificity within available resources:
"Synthesis teams could commit all their resources to the searching stage of a systematic review, even so it would be impossible to develop the perfect search strategy to identify all relevant studies. When developing search strategies, it is important to consider what trade-offs synthesis teams are prepared to make between sensitivity and specificity given the likely available resources for their review. A highly sensitive search strategy reduces the risk of missing key studies relevant to the review but will increase the total number of records from bibliographic databases that need to be screened." (Grimshaw, 2010).
Using a framework to develop your research question can help to identify the main concepts of your review topic. Whichever framework you use, try to identify the key/main concepts of your question and the synonyms / related terms that might be used to describe each of those concepts:
Image from UCSF Library: https://guides.ucsf.edu/c.php?g=126216&p=825824
It is not always necessary to include all of your main concepts in the search. For example, in some PICO questions, the outcome is implied and does not need to be included. In other cases, any and all outcomes might of interest and so the search strategy can leave this out to keep it open.
When conducting comprehensive database searches, the search terms for each concept should consist of both subject headings (indexed terms) where available, as well as keywords, in order to increase the sensitivity of the search.
It can be extremely helpful if you already know of a few eligible studies for your synthesis. If the eligible studies have been indexed in databases such as MEDLINE and Embase you can review the subject headings that were applied to these studies to harvest a list of relevant search terms. Relevant studies can also be used to identify additional keywords that are relevant for the search.
When conducting a synthesis, the search process is usually very comprehensive as most syntheses strive to find all studies that are relevant to the review topic.
Some syntheses will include the appropriate study design(s) as part of the search strategy (the 'S' in PICOS and PICoS above).
Identifying particular study designs in a database search is not as straight-forward as, for example, limiting to the publication type "Randomized Controlled Trials" in PubMed. This is partly because not all indexed records get assigned the appropriate publication type but also because databases like Ovid MEDLINE/PubMed include non-indexed citations that either have not or will not receive indexing to indicate the study design.
Search filters ("hedges") have been developed for different databases to do just that. These filters can be applied as published or they can be revised to increase the specificity or sensitivity of your search.
Study design filters:
It may also be tempting to exclude publication formats such as letters from your search to reduce the number of results.
If your review topic implies that the research subjects are human, as is often the case with qualitative research, it is advisable to resist the temptation to limit to humans. However, if your search results retrieve a large number of animal studies, you may consider limiting to human studies.
Limiting to humans is not as straight-forward, for example, as using the limit for "Humans" in Ovid MEDLINE. This is partly because indexed records in a database may not have been assigned a subject heading for the appropriate species but also because databases like PubMed/Ovid MEDLINE include non-indexed citations. These records either have not or will not receive indexing to indicate whether the article discusses animals and/or humans.
The recommended method for limiting to human studies in a database is to exclude indexed records that have been assigned a subject heading for animals but that have not been assigned a subject heading for humans. This will remove indexed records that discuss animals only.
For example, this command would read as follows:
|Ovid MEDLINE||*Search# NOT (exp animals/ NOT humans/)|
|Ovid Embase||*Search# NOT ((exp animal/ or nonhuman/) NOT exp human/)|
*Where the Search# represents the search line that you want to limit to human studies.
It is important to note that limiting to human studies is not equivalent to excluding animal studies since the latter would remove studies that discuss both humans and animals. Therefore do not use the command: Search# NOT animals/ (for example).
The most common search errors reported were:
Grimshaw J. A guide to knowledge synthesis: a knowledge synthesis chapter. 2010:1-56. http://www.cihr-irsc.gc.ca/e/41382.html.
Iansavichene, A. E., Sampson, M., McGowan, J., & Ajiferuke, I. S. (2008). Should systematic reviewers search for randomized, controlled trials published as letters?. Annals of Internal Medicine, 148(9), 714-715.
Lefebvre C., Glanville J., Briscoe S., Littlewood A., Marshall C., Metzendorf M.-I., Noel-Storr A., Rader T., Shokraneh F., Thomas J. & Wieland L.S. Chapter 4: Searching for and selecting studies. In: Higgins J.P.T., Thomas J., Chandler J., Cumpston M., Li T., Page M.J. & Welch V.A. (Eds). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane, 2019. Available from www.training.cochrane.org/handbook.
Stern, C., & Kleijnen, J. (2020). Language bias in systematic reviews: you only get out what you put in. JBI evidence synthesis, 18(9), 1818–1819. https://doi.org/10.11124/JBIES-20-00361