Skip to Main Content

Public Health

Guide to library research for evidence-informed public health practice.

Introduction

Evaluating your sources is a crucial step of the information research process. 

  • Critical appraisal is the process of assessing the quality of research study methods to determine if the reported outcomes are trustworthy, meaningful and relevant to your situation. Critical appraisal checklists are available to help you evaluate research studies in the form of intervention studies, observational studies, qualitative research, systematic reviews, and more.
  • Specific evaluation criteria to consider for grey literature sources is provided further down the page followed by general considerations for evaluating information by means of two different (but similar) checklists that you can apply.

Appraising Grey Literature

Grey literature is the term used to describe literature that is not formally published as a book or journal article (Higgins, 2011), including:

  • conference proceedings
  • preprints
  • dissertations and theses
  • governmental or private sector research
  • ongoing or unpublished clinical trial data
  • statistical publications
  • internal reports or working papers
  • technical reports

The critical appraisal checklists listed further up the page can be used to evaluate research study designs such as systematic reviews, RCTs, cohort studies, etc., regardless if the research is formally published or otherwise provided as a conference paper, pre-print, etc. Evaluating studies in the form of grey literature becomes more difficult when limited information is provided, which can be the case for conference abstracts, clinical trial data etc.

Grey Literature Checklists

  • The AACODS Checklist (Authority, Accuracy, Coverage, Objectivity, Date, Significance) was developed to appraise the widest range of grey literature sources possible (Tyndall, 2010)
     
  • The Ontario Agency for Health Protection and Promotion also offers the following considerations about the presence (reporting) and the appropriateness (validity) of the items described below may help to assess the quality of the source (2015):

 

 Reporting Validity
Author/ publisher Who is the author, are they who they claim to be?

Does the author/host have an agenda?

Publishing body:

  • Are they a well-known provincial/national government-affiliated public health agency?
  • For-profit company?
  • Political or philosophical agenda? (e.g. Fraser Institute vs. Canadian Centre for Policy Alternatives) 
Host/sponsor Does the hosting site have a sponsor?

Is the information shared for commercial purposes?

Does the site sponsor have a political or philosophical agenda? 
Facts and references Are references included? 

Are references to research evidence?

Is the information based on research/data/analysis or is it opinion piece?

Can you verify some cited sources, facts?

How current are the references? 
Currency Is a publication or revised date provided?  Is the information current?

Evaluating Sources Checklist

The following checklist can be used to appraise both scholarly and non-scholarly sources.

Purpose Why was the resource written? Was the author's purpose to inform, persuade, or to refute a particular idea or point of view?
Audience Is the resource intended for the general public, scholars, professionals,etc.
Authority What are the author's qualifications? Consider author's educational background, past writings and experience. Is the author associated with an organization or institution? Who is the publisher? Are they well known? Does any group control the publishing company?
Accuracy Is the information covered fact, opinion, or propaganda? Facts can be usually verified. Opinions evolve from the interpretation of facts. Are the author's conclusions or facts supported with references?
Timeliness When was the information published? Is the date of publication appropriate for your topic?
Coverage Is it relevant to your topic? Is the topic covered in depth, partially or is it an broad overview? Does the resource add new information, update other sources or substantiate other resources that you have consulted?
Objectivity Does the author present multiple viewpoints or is it biased? How do critical reviews rate the work?

CRAAP Checklist for Evaluating Web Resources

It's easy to find information on most any topic on the Web but whether the information found is reliable, up-to-date and unbiased is the question researchers face. Since anyone can post anything on the Web, it is very important to critically examine the information and the website. The CRAAP Test is a list of questions to help researchers evaluate sources of information. It is a tool to help you think critically about the quality of health information.

Currency

  • When was the information published or posted?
  • Is it current enough for your topic?
  • Has it been revised or updated?

Relevance

  • Does the information relate to your topic or answer your question?
  • Who is the intended audience?
  • Is the information at an appropriate level (i.e. not too elementary or advanced?)
  • Have you looked at a variety of sources before determining to use this one?
  • Are the topics included explored in depth?

Authority

  • Who is the author or creator?
  • What the author's qualifications or credentials in writing about this subject?
  • How reputable is the publisher?
  • Are there organization affiliations? Are they reputable?
  • Does the information provide references or sources of data? 

Accuracy

  • Where does the information come from?
  • Is the information supported by evidence?
  • Has the information been reviewed or refereed?
  • Can you verify the information in another source or from personal knowledge?
  • is the content primarily opinion? Or is it balanced with multiple points of view?
  • Does the language or tone seem unbiased and free of emotion?

Purpose

  • What is the purpose of the information? Is it to inform, teach, sell, entertain or persuade?
  • Do the authors make their intentions or purpose clear?
  • Is the information fact, opinion or propaganda?
  • Does the point of view appear objective and impartial?
  • Are there political, ideological, cultural, religious, institutional or personal biases?

The CRAAP Test was developed by a librarian at California State University, Chico.