Health Library: Evidence-Based Practice: Appraise the Evidence

In this Guide

Appraise the Evidence

Critical analysis is a way or examining the research to assess its validity and relevance. It is an essential step in making sense of the research evidence. You should ask the following questions:

What are the key terms and concepts?
Importance of the research question being answered
How relevant is this article to my specific question?
Strengths and weaknesses of the articles study design and methodology
How authoritative and credible is this source?
Any potential conflicts of interest
Legitimacy of conclusions

Critical Appraisal Tools

Below is a list of critical appraisal tools used in evidence-based medicine:

The Aboriginal and Torres Strait Islander Quality Appraisal Tool and Quality Appraisal Tool Companion Document.
Checklists for Assessing Study Qualities - checklists to assess study design, risk of bias and reporting
Critical Appraisal Skills Program checklists (CASP) - offers checklists for 8 different types of research, including systematic reviews, qualitative studies, randomised controlled trials, case control studies, and diagnostic studies. Each checklist contains both yes/no and open ended questions.
Joanna Briggs: Critical Appraisal Tools - produces 13 tools used to appraise different studies or levels of evidence. Each checklist contains a series of critical appraisal questions and ends with an overall appraisal decision.
A Measurement Tool to Assess Systematic Reviews (AMSTAR) - a tool to appraise systematic reviews of randomised controlled trials. Systematic reviews made up of non-randomised trials can be assessed using AMSTAR 2.
Newcastle-Ottawa scale (NOS) - a tool for appraising and assessing non-randomised studies included in a meta-analysis.
Oxford Centre for Evidence-Based Medicine (CEBM) - critical appraisal tools for systematic reviews, randomised controlled trials, and diagnostic and prognostic studies.
QUADAS-2 - a tool for assessing the quality of diagnostic accuracy studies.
RoB2.0 tool - Cochrane risk of bias tool for randomized trials.
Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) - evaluation checklists for cohort, case-control and cross-sectional studies.

Understanding Terminology

The following terminology is important in critical appraisal and will be helpful to determine the validity of research. Additianal terms can be found in the Centre for Evidence Based Medicine Glossary

Applicability	Whether a particular treatment or form of care that was demonstrated as effective in a trial could be expected to provide the same effect for an individual or group in a specific clinical or population setting
Bias	Any influence that may cause a distortion of results in a study and undermine validity. Bias may originate from various sources, such as allocation of patients, measurement, interpretation, publication, and review of data
Confidence interval (CI)	Confidence Interval (CI) is the interval within which the population parameter (the ‘true’ value) is expected to lie with a given degree of certainty, for example 95%
External validity	External validity is the degree to which the results of a clinical study can be applied to a setting other than the one studied.
Generalisability	The ability to reliably apply the results of a study to other populations, based on the characteristics of the subjects, size of the sample, the setting, and trustworthiness of the study
Intention-to-treat	A method of analysis for randomized trials in which all patients randomly assigned to one of the treatment groups is analysed with that assigned group, regardless of whether or not they completed or received the treatment.
Internal validity	A study has internal validity if it is free from bias or systematic error and the results seen are due only to the intervention.
p value	p value is the probability that a particular result would have happened by chance.
Power	Ability to detect a difference between two experimental groups if a difference exists.
Relative risk (RR)	Relative Risk (RR) is the ratio of the probability of outcome in the treatment and control groups. This expresses the risk of the probable outcome in the treatment group relative to that in the control group.
Reliability	The extent of the measurement of trustworthiness of the results.