Research Impact: Measuring Impact

In this Guide

Measuring Impact

Most research will be best represented using a combination of quantitative and qualitative approaches to measure impact, although this can differ vastly by field.

The following is a broad overview of medicine and health 

Medicine and Life Sciences

• Feature wider collaboration and a more rapid rate of publications - can expect higher citation rates than other disciplines such as mathematics.
• Publish frequently in journals but also produce conference papers, books and book chapters and reports, generating a vast quantity of literature that cites within itself
• Impact in these areas tends to be more quantitative and metric focused, often measured in relation to the number of citations the research generates

Citation Analysis:

• Citations Count
• Citation patterns
• Citer Analysis
• h-index
• Journal Metrics

Other forms of Impact Analysis:

• Altmetrics
• Influencing Practice, Policy and Programs

Citation Analysis

Citation Analysis is an indication of the quality of the article.

• Keep in mind to only compare citation counts of articles that are published in the same field.
• Though these metrics indicate quality in a general manner it should be treated carefully. For example an article could have great impact and a large citation count, not because it represents great quality but because the science is poor and the impact has been catastrophic.
• Citation counts and patterns can be found in most databases including PubMed and Google Scholar.

Citation Counts

Citation counts represent the number of times a scholarly document has been cited by other researchers. It is the basic tool of citation analysis. 

Citation Patterns

Citation patterns are dependent on citation counts. They can be useful in showing how your research performs over time.

• Citation items in each year shows peaks and troughs which can reflect the research cycle. This can help you show your productivity

• Citations in each year can show that your work is continuing to grow in influence e.g.

. 

Citer Analysis

Citer analysis provides qualitative measures of your research impact by providing information about the citations of your work, including:

• who is citing your research
• where they are
• what institution they're from
• in which publications they have published
• in which discipline they have published

Finding out who is citing you and where they come from can help you show:

• significance of impact
• research trends
• evidence that your work is influential across multiple disciplines

Scopus and Web of Science provide graphic and analytic tools to help gather this information. If you do not have access to these resources through an Academic institution, Google Scholar can provide similar information but requires more manual effort.

Altmetrics

Altmetrics for tracking scholarly communication on the Web.

Altmetric scores take into account the number of times an article is read or mentioned and is weighted by the importance of the mentioner and by the source. For example, an article mentioned in a newspaper would be weighted more heavily than a mention in a tweet by an individual.

Many journals include some altmetrics on their websites that include views, downloads and social media mention

Mentions of articles are tracked in the following source types:

• mainstream media e.g. newspapers
• tweets
• blogs
• policy documents
• multimedia
• online reference managers
• social media
• manual additions

Altmetrics:This is a metric commonly used to track scholarly communication on the Web. This tool has been integrated into many databases such as Science Direct and also journal websites such as the British Medical Journal.

Influencing Policy, Practice and Programs

Research impact on policies, programs and practice can be illustrated by the number of mentions in Government and Non-Government Organization documents such as white papers, parliamentary papers, reports, program evaluations etc.

• Tracking and measuring research impact on policies, programs and practice can be difficult and will need a variety of web based tools, from Google alerts to database searches, tailored specifically to your needs. Please see your Librarian for help. 

Primary Health Care Research Information Services (PHCRIS) provides researchers with practical guides, tips, videos, fact sheets, on how to influence policy development.

• Maximizing the Influence of Your Research on Policy
• Introduction to Knowledge Exchange
• Introduction to Research Impact and Policy
• How to Communicate your Research to Decision Makers