12

Introduction

Research metrics are measurements used to capture the impact of a research. These metrics capture quantitative data at the article, journal and author level. Tools used to capture this information include citation counts, h-index, and journal impact factor. There is no one measurement or number that can perfectly capture the impact of a project or author, so it is important to consider the various methods and measurements together.

While there are some flaws to these measurements, measuring research impact is important for various reasons. Scholars may use author-level measurements to recognize their impact in their field and to support their progress toward promotion and tenure or communicate their influence to funders. Journal-level measurements may help authors understand the relative importance of a journal and sway decisions on where they want to publish.

Types of Measurements

Methods for measuring research impact look at the article level, author level, and journal level. Since research and citing practices differ across disciplines, every discipline considers these metrics differently, making it difficult to use these metrics across fields.

Table of measurements adapted from Quick Reference Cards for Research Impact Metrics, Elsevier, CC-BY-NC-SA

Citation Count

The Citation Count is a simple measurement for a particular article, journal or researcher. The citation count is the total number of times the article, journal, or researcher have been cited in other works. Typically high citation counts are associated with more influential works or authors. However, as with all citation-based measures, it is important to consider how self-citations or other challenges can impact this measurement.

h-index

The h-index measurement is used to measure an author’s influence and citations. The h-index is calculated by counting the number of publications that an author has been cited by others at least that same number of times. For example, an h-index of 4 means that an author has had 4 articles that have each received at least 4 citations. Likewise, an h-index of 11 means that an author has had 11 articles each cited at least cited 11 times. The h-index is beneficial as it is not skewed by few highly cited articles or many poorly cited articles. The h-index is provided by various sources, including Web of Science, Scopus, and Google Scholar.

Journal Impact Factor

The Journal Impact Factor measurement was created by Eugene Garfield in 1955 to understand which journals published the most articles as well as which journals contained the articles that are most highly cited in that discipline. The Journal Impact Factor is calculated by taking the ratio between a journal’s citations and number of recent citable items published. Typically, a journal with a high Impact Factor is associated with being a prestigious or important journal in its field. Because citing behaviors vary in different fields, Impact Factors are difficult to compare across disciplines.

Altmetrics

Altmetrics is used to capture the impact of an article beyond traditional citations. This measurement can provide insight on the influence an article has on groups outside of research and academia. Altmetric scores are calculated from an automated algorithm to help understand the attention a research article or other output has received. The various types of sources, such as newspapers or Twitter, are weighted differently when calculated. For example, a policy document mention is weighted higher than a Facebook mention. In addition to a score, Altmetrics also provide a colorful visual representation of the types of sources mentioning the item. This is referred to as the Altmetric Donut.

Figure of Altmetric color code from Altmetric.org

Other Measurements

Citation count, h-index, Journal Impact Factor, and Altmetrics are some of the most common tools to measure research impact, however there are various other tools, including Scimago Journal Rank, CiteScore, Source Normalized Impact Per Paper, and CiteScore. For example the Source Normalized Impact Per Paper (SNIP) number helps to account for differences in citing and publishing norms across fields. Additionally, the Document Count captures the number of items published by an individual or group of individuals. The Document Count helps to capture a researcher’s productivity, but not necessarily their impact.

Understanding Limitations with Metrics

Impact measurements have limitations in the data they capture. No one metric tool tells the whole story and errors are possible in each measurement. Citation counts can be incorrect from author name spelling errors, variations, or if more than one author share the same name. Self-citing can also presents a challenge to citation counts. Self-citing can artificially inflate metrics, however most systems attempt to account for this.

Journal-level measurements can also have errors. Journals that contain reviews in addition to articles may have skewed journal impact factors if the reviews are cited.

These metrics also do not capture the qualitative data, or what the scholarly conversation citing these items looks like. The number of citations can indicate significance, but not whether the attention is positive or negative. For example, a high Altmetric score does not necessarily mean the item has been positively received.

Tips for Assessing Impact

• Impact metrics are imprecise and take time to develop.
• Combining different metrics can help you understand the impact of your work.
• Journal Impact Factors vary across disciplines and may be less relevant with broad access to materials through digital distributions.

Quick Reference Card

Quick Reference Card, by Elsevier Library Connect & Jenny Delasalle

https://www.recursoscientificos.fecyt.es/sites/default/files/els_lc_metrics_poster_researcher_v3.0_2016.pdf