Bibliometrics is the quantitative analysis of journal articles, books and other publications, which can be used to understand elements of research performance and impact. With the acquisition of two bibliometric databases, SciVal and InCites, Toronto Metropolitan University (TMU) Libraries has published a Responsible Use of Bibliometrics Statement to provide guidance for the TMU research community and raise awareness of the strengths and limitations of bibliometric indicators. This statement joins a growing list of responsible metrics statements from institutions and libraries across Europe and North America.
Responsible use of bibliometrics is especially important in the current Canadian research funding landscape now that the Tri-Agency funders have affirmed the Declaration on Research Assessment (DORA). NSERC has also released guidelines on research assessment that advises against using “surrogate measures”, such as bibliometrics:
The quality and impact of contributions to research should be assessed directly, where possible. Surrogate measures of quality and impact, such as the prestige of a publication venue or citation-based metrics (e.g., journal impact factor or h-index) must not be used as they introduce bias in the merit review process.[1]
Reliance on bibliometric measures in research assessment have been shown to be potential obstacles in the area of equity and inclusion, and can contribute to “structural inequalities in academia.”[2] For example, Diana Kwon cites political philosopher Perry Zurn on issues of citational justice: “We have really strong evidence that women are undercited and that people of colour are undercited.”[3] Responsible use of bibliometrics statements advocate for the direct and holistic evaluation of research, which can work towards addressing these inequities.
Direct evaluation of research also includes expanding which outputs are valued as part of scholarly knowledge creation and should therefore be considered in research assessment. NSERC, in their comprehensive “Forms of contributions to research”, points out that while scholarly journal articles are a valued research output when it comes to assessment, other forms of output and their impact should also be assessed.[4] A more expansive view of research contributions may include: equity and inclusion work in the research ecosystem, community service, policy impact, knowledge translation and mobilization activities, creation and curation of datasets, creation of intellectual property, and support for traditional knowledge and Indigenous Ways of Knowing.[5]
Educating the research community regarding the responsible use of bibliometric indicators is foundational to TMU Libraries’ emerging bibliometrics and research impact services. TMU Libraries’ Responsible Use of Bibliometrics Statement provides guidance for the responsible use of bibliometric tools, but also advocates for a holistic view of research performance evaluation, which includes both qualitative evaluation and quantitative indicators.
Read TMU Libraries’ Responsible Use of Bibliometrics Statement here: https://library.torontomu.ca/bibliometrics-statement/
________________
[1] Natural Sciences and Engineering Research Council of Canada. (2022, December 1). Guidelines on the assessment of contributions to research, training and mentoring. Government of Canada. https://www.nserc-crsng.gc.ca/NSERC-CRSNG/Policies-Politiques/assessment_of_contributions-evaluation_des_contributions_eng.asp
[2] Hatch, A., Barbour, G., & Curry, S. (2020, August 18). The intersections between DORA, open scholarship, and equity. DORA News and Announcements. https://sfdora.org/2020/08/18/the-intersections-between-dora-open-scholarship-and-equity/
[3] Kwon, D (2022): The rise of citational justice: how scholars are making references fairer. Nature 603, 568-571. https://doi.org/10.1038/d41586-022-00793-1
[4] Natural Sciences and Engineering Research Council of Canada. (2022, December 1). Guidelines on the assessment of contributions to research, training and mentoring. Government of Canada. https://www.nserc-crsng.gc.ca/NSERC-CRSNG/Policies-Politiques/assessment_of_contributions-evaluation_des_contributions_eng.asp
[5] Natural Sciences and Engineering Research Council of Canada. (2022, December 1). Guidelines on the assessment of contributions to research, training and mentoring. Government of Canada.
https://www.nserc-crsng.gc.ca/NSERC-CRSNG/Policies-Politiques/assessment_of_contributions-evaluation_des_contributions_eng.asp