Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact

Edited by Blaise Cronin and Cassidy R. Sugimoto. Cambridge, MA: MIT Press, 2014. 432 pp., 33 figs. $35.00 (paperback). ISBN 978-0-262-525510

In a fit of genius, J. Britt Holbrook and colleagues from the Center for the Study of Interdisciplinarity at the University of North Texas proposed a list of 56 Indicators of Impact, ranging from “# of citations” to “meetings with important ppl.”[1] Subsequently published in the hallowed pages of Nature under the heading “Research impact: We need negative metrics too,” Holbrook’s index represents a fitting case study from which the theme of Blaise Cronin and Cassidy Sugimoto’s book emerges; the depth of what we know or can know about measuring the impact of a scholarly object is no longer contained in the h-index alone. Situating itself in the midst of a complicated time for academic publishing, Beyond Bibliometrics offers a collection of essays from an international group of academics, information scientists, publishers and alt/bibliometricians, all these providing clarity, context and challenges to our current moment of measurement.

Those beyond the academy, and perhaps some within its walls, will find more questions than answers culled in this text. However, Cronin and Sugimoto provide important structure to the discourse, organizing under five major themes: History, Critiques, Methods and Tools, Alternative Metrics, and Perspectives. The authors, given the breadth of possible perspectives, tackle each of these subtopics heroically. The more accessible essays begin and end the book. Cronin and De Bellis paint the incredibly helpful background of how, where and why letters passed between thinking people eventually (d)evolved to incorporate the “macro (country), meso (discipline), or micro (program) level” indicators of research performance (6). Two important discussions close the book, addressing and introducing the publisher and public policy perspectives on research impact. From an on-the-ground point of view, these four selections provide necessary understanding on where we have been, where we are, and where we are going in regards to scholarly communication.

The meat of Beyond Bibliometrics is satisfyingly rich with practitioner knowledge. Yves Gingras proposes that before advancing boldly into the measurement matrices there should be a thorough review of the “epistemological foundation – that is, whether these indicators really have any definite meaning and thus measure what they are supposed to measure” (111); i.e. who evaluates the evaluators? Leydesdorff plays, literally, with bibliometric data to form network visualizations, which speak to the recognizable patterns of connectivity within and across a field. Kurtz and Henneken poke at the concept of “recommendation,” which we are all so used to from Netflix or Amazon, and examine that feature in the greater context of the scholarly information system that is expanding exponentially every year. The Alternative Metrics subsection is particularly palatable as it fleshes out the sexy that is giving bibliometrics its moment in the higher education spotlight. Hinted at by Cronin in his early essay, and taken up here by his co-editor, Sugimoto’s “Academic Genealogy” is an exploration in altmetrics that is fascinating and novel, studying “the transfer of disciplinary knowledge… by making visible the contributions of mentors” in facilitating the growth of new scholars in their doctoral students (366).

The editors’ keen eye for balance is noted and noteworthy. Practical applications are paired with theoretical treatises, and as a whole Beyond Bibliometrics instills a sense of confidence in the reader’s handle on the evolving modes of how we decide something is important or not. Fittingly, the collection skirts the power dynamics at play in impact measurement; regardless of how many re-tweets this book review gets, neither I nor the authors have much to say about how, where and why that specific number weighs differently than any other. Returning again to Holbrook’s 56 Indicators of Impact, it is comforting to know that rabble rousing, muckraking, arrests, and angry letters from important people are on the table right next to Place of Publication. Cronin and Sugimoto can count on their collection feeding the Holbrookian in all of us.

Micah Vandegrift

Florida State University