METRICS 2015 – ASIS&T SIG/MET Workshop on Informetric and Scientometric Research
Originally published in: Haustein, S. (2016). SIG/MET METRICS 2015: Workshop on Informetric and Scientometric Research. Bulletin of the Association for Information Science and Technology, 42 (3): 24–27.
For the fifth time SIG/MET, ASIS&T’s Special Interest Group for the measurement of information production and use, held its workshop on November 7, 2015 at the ASIS&T Annual Meeting in St. Louis. Organized as a full-day workshop by SIG/MET chairs and officers Isabella Peters (ZBW Leibniz Information Centre for Economics, Germany), Stefanie Haustein (University of Montreal, Canada), Chaoqun Ni (Simmons College, USA), Kun Lu (University of Oklahoma, USA) and Timothy D. Bowman (Åbo Akademi University, Finland), METRICS 2015 was organized around 15 presentations and 3 posters and attracted 45 participants. In addition to submitting a peer-reviewed poster ahead of the workshop, participants were for the first time invited to bring posters with their latest findings to discuss them during two open poster sessions. Workshop presentations, posters and discussions revolved around SIG/MET’s central topics related to the measuring of information including metrics research and application in bibliometrics, informetrics, altmetrics and information retrieval as well as social network analysis and visualization of scholarly communication.
Attendees of the METRICS 2015 workshop at the ASIS&T Annual Meeting in St. Louis.
The workshop opened with a session devoted to bibliometric case studies. George Lan (Elsevier) presented on the impact of interdisciplinary publications. He differentiated between multiple audiences or “consumers” of academic research comparing the effect of the level of interdisciplinarity of papers on their citations from articles and patents as well as downloads. Philippe Mongeon (University of Montreal) demonstrated the positive effect of funding on researchers’ collaboration networks. Based on researchers in Quebec, he found a linear positive correlation between the funding amount and the number of new co-authors of funded authors. While Mongeon’s analysis of collaboration patterns was quantitative and based on co-authorship patterns, Katherine W. McCain (Drexel University) took a different and more qualitative approach to identify scientific collaboration: she analyzed personal mentions of researchers in the acknowledgement texts of research papers using the starlet sea anemone as a model organism. Acknowledgements were classified into ten different types including “providing animals”, “providing comments on manuscript” or “inspiration, valediction”.
The second session of the METRICS workshop focused on information retrieval and its relationship with bibliometrics. Kun Lu (University of Oklahoma) and Dietmar Wolfram (University of Wisconsin-Milwaukee) dedicated their study to the problem of computational limitations that occur due to an ever growing amount of data and increasingly large corpora. In their presentation entitled “Vocabulary size and its effect on topic representation for informetric and information retrieval data processing”, they showed based on an analysis of three datasets that processing could be increased by cutting off the so-called “long tail” without affecting retrieval results. Andrea Scharnhorst (Royal Netherlands Academy of Arts and Sciences KNAW) introduced Ariadne, an interactive context explorer for bibliographic data. Ariadne is trained on 65 million publications from OCLC’s ArticleFirst database but can be used as an exploratory tool for any dataset. Ariadne’s algorithms were tested and compared to other methods of topic extraction and visualization in the study by Theresa Velden (University of Michigan) and colleagues, who concluded the information retrieval session. The contribution entitled “Same data, different results? On a comparative topic extraction exercise” presented a systematic comparison of different approaches and algorithms to visualize the structure of an astrophysics and astronomy dataset with more than 110,000 papers. Velden emphasized that several solutions are valid and that the aim of the exercise was to find the best solution for a given purpose. So-called “blind spots” of some of the methods were identified and visualized.
Similar to past workshops, SIG/MET recognized outstanding contributions of students with a best student paper award sponsored by Elsevier. This year the jury determined two winners, who were given the opportunity to present their papers during the third session of the workshop. Misha Teplitskiy (University of Chicago), who co-authored his paper “Amplifying the impact of open access: Wikipedia and the diffusion of science” with fellow student Grace Lu and supervisor Eamon Duede, showed that open access and high impact factor journals were more likely to be cited in Wikipedia. The paper “Evolution of iSchool movement (1988-2013): A bibliometric view”, co-authored by McGill and University of Montreal PhD students Fei Shu and Philippe Mongeon, won the other best paper award. Shu presented the history of the iSchool movement based on more than 4,000 LIS dissertations indexed in ProQuest. He highlighted that interdisciplinarity increased over the years and that computer science gained importance. Elsevier’s sponsorship allowed SIG/MET to award each of the two best student papers with $500 and finance ASIS&T student memberships for all students participating in the workshop.
Best student paper award winners Misha Teplitskiy (left), Fei Shu (middle) and Philippe Mongeon (right) with SIG/MET Chair Isabella Peters.
Altmetrics has been a hot topic of the metrics and scholarly communication community in the past years. This trend is also reflected in the METRICS 2015 program, as the fourth session consisted of four talks on alternative indicators and altmetrics. Stacy Konkiel (Altmetric) presented an exploratory case study based on scientific papers cited in public policy documents. She emphasized that policy document represent an interesting source for societal impact metrics but that the set of documents covered by Altmetric is still limited to a small number of selected sources. Adèle Paul-Hus (University of Montreal) presented the results of her study co-authored with Philippe Mongeon and Fei Shu. The three PhD students from Montreal identified the Twitter accounts and tweeting activity of faculty members of LIS schools accredited by the American Library Association (ALA) and compared their Twitter followers to their co-authorship networks. They concluded that connections on Twitter reflected mostly institutional affiliations instead of research interests. Jim Pringle (Thomson Reuters) introduced “A new interest indicator based on researcher behavior in the Web of Science”. The new usage count was designed to complement the article citation count by a usage indicator to enhance article discovery. Pringle presented interesting disciplinary and journal-based patterns comparing usage and citation counts. The alt(ernative) metrics session concluded with the presentation by Stefanie Haustein (University of Montreal), who together with her colleague Rodrigo Costas (CWTS, Leiden University) analyzed terms appearing in Twitter bios of users mentioning scientific papers to determine who is tweeting about science. Their preliminary findings suggest that a large share of tweeters identify as academics, which contradicts the assumption that tweets reflect impact on society at large.
Best altmetrics paper award winners with Stacy Konkiel of Altmetric: Fei Shu, Stefanie Haustein, Stacy Konkiel, Adèle Paul-Hus and Philippe Mongeon (from left to right).
For the first time at the SIG/MET workshop, Altmetric and figshare sponsored an altmetrics paper award. The jury decided to award two contributions: the papers by Mongeon, Paul-Hus and Shu as well as Haustein and Costas were both selected as the best SIG/MET altmetrics papers and awarded $500 each.
Bradford Demarest (Indiana University) presented his doctoral research on comparative discourse epistemetrics in the session entitled “Broadening the scope of bibliometrics”. Analyzing the abstracts and full texts of Philosophy and Psychology papers, Demarest showed that the two disciplines can be well distinguished based on the adverbs and pronouns used.
The workshop concluded with two presentations showcasing the application and use of metrics. Betsy Martens (University of Oklahoma) presented the Oklahoma Mesonet platform and analyzed more than 650 papers citing the tool. Edwin Henneken (Smithsonian Astrophysical Observatory) demonstrated the new retrieval interface of the Astrophysics Data System (ADS). After the search interface of ADS had remained practically unchanged since 1992, the new platform offers a variety of analytical and visual possibilities to enhance retrieval with a modern interactive design. New functionalities can be applied to any set of results and include a variety of metrics, histograms and time series as well as co-citation and co-author network visualizations.
More information about the SIG and the METRICS 2015 workshop, including abstracts and slides of all presentations, can be found on SIG/MET’s website at www.asis.org/SIG/SIGMET/. For the latest information and news, follow SIG/MET’s mailing list (http://mail.asis.org/mailman/listinfo/sigmetrics) and Twitter account (https://twitter.com/sig_met).
METRICS 2015 Workshop Program
About the author
Stefanie Haustein is chair of the ASIS&T Special Interest Group/Metrics (SIG/MET). She is a postdoctoral researcher at the Canada Research Chair on the Transformations of Scholarly Communication at the University of Montreal, Canada. Her current research focuses on social media in scholarly communication and altmetrics and is supported by the Alfred P. Sloan Foundation. Stefanie is a visiting lecturer at Heinrich Heine University Düsseldorf and co-chair of the NISO altmetrics working group on data quality. She can be reached at firstname.lastname@example.org or found on Twitter via @stefhaustein.