|ASIS&T 2006||START Conference Manager|
We took advantage of data and tools for citation analysis studies increasingly available on the Web and compared rankings of cited authors in the XML research field by the three different citation counting methods. Results clearly showed that the number of top-ranked authors shared between author rankings resulting from different counting methods was quite small, and that rankings by complete and fractional counts were highly correlated but were significantly different from that by straight counts. An analysis of the shifting patterns of authors between different rankings indicated that fractional counts are least biased among the three methods compared because the assumption underlying straight counts, i.e. that the first author of a paper contributes solely or significantly more to the paper than the other authors, frequently does not stand true, and complete counts unfairly favor authors who are members of large-group collaborations.
We thus provide strong empirical evidence for the bias associated with straight counts, and for the lack of significant correlations between author rankings by straight counts and other counting methods. We also demonstrate the capability of emerging tools such as CiteSeer in supporting various counting methods, which contests a major justification for the continued use of straight counts in citation analysis studies – the prohibitive costs involved in counting non-first cited authors using the ISI databases. Although it may require more technical skill and effort to work with them, these tools reduce the costs significantly. More importantly, these tools allow us to go beyond what the ISI databases have supported to explore more sophisticated citation counting methods for more balanced and less biased evaluations of scholarly impact. This may in turn contribute to the emergence of a more effective scholarly communication system in which scholarly work is evaluated based on how many users it has reached in all rather than on how many users it has reached who have published in journals indexed by certain databases such as the ISI databases.
|START Conference Manager (V2.52.6)|