A December editorial in the Journal of Cell Biology questions the data behind the ISI Journal Citation Rankings (the impact factors published by ISI).
Published online 17 December 2007
doi:10.1083/jcb.200711140
The Journal of Cell Biology, Vol. 179, No. 6, 1091-1092
© The Rockefeller University Press, 0021-9525
Excerpt:
The integrity of data, and transparency about their acquisition, are vital to science. The impact factor data that are gathered and sold by Thomson Scientific (formerly the Institute of Scientific Information, or ISI) have a strong influence on the scientific community, affecting decisions on where to publish, whom to promote or hire (1), the success of grant applications (2), and even salary bonuses (3). Yet, members of the community seem to have little understanding of how impact factors are determined, and, to our knowledge, no one has independently audited the underlying data to validate their reliability.
Related topic:
Declan Butler, Free journal-ranking tool enters citation market, Nature News, January 2, 2008. Excerpt:
A new [OA] Internet database lets users generate on-the-fly citation statistics of published research papers for free. The tool also calculates papers’ impact factors using a new algorithm similar to PageRank, the algorithm Google uses to rank web pages. The open-access database is collaborating with Elsevier, the giant Amsterdam-based science publisher, and its underlying data come from Scopus, a subscription abstracts database created by Elsevier in 2004.
The SCImago Journal & Country Rank database was launched in December by SCImago, a data-mining and visualization group at the universities of Granada, Extremadura, Carlos III and Alcalá de Henares, all in Spain….
The new rankings are welcomed by Carl Bergstrom of the University of Washington in Seattle, who works on a similar citation index, the Eigenfactor, using Thomson data. “It’s yet one more confirmation of the importance and timeliness of a new generation of journal ranking systems to take us beyond the impact factor,” says Bergstrom….
Thomson is also under fire from researchers who want greater transparency over how citation metrics are calculated and the data sets used. In a hard-hitting editorial published in Journal of Cell Biology in December, Mike Rossner, head of Rockefeller University Press, and colleagues say their analyses of databases supplied by Thomson yielded different values for metrics from those published by the company (M. Rossner et al . J. Cell Biol. 179, 1091–1092 ; 2007).
Moreover, Thomson, they claim, was unable to supply data to support its published impact factors. “Just as scientists would not accept the findings in a scientific paper without seeing the primary data,” states the editorial, “so should they not rely on Thomson Scientific’s impact factor, which is based on hidden data.”
Citation metrics produced by both academics and companies are often challenged, says Pringle. The editorial, he claims, “misunderstands much, and misstates several matters”, including the authors’ exchanges with Thomson on the affair. On 1 January, the company launched a web forum to formally respond to the editorial.
More Alternatives:
Eigenfactor.org: the Eigenfactor score is a measure of the journal’s total importance to the scientific community (includes citation ranking, journal price, disciplinary differences, etc.)
H-index: for the impact factor of individual scientists, rather than journals.
