The Thrall of the Citation Index. (August 17, 2009)

 Last Thursday Zoë Corbin*, writing in The Times, asked the rhetorical question, "Do academic journals pose a threat to the advancement of science?" and took some 4,000 words to answer it.

Here are some key points.

 The pressure to publish in top journals has increased even further with the recent announcement by the Higher Education Funding Council for England that citations will be available for use by panels to help them judge the quality of academics' output in the new research excellence framework. As academics strive to increase their citation counts, it seems likely that the new system will only serve to intensify the publish-or-perish mentality.

From one point of view, such an approach makes perfect sense, and Hefce is only the latest in a long line of various career-making and grant-awarding bodies that rely on journal metrics. Funders need a way of dispensing precious resources to the best people, and academics are more likely to cite others' work if it advances the field. Why not tap into the evidence of strong scholarship offered by journal metrics to help make a more objective and less burdensome judgment of quality?

"The hegemony of the big journals has enormous effects on the kind of science people do, the way they present it and who gets funding," observes Peter Lawrence, a researcher in the department of zoology, University of Cambridge, and emeritus scientist at the Medical Research Council's Laboratory of Molecular Biology.

"Academic credit goes with publishing in a handful of really prestigious journals and, as a result, they exert a horribly powerful influence," notes Richard Smith, former editor of the British Medical Journal...

"(Journal metrics) are the disease of our times," says Sir John Sulston, chairman of the Institute for Science, Ethics and Innovation at the University of Manchester, and Nobel prize-winner in the physiology or medicine category in 2002. He is also a member of an International Council for Science committee that last year drafted a statement calling for collective action to halt the uncritical use of such metrics.

Sulston argues that the use of journal metrics is not only a flimsy guarantee of the best work (his prize-winning discovery was never published in a top journal), but he also believes that the system puts pressure on scientists to act in ways that adversely affect science - from claiming work is more novel than it actually is to over-hyping, over-interpreting and prematurely publishing it, splitting publications to get more credits and, in extreme situations, even committing fraud.

Richard Horton, editor of The Lancet [says] "If I could get rid of the impact factor tomorrow, I would. I hate it. I didn't invent it and I did not ask for it. It totally distorts decision-making and it is a very, very bad influence on science,"

Peter Lawrence has written extensively, in Nature and elsewhere, about what he calls the "mismeasurement" of science. His view is that a reliance on journal metrics is damaging science and has created a new generation of scientists obsessed with how many publications they have to their credit. [And] he points the finger... [at]"the bureaucrats that give you a grant if you get a paper published but not if you don't". Above all, he blames the scientists who have been complicit in giving journals substantial power.

According to Smith, one of the most unfortunate consequences of the elite journals' power is that they are holding back progress towards open access, the alternative publishing model where papers are free for the public to read, but publishing and peer-review costs (which all parties agree are substantial) are paid by funders.

Smith says: "Journals are getting rich off the back of science without, I would argue, adding much value." Publishers, he says, take scientists' work, get other scientists to peer review it for free, and then sell the product back to scientists' institutions through subscriptions.

Horton believes that to survive, journals need to rediscover their original purpose and ethos. He argues that they do have an important dual role - setting a strong public agenda around what matters in science and communicating the public's concerns back to the scientific community. "Right now they are doing very badly at both."

Smith sits on the board of the Public Library of Science (PLoS), the open-access journal publisher that produces PLoS One - a journal that is both open access and has a philosophy of minimal peer review... "The philosophy is, let's get it out there and let the world decide ... That way, there is much less power in the hands of journals and it is a less distorted view of the world."

Ms Corbin also devotes space to the derivation of "Impact Factors" noting that: "Properly used, the Journal Impact Factor indicates relevant information about a journal as a whole, namely the extent to which its recently published papers are cited in a given year. It does not reveal anything concrete about a specific paper or a specific author published in the journal, and goes on to say: "Given the statistical evidence of the actual occurrence of citation across all papers in a journal, and the time span of the citation of individual papers, the relationship between a single paper or author and the impact factor of the publishing journal is tenuous."

Finally, here is a point made in one of the comments engendered by Ms Corbin's article: "Journals like PLoS ONE are...  explicitly excluding subjective criteria such as "impact", "novelty" and "importance" from their peer review criteria. At the same time they strive to maintain absolutely top-level critical review in terms of scientific rigor of submitted manuscripts.

__________________________________________________

*Zoë Corbyn specialises in research and science policy. She has worked at Times Higher Education since September 2007, prior to which she was a reporter for Research Fortnight magazine. She has an MSc in science communication from Imperial College and a BSc in chemistry from Murdoch University in Perth, Western Australia, where she comes from.