British Academy's Apologia for Peer Review. (September 4, 2007)

      The Guardian reports the British Academy, the UK's national academy for the humanities and social sciences, today issued a report in support of peer review with a quote of Joan Sieber, a psychologist at California State University: "One suspects that peer review is a bit like democracy - a bad system, but the best one possible."

According to Albert Weale, professor of government at the University of Essex and chair of the committee responsible for the report, peer review is "the essential backbone to knowledge and the crucial mechanism in maintaining its quality", while Robert Bennett, professor of geography at Cambridge University, says it is "an essential, if imperfect, practice for the humanities and social sciences".

And in defence of the system it responds to critics who claim peer review is biased against innovation and that journal editors "play safe" and are "friendly to their own work", the report points to universities and research councils who are increasingly awarding grants for risky, avant-garde research projects.

It also warns to be wary of false assumptions, i.e.: "It is important not to commit the fallacy of assuming that, because high quality will be innovative, that innovative is necessarily high quality ... other criteria include: accuracy, validity, replicability, reliability, substantively significant, authoritative and so on."

Marian Hobson, professor of French at Queen Mary, University of London, says: "If done properly, [peer review] entails bibliographical searches, checking of statements, repeated visits to the university library, not just to Google. Yet this kind of activity counts for nix, nothing, zilch in the research assessment exercise [in which every active researcher in every university in the UK is assessed by panels of other academics to receive grants for their research]."

In one suggestion the report recommends that the importance of peer reviewing should be better reflected in the research assessment exercise. "Those responsible for the management of universities and research institutes need to ensure that they ... encourage and reward peer review activity," it says.

It goes on: "As we conducted our review, we were struck ... by the extent to which there is little attention to training in peer review. Training is important, not just in itself, but because of the privileged position that peer reviewers enjoy.

"By virtue of reading a paper, reviewers can acquire access to original data sets, new empirical results or innovative conceptual work. In the business world, these would count as commercial secrets. In the academic world, the ethos is that reviewers are part of the gatekeeping system, the ultimate rationale of which is the fast and efficient dissemination of research findings.

"The integrity of the peer review system is therefore of great importance. One of the ways in which that integrity is maintained is through its dependence upon professional and unselfish motivations, and this in turn suggests the importance of training in the professional and ethical conventions of the practice."

The British Academy reports concludes with a warning to the government: plans to overhaul the way research is assessed after next year will change peer review for the worse, especially in the humanities.

And Marian Hobson says: "Metrics [which is expected to be the backbone of Britain's RAE after 2008] is helpful in giving a kind of overview measured in terms of items. A bit like a waistline measurement. It doesn't give much of an idea of whether they are slim or fat, unless they are at the extreme ends of the spectrum."