News & Views item - July 2011

 

Peer Review Across the Atlantic. (July 29, 2011)

A couple of recent articles in  ScienceInsider deal with a UK House of Commons Science and Technology Committee report on Peer review of scientific publications while earlier this week the US House of Representatives Subcommittee on Research and Science Education held a hearing on proposed changes to the National Science Foundation's (NSF's) merit review process which if implemented would require research  to benefit society beyond merely advancing scientific knowledge.

_______________________________________

 

As regards the House of Commons report it comments: "In view of the importance of evidence-based scientific information to government, it seemed appropriate to undertake a detailed examination of the current peer-review system as used in scientific publications [and while]...it is considered by many as important and not something that can be dispensed with [t]here are, however, many ways in which current pre-publication peer-review practices can and should be improved and optimised..."

 

The Committee singled out "the use of pre-print servers, open peer review, increased transparency and online repository-style journals" for consideration "to reduce the necessary burden on researchers, and also help accelerate the pace of publication of research", but go on to endorse "post-publication peer review and commentary [which] represents an enormous opportunity for experimentation with new media and social networking tools".

 

And they also concerned themselves with the use of "journal Impact Factors" which they see as having "a direct effect on the careers of researchers and the reputations of research institutions. We therefore have concerns about the use of journal Impact Factor as a proxy measure for the quality of individual articles... We have heard in the course of this inquiry that there is no substitute for reading the article itself in assessing the worth of a piece of research."

______________________________________

 

Last month the US National Science Board (NSB), the governing board of the National Science Foundation (NSF) proposed draft changes to the Merit Review Criteria for reviewing research proposals. In particular the changes proposed for the so-called "broader impact" criterion "which requires that research have benefits to society beyond merely advancing scientific knowledge".

As it stands researchers are currently asked to specify how their research will benefit society in the form of education and outreach, enhanced research infrastructure, and/or broadened participation of underrepresented groups. However, according to ScienceInsider the draft proposal "lists nine 'national goals' that projects should strive to advance 'collectively', including increasing national security and economic competitiveness".

 

Fronting the House of Representatives Subcommittee on Research and Science Education on July 26 Keith Yamamoto, vice chancellor for research at the University of California, San Francisco, emphasized the importance of using the merit review committees to judge proposals solely on their scientific merit.  Professor Yamamoto told the Committee that it was inappropriate for reviewers to be asked to "step outside of their areas of expertise" and to "make guesses" if proposals are meeting national goals. Furthermore he believes it is impossible to make such assessments at the level of individual projects. In short that should be the job of governmental funding mechanisms and the mission-driven agencies that support research in areas such as health, environment, or national security -- not the NSF.

 

Professor Yamamoto, who has always been opposed to NSF's broader impacts criterion, told ScienceInsider after the hearing that the proposed changes will only make things worse by: "threaten[ing] the foundation of really fundamental investigation" whose broader impact cannot be determined from the outset. "I'm very concerned about the proposed changes."

 

 

ScienceInsider went on to report:

 

But two other witnesses, Jorge José, vice president of research at Indiana University and Nancy Jackson, president of American Chemical Society, were more supportive of the broader impacts criterion. Both agreed that other criteria besides scientific merit are needed in order to distinguish between the many competitive proposals, since not all of them can be funded.

The NSB task force expects to finalize the new criterion this fall. The recommendations will then be presented to the full NSB for approval at the board's December meeting, an NSF spokesperson says.