News & Views item - December 2007 |
NIH Peer Review System Under Critical Scrutiny. (December 8, 2007)
Credit: UCSF
and ScienceNOW Cause for celebration? UCSF's Keith Yamamoto is leading a committee to re-imagine peer review at NIH. |
The US National Institutes of Health last examined its system of peer review 8 years ago but not all types of grants were covered.
This time biochemist Keith Yamamoto, executive vice dean of the University of California, San Francisco, School of Medicine, told ScienceNOW: "Elias [NIH Director Elias Zerhouni] said, 'Look at the whole thing,' ".
This past week at a meeting held at NIH headquarters in Bethesda, Maryland the members of the advisory committee to Dr Zerhouni debated everything from doing away with the current scoring system on grant proposals to incentives that might improve the quality and motivation of reviewers.
The fact is the peer review at NIH as elsewhere is under increasing strain. For example Mary-Claire King, a geneticist at the University of Washington, Seattle, who was not a member of the peer-review team told ScienceNOW: "'we are desperately worried' about new investigators, who find it hard to land grants and may turn away from science."
A request from the NIH for submissions from interested parties elicited over 2,600 responses.
Some of the suggestions:
develop an "editorial board" model, in which grant proposals that contain certain technical details could be sent to experts who would consider those elements alone and report back to the study section. This could help reduce the size of study sections, which have ballooned to as many as 80 people to accommodate all the expertise needed;
encourage reviewers to be more direct about proposals that seem hopeless and seek fewer revisions;
have each reviewer rank only the top 10 applications rather than scoring them all;
find ways to get and keep the best reviewers in the system -- including permitting reviewers to rate the reviews of their colleagues, which they can’t do now.
brief reviewers on work NIH already funds to help avoid overlapping grants, "Are we studying the same protein head to toe?" Lawrence Tabak, director of NIH's National Institute of Dental and Craniofacial Research and co-chair of the internal and external working groups asked rhetorically.
ScienceNOW concludes: "For those applying for grants, possibilities being seriously considered include:
streamlining applications from 20 pages to seven, essentially doing away with preliminary data and instead focusing on a project's potential impact, and
allowing resubmission of grant proposals only rarely, rather than the usual two times permitted today.
Now, say Yamamoto and Tabak, firm recommendations need to be presented to Dr Zerhouni by February.
It might be of interest and even useful for those in the newly elected Australian government who are charged with relegating its predecessor's Research Quality Framework as well as the administrators of the ARC and NHMRC to see just what those 2600 submissions received by the NIH contain.
Note added December 17, 2007 -
[Your readers] might be interested that we too
ARE looking at this, have just had Toni Scarpa from NIH here (he runs their
external review process) and will have the Director of NIH, Dr Elias Zerhouni,
here at NHMRC in January to help further with this task. Like NIH, I think
there are ways we can do this better!
Professor Warwick Anderson
Chief Executive Officer
National Health & Medical Research Council