The Conundrum for the Peer Reviewer. (September 23, 2010)

Kendall Powell, an American freelance science writer, writes, in the latest issue of  Nature, of the tribulations of peer reviewers assessing grant requests received by the American Cancer Society (ACS).

One key element in her article has relevance for all granting bodies and is eminently relevant for the Australian Research Council and the National Health and Medical Research Council.

Reviewers say that they feel forced into making impossible choices between equally worthy proposals, especially when success rates are less than 20%.

"That's in a range where you have lost discrimination," says Dick McIntosh, professor emeritus of cell biology at the University of Colorado in Boulder. "That's a situation where you are grading exam papers by throwing them down the stairs."

The chairman of the ACS [review] panel agrees. "Deciding between the top grants, I don't want to say it's arbitrary, but it's not really based on strong criteria," he says. "It's subtle things."

At the ACS, the largest private non-profit funder of cancer research in the United States, the average success rate for grant applications has slipped by a few percentage points in the past two years to roughly 15%, owing largely to fewer donations — the organization's sole source of income — in the economic downturn.

At the National Institutes of Health (NIH) in Bethesda, Maryland, which funds the majority of biomedical research in the United States, several years of flat federal funding combined with a rise in the number of applications means that 21% of research-project grant applications were funded in 2009, down from 32% ten years earlier.

The chairman of the ACS review panel told Ms Powell: "It makes me sad that people who are really strong are struggling to get funded."