News & Views item - March 2013

 

 

Do You Really Want to Become a Scientific Researcher? (Msrch 21, 2013)

While the conservationists are exercised as to the rate of extinction of the planet's flora and fauna, there is upon us the age of the Bureausaurs.

 

Rather than putting in a worthwhile effort to improve the design, implementation and evaluation of research grant proposals, the National Health and Medical Research Council (NHMRC) exhibits a sclerosis which gives every indication of being subject to Newton's 1st Law, while universities have manipulated the worse-than-useless ERA -- which not only squanders millions of dollars but usurps thousands of hours of researchers time.*

_____________________________________________________

 

 

Jump in ERA Scores the Product of Careful
Manicuring of Departmental Submissions

NHMRC Grant Submission Bloat Seen as
Costly Waste of Time and Funds

                 

             Bernard Lane in the 20 March 2013 Australian


CLEVER reporting, rather than a surge in knowledge, helps explain the surprisingly good results of the Excellence in Research for Australia [ERA] quality audit, new analysis shows.

"It's hard to believe there's been a dramatic increase in quality in such a short time, but there's no doubt universities have managed their submissions more strategically," said Frank Larkins, the former University of Melbourne research chief who did the analysis. He called for a review of the rules before the next ERA, set for 2015, and "a serious policy debate" about use of the results to target research funding.

According to last year's ERA outcome, most universities increased their overall excellence ratings, despite this round adding only two fresh years to the six-year audit period covered by the first ERA in 2010.

Institutions were able "to optimise their performance" thanks to more flexible rules for assigning outputs among "four-digit" codes of research, Professor Larkins said in his LH Martin Institute paper.

The ERA assesses outputs in two-digit and four-digit fields of research, the latter being more specific. Last year, most universities ended up being assessed in fewer four-digit codes than in 2010.
One reason was a higher threshold for outputs before any assessment took place; but it was also due to "some universities not making as many submissions in four-digit (codes) where they scored below world standard in the 2010 round", Professor Larkins said, adding that 23 of 40 universities ended up with fewer assessments at the four-digit level, and the University of Western Australia was "the standout".

Its four-digit level assessments fell from 84 in 2010 to 62 last year, and the university "achieved 100 per cent of its ratings at or above world standard in 2012 compared with 76 per cent in 2010", he said.

Last year, the universities of Sydney, Melbourne and Queensland all were assessed in more than 95 four-digit codes.

UWA deputy vice-chancellor (research) Robyn Owens said universities had gone into the first ERA with a poor understanding of how it worked.

And the rigid rules in 2010 for assigning outputs to research fields meant the full impact of a highly cited paper could be lost. "It was clearly strategically advantageous to try to focus your research rather than spread it thinly over a vast number of (four-digit) codes," Professor Owens said.

Letter in Nature 21 March 2013 by
 Danielle L. Herbert, Adrian G. Barnett & Nicholas Graves

Queensland University of Technology, Brisbane, Australia.

We found that scientists in Australia spent more than five centuries' worth of time preparing research-grant proposals for consideration by the largest funding scheme of 2012. Because just 20.5% of these applications were successful, the equivalent of some four centuries of effort returned no immediate benefit to researchers and wasted valuable research time. The system needs reforming and alternative funding processes should be investigated.

 

We surveyed a representative sample of Australian researchers and found that preparing new proposals for the National Health and Medical Research Council's project grants took an average of 38 working days; resubmitted ones took 28 days on average. Extrapolating this to all 3,727 submitted proposals gives an estimated 550 working years of researchers' time (95% confidence interval, 513–589), equivalent to a combined annual salary cost of Aus$66 million (US$68 million). This exceeds the total salary bill (Aus$61.6 million) at Melbourne's Walter and Eliza Hall Institute, a major medical-research centre that produced 284 publications last year.

 

The grant proposals we analysed were typically 80–120 pages long. If these were more focused, it would reduce preparation costs and could improve the quality of peer review by reducing workloads.

 

 
   

_____________________________________________

*Currently there is no indication that the Australian Research Council is any better as regards "putting in a worthwhile effort to improve the design, implementation and evaluation of research grant proposals".