ARC Undertakes Review of Its System of Peer Review. (September 16, 2009)

On September 14^th the Australian Research Council (ARC) announced the publication of a Peer Review Processes Consultation Paper.

In making the announcement Australian Research Council Chief Executive Officer Professor Margaret Sheil said:

Peer review underpins the management of the National Competitive Grants Program, enabling the delivery of quality funding outcomes that maintain researcher confidence. It is therefore vital that ARC peer review processes are efficient and effective, and up-to-date with the changing research environment. I welcome your comments on the Consultation Paper, and will take them on-board to ensure that the revised processes instil confidence and allow the ARC to continue achieving its mission to deliver policy and programs that advance Australian research and innovation globally and benefit the community.

The ARC Peer Review Processes Consultation Paper includes a number of proposed changes to existing ARC peer review processes, including the:

• implementation of a revised assessor structure;

• introduction of new processes for assigning proposals to assessors, and

• development of alternative mechanisms for ranking proposals.

 I look forward to reading your comments, which will help the ARC in its continued support of the Australian Government’s investment in high-quality fundamental and applied research that produces benefits for all Australians.

One of the main elements for consideration is to give more weight to the views of specialist reviewers and in conjunction the ARC would consider new incentives to counter the chronic shortage of reviewers, such as reporting reviewer performance to universities to boost promotion credentials. And it is also looking into what it calls research opportunity and performance evidence (ROPE) which would "take into account constraints faced by some applicants, such as teaching load or career interruptions. "I am very keen to get reviewers to take into account opportunity for research as well as absolute track record," Professor Sheil told The Australian's Higher Education Supplement, and continued, "I want to make sure that the people in teaching-research positions, who are the backbone of our university system, have the same opportunity to demonstrate their research capability."

Commenting on the initiative the Australian Academy of Science secretary of policy Bob Williamson said: "At the moment the system doesn't really encourage novel or offbeat ideas, and the value of flexibility will be that new and younger scientists will have more of a chance; the challenge is to find ways to ensure that our national policies simultaneously fund the best science and also encourage early-career scientists, those who have had career breaks or taken unusual career paths, and those with very bright ideas who may not have a conventional track record."

In the view of Ken Baldwin, president of the Federation of Australian Scientific and Technological Societies (FASTS): "The paper recognises that the issues are not simply that of (the strains placed on) efficiency due to increasing applications but also go to fundamental problems including fairly evaluating track record across a wide range of research opportunities and outputs."

Professor Sheil emphasised to the HES' Andrew Trounson: "We need to get much better matching of the reviewers to the expertise and we need to put more weight on the most expert reviewers."  But she also hopes the ROPE approach might make the humanities and social sciences more competitive, given that few researchers in those disciplines are research-only.

And getting to specifics she told Mr Trounson that "in the case of Discovery Grants, the ARC is proposing that specialist reviewers each assess five to 10 proposals a year compared with a present average of two, and that each proposal receive three to five assessments. It estimates that it would need about 2000 to 4000 peer reviewers each year".

Unfortunately imbedded in the problems of significantly improving peer review, the assessment of research applications and the awarding of sufficient resources to adequately support meritorious proposals is the difficulty of obtaining the services of competent reviewers prepared to devote the necessary time, objectivity and care. Quite apart form anything else, the ARC must have placed at its disposal the resources to attain its goals. Frankly, with the Minister for Innovation, Industry, Science and Research preoccupied with ramming through his Excellence for Research in Australia (with an admitted $35 million allocation already earmarked to set it up) and university administrations shaking the money tree as hard as possible to gain the maximum possible for block grants, Professor Sheil is going to have to get some powerful political allies if she has any hope of having meaningful reforms implemented for direct and oncost funding of the guys actually doing the research.