The ERA Elicits Discussion -- Robin Batterham Has a Suggestion. (March 2, 2010)

The first issue this year of the Australian Universities' Review (vol.53, #1) contains a 7-page paper of  "A ten year citation analysis of major Australian research institutions" by Robin Batterham,  University of Melbourne's Kernot Professor of Engineering, current president of the Australian Academy of Technological Sciences and Engineering (ATSE), and former Chief Scientist for Australia from 1999 ˗ 2005.

Professor Batterham sums up his assessment of the the citations listed by Thomson ISI Essential Science Indicators 2000 ˗ 2009:

The introduction of the Excellence in Research for Australia scheme has heightened debate amongst research institutions over the use of metrics such as citations, especially given the ready availability of citation data. An analysis is presented of the citation performance of nine Australian universities and the Commonwealth Scientific, Industrial and Research Organisation (CSIRO) that indicates that Australian Institutions perform significantly better than the global average. That said, the question is raised as to whether we are setting the bar too low. Finally, a tentative link between citation performance and application to innovation is noted.

Now the nations in the world number around 195 so saying that Australian Institutions perform significantly better than the global average isn't really saying all that much. How we perform compared say to the top half of OECD nations may be rather more meaningful.

Professor Batterham is circumspect in his paper's conclusion:

 An argument is outlined that high rankings on citations are an indicator of more effective innovation. The analysis of a ten year performance window for ten of the top publishing institutions in Australia certainly supports the claim that Australia’s performance is well above average.
An unanswered question logically follows of how much better could we do and whether this entails focusing effort on our top performers and those that are close to the top. This is a topic that demands a deeper analysis than this paper, with its aggregation at an institutional level. (emphasis ours)

However, Jill Rowbotham in her article in today's Australian quotes Professor Batterham:

We should concentrate more of our research funding on those that are performing at the higher levels... the relatively large number of institutions publishing in particular fields and ranking well below world average performance raises interesting questions. Uncomfortable as the question is, should our limited research funds be used this way or are we better bringing those near the top (say, the top 25 per cent) up to the top 10 per cent?

Excellence in research has many advantages. In particular, it facilitates more international collaboration, and hence more impact and innovation. So I ask: why should we fund research that is less than excellent? The proposition is hard to sustain so we are left with the uncomfortable position of concentrating research funds at the excellent end of the spectrum.

Professor Batterham's paper addresses the matter of scientific literature citation as determined from Thomson Scientific compilations. He notes that on the basis of citations within the group of the ten Australian institutions he assessed (the Group of Eight Universities, the University of Newcastle and CSIRO) the relatively large number of institutions publishing in particular fields and ranking well below world average performance raises interesting questions.

With the exception of the publications from CSIRO the vast majority of the funded research projects were proposed by principal investigators and were peer reviewed, for example, by the ARC or MHMRC. Surely if the research quality is wanting, it is the peer review system dealing with the submitted proposals that needs improving. Imposing an ERA as a sledgehammer with which to reshape university research is ludicrous.

Institutions do not do research, university administrators (with rare exceptions) do not do research. As long as the resources for research pass though the current multistage system before reaching the primary source of creativity Australian research will, at best, remain in its current relative position. The ERA with its emphasis on retrospective analysis and apportioning funds to institutions which should go directly to researchers will simply exacerbate matters.

Why should the brilliant research group at a minor university be penalised if its peer groups within the university are mediocre?  If it's argued that it will be for the common good. That's blatant rubbish.