Australian Technology Network Universities Want Impact Back in the Mix for Judging Excellence. (July 27, 2011)

To quote the Silver Blonde when she stuck her head around the door this morning: "It's a mad world".

Way back during the last decade when Kim Carr was shadow minister for science and a few other things he rubbished the Howard Coalition's championing the judging of  the impact of research on the national economy as a criterion of its Research Quality Framework. With the advent of Senator Carr becoming Minister for Innovation, Industry, Science and Research and the promulgation of Excellence in Research for Australia (ERA) the use of impact was removed from consideration.

As the years rolled by and the outcome of the 2010 trial of the ERA was published, the Australian Technology Network (ATN) universities were less than overjoyed with the results. After all they tended to emphasise applied research over fundamental research.

Senator Carr who, like Margaret Thatcher, is not one for easy turning  warned according to  The Australian's Andrew Trounson, that attempts to measure impact could be labour-intensive, unreliable and, in the case of the research quality framework, vulnerable to rorting. "Experience has shown that measuring impact is difficult and highly subjective."

Of course experience has also thrown in to doubt the validity of relying on citation data. According to the Higher Education Funding Council for England's recently released Assessment framework and guidance on submissions assessment panels may not use citation data as a proxy for evaluating submissions through reading them. Furthermore, the ranking of journals for judging research quality is also deemed improper.

Now the ATN has launched a new drive to have impact considered as part of research assessment in divving up block research grant funding (England's Research Excellence Framework is to have 20% of its scoring based on impact). And the Group of Eight (Go8) universities is coming in on the approach. Go8 executive director Michael Gallagher told Mr Trounson his members were keen to be involved. "We are working with the ATN to explore the use of appropriate indicators and case study models to make more visible the contributions of research to economic, environmental and social purposes," which has something of the ring of "let's keep the bustards honest" about it.

In deputy vice-chancellor (research) of Queensland University of Technology, Arun Sharma's view: "We feel that at some point there will be a policy shift in this direction and the ATN wants to refine some of these mechanisms and contribute to the debate." and with remarkable creativity the ATN has christened it Excellence in Innovation for Australia, a companion indicator to the ERA.

According to Professor Sharma the best way to measure impact was to have researchers make qualitative statements, such as through case studies, that could be informed by quantitative measures and then assessed by an independent panel. The panel could include relevant stakeholders such as industry. And a proposed trial would seek to measure environmental, social, cultural and economic impacts. Such a broad measure was needed to fairly assess impact in disciplines such as the humanities and social sciences.

And as Daine Alcorn, RMIT University deputy vice-chancellor (research), sees it, it's important to get support for the effort across the the sector. It also could be open to research bodies such as the CSIRO. Professor Alcorn said having a measure that sat outside ERA might be more palatable to the Australian Research Council.

However, as Mr Trounson sums up: "it is unclear what body would administer the EIA".

On the other hand why not start from scratch and improve the system of peer review; we would suggest making use of modern communications technology for openers and providing adequate funding to grant recipients where grants are awarded on the basis of a proposals merit.

Isn't it well past time to give primacy to the researcher rather than the institution?

In February last year TFW asked:

Will the now sometime presence of Australia's quiet, but plain speaking, 2009 Nobel Laureate, Elizabeth Blackburn, stir our governmental representatives to up the proportion of carrot to stick when moving to improve the country's tertiary education / research sector?

"I think there are tremendously good scientists in Australia but sometimes I just feel, are they really being able to run with it in the way they are capable of?" Professor Blackburn asked rhetorically of Mr Trounson, and went on to say that in the US, she benefited from a five-year grant that allowed her to follow her nose without having to write up "damn little" reports and catalogue milestones on a regular basis. "This was the perfect setting and I'm not aware that I would have been able to do that [here]." And to make sure she was getting her message across: "Short dribs and drabs" of money with tight constraints on basic research are in her opinion wasteful.

If the welfare of Australia's citizenry in the 21st century will be dependent on a knowledge economy the evolution of its educational system needs radical improvement and that can't happen unless the resourcing of its tertiary sector [in the broadest sense] is markedly and sustainably improved.

Anyone for tennis?