Australian Technology Network of Universities Employs RAND Europe to Assess Research Impact Assessment. (April 3, 2013)

The Australian Technology Network of universities (ATN)*employed Britain-based RAND Europe to analyse the cost effectiveness of assessing research impact on defence, economics, environment and society.

Andrew Trounson in today's Higher Education section of The Australian reports RAND Europe as concluding that a full assessment of Australia's 39 universities "could require up to 1500 case studies costing up to $15 million [and to] justify that cost, about $150m in reward money would have to be distributed [which is] more than double the present funding allocated on assessments of research performance".

Going into further detail: "Rand estimated the cost of preparing a case study and assessing it would be $5000 to $10,000 each. Based on the British impact assessment and adjusted for population and the number of universities, RAND estimated that up to 1500 case studies could be submitted under an Australian version, or 38 a university. The total cost would be $7.5m to $15m."

Vicki Thomson, the Executive Director of the Australian Technology Network, told Mr Trounson that it's premature to be focusing on cost when the policy value of assessing impact still needed to be established: "We have to demonstrate first that it is good policy," Ms Thomson said. "Taxpayers must be satisfied that research output is value for money. Research must have impact for the economy and for people's lives, and that impact must be adequately assessed using a relevant methodology."

However, while the report states: "We believe the Australian higher education community is in the forefront of global discussions (on research impact) and further advanced in thinking through ways of how to engage in these efforts," it also noted as Mr Trounson reports that "the four areas of assessment -- defence, economics, environment and society -- might be too broad. Each of the four panels would have to assess on average 375 case studies in a scaled-up exercise".

Furthermore, RAND Europe warned that the universities could game the assessment say by "cultivating media coverage to coincide with assessment periods, and rushing or delaying research to fit assessment periods."

And finally, Mr Trounson points out that the Australian Research Council's chief, Aidan Byrne, has expressed concern at the potential cost of scaling up case studies and has suggested using proxies for impact.

Or should the ARC concentrate on improving its peer review system in order to select for support the best research.

_____________________________

*Curtin University, Queensland University of Technology, Royal Melbourne Institute of Technology, University of South Australia and the University of Technology, Sydney.