CEO of the Australian Research Council Cautions: Beware the Efficacy of Reward for Adjudged Impact. (March 13, 2013)

With Apologies to Baroness Orczy

They seek it here, they seek it there,
Those bureaucrats seek it everywhere.
Is it in heaven? — Is it in hell?
Those damned, elusive IMPACTS.

This past July Aidan Byrne took on being Chief Executive Officer from Margaret Sheil of the Australian Research Council (ARC). Prior to taking up the role his research at the Australian National University involved the use of gamma-rays as probes to determine the structure of heavy nuclei and as probes in the examination of the atomic level structure of materials (especially semiconductors); he has published over 200 papers.

By and large Professor Byrne has over the past 8+ months held the ARC on course with perhaps his most notable achievement bringing the ARC in line with the NHMRC and CSIRO in mandating a 12 month deadline for giving open access to publications based on ARC funded research.

He has also given qualified endorsement to what The Australian reported as: "developing some centralised, organised attempt to measure the economic and social impact of research."

Today The Australian's Andrew Trounson reports Professor Byrne is "wary of attaching too much in the way of reward payments to the ERA [Excellence in Research for Australia] or impact because such measures were imprecise. To make significant funding contingent on performance would be unfair".

The ARC head went on to say:

You have to be careful if you link a lot of money to exercises like this. In order to be fair you have to make the measure very precise and that makes them very large and complicated. If possible you want to do these exercises in a way that best uses the information that we have available already. The case studies will require additional collection of information, so should one chose to use it, one should do it in as minimalist a way as possible. I see it as a component of measuring impact, but again you could probably collect or gather together other information that we already have and do an exercise with a much lighter touch.

The ARC head then told Mr Trounson that:. "the ARC and the Department of Innovation would release a discussion paper on measuring impact within the next month or so. To that end the discussion paper will canvas ideas to assess not actual impact but what Professor Byrne called pathways to impact. These could include measures such as the number of PhD graduates universities produced and the number of contracts and grants university staff had with business and industry."

The atomic physicist's parting comment:

You do want to be able to cross-correlate the research activity profiles with the impact profiles, so you need a common framework on which to put them. We have a diverse sector and forcing universities to be all measured against a single yardstick (ERA) isn't doing them justice.

We can only hope that the bureaucratic obsession with finding the apis philosophorum of research impact -- thereby defining them the equivalent of 21^st century alchemists -- won't cripple Australian research.