|
Opinion- 16 July 2005 |
|
Change Can Equal Progress or Regression |
 |
Australia's Research Quality Framework
(RQF) was first
mooted in May 2004 when the Prime Minister, John Howard, announced that the
Australian
Government would establish Quality and Accessibility Frameworks for
Publicly Funded Research as part of the Backing Australia’s Ability – Building
our Future through Science and Innovation. The rationale for the exercise is
to distribute public research funding to support only the best research.
The conundrum of course is to determine who should pass judgement and on what
bases. Clearly the Coalition government is not satisfied that the instruments
currently in place such as the Australian Research Council, the National Health
and Medical Research Council, and the mechanisms set in place to oversee
research at the publicly funded research agencies such as CSIRO and ANSTO are
doing the job the way the government wants.
In the months following the Prime Minister's announcement the Minister for Education, Science and Training, Brendan Nelson, delivered up details of how his department would flesh out the Prime Minister's intentions with regard to devising an RQF. 1, 2, 3, 4, 5
Following an initial meeting of the thirteen member expert advisory group (EAG),
chaired by Sir Gareth Roberts, in
March this year, an Issues Paper was released late that month which elicited
165 submissions from interested
groups and individuals. This was followed by
approximately 370 people* attending the public consultation forums that were held
in each capital city during April 2005 and then the 61 page "Research
Quality Framework: Assessing the quality and impact of research in Australia
Advanced Approaches" was circulated for discussion at the June 2nd
Research Quality Framework Stakeholders Forum and
workshops held in
Canberra. According
to Professor Roberts it was "the result of a great deal of hard work by members
of the EAG and the small group of support
staff at DEST led by Anne Byrne. The next day the second meeting of the
Expert Advisory Group was held.
Of particular interest are five documents released by Dr Nelson following the forum and the meeting of the EAG the next day:
Perhaps the most important outcome of the forum, workshops and subsequent EAG meeting is the call for the de facto extension of the deadline for delivering a final advisory paper to the minister. The initial timetable called for the EAG to deliver its final report to the minister by December this year. The president of the Australian Vice-Chancellors' Committee, Professor Di Yerbury, summed the matter succinctly in her remarks:
On the issue of timing we very much agree with [the EAG Chairman] Sir Gareth [Roberts] that we should take more time to work through the implementation, the trialing, and what we learn from the trialing of this [RQF] exercise. It is more important to get it done right than to get it done quickly. As the Minister said, the implications of the RQF are very significant, and will reach out far into the future. Don't let's put the exercise at risk by rushing it.
In the summary of the forum's outcomes the last two points of agreement listed are the need for time to fully develop the RQF while also meeting the minister's expectations in reporting progress and recommendations by December 2005, and refinement of the details of the RQF model is to proceed in 2006 with implementation in 2007. The forum then suggests that core principles will be agreed by October 2005 with detailed design work and guidelines to be developed through 2006. Whether or not the process of refinement through 2006 will significantly involve individuals outside the employ of federal government's ministries is not indicated.
At this point it is instructive to examine the formal presentation made to the forum by the Chairman of the Expert Advisory Group, Professor Roberts, Dr. Nelson's address and then Professor Roberts' closing remarks. Of course those in attendance weren't to know that Dr Nelson had already decided to sack the Board of the Australian Research Council so that beginning in 2006 its CEO will report only to him.
Interestingly, so far as we know there are no data available to demonstrate that a Research Assessment Exercise (RAE) as practiced in the UK has had a beneficial effect on British research because the RAE has been coupled with marked increases in resources made available to the British research sector. Had the grant assessing mechanisms then in place continued and evolved without the influence of the RAE would British research improved less, as much or more? We don't know. As you will see below Sir Gareth states it has been very beneficial but conclusive data are lacking.
There is every indication that in the case of Australia there will be no significant increase in funding made available in conjunction with the introduction of a Research Quality Framework. And yet the man chosen by Dr Nelson to Chair the Expert Advisory Group told the The Australian Financial Review this past March 6,
I would be on a big crusade to say there should be additional funding. When you're introducing changes, you don't want to lower morale. You want to make sure the more research-intensive universities get more research monies that they can really use to good advantage to do leading-edge research, [but] at the same time, you want to make sure there is additional funding for the less research-intensive universities to develop a new mission to help their communities. [Australia needs to] double or triple [such] third-stream funding.
And so far, with all of Dr Nelson's posturing, and browbeating of the academic and research sectors his "reforms" have demonstrated no significant improvement in the functioning of the universities, or the publicly funded research agencies. Indeed the opposite may be suggested. That doesn't mean that alterations to the methods for assessing funding allocations aren't warranted; the academic and research communities have requested them for some years. For well over a decade the paramount problem has been, and particularly with the advent of the Howard government, an over reaching top down approach toward university and research funding coupled with a systematic diminution of support based on the nation's GDP, not withstanding that the government prides itself on fostering one of the world's most buoyant economies.
So is the RQF primary for use as another vehicle by the federal government to further reduce the independence of academics and researchers? If so it is another step in "mediocritising" the universities and research establishments, for the best will find havens elsewhere.
It is worth recalling that in 2001 when the Prime Minister, John Howard, anointed his new minister for Education, Science and Training, Brendan Nelson, he instructed him, or so Dr Nelson told his audience when he gave the Chalmers Oration at Flinders Medical Centre in Adelaide on July 17, 2003,
When you become a Cabinet Minister, the Prime Minister writes you ... a Charter Letter. It sets out the Prime Minister's expectations of what you will do and what will be the priorities for you in your portfolio. [I]n relation to Universities, [it] said that I should understand and enunciate the importance of higher education to the Australian community, and I should continue to progress workplace relations reform in the sector." [our emphasis]
To return to the present, or at least last month Professor Roberts told the Stakeholders Forum,
[Dr Nelson], like me and I am sure all of you here today, recognises that there has been an absence of a consistent approach to measuring research quality and impact in Australia. Without this assurance, the argument for further public investment in research is not as persuasive as it should be...
Minister Nelson has presented us with a marvellous opportunity to establish a robust framework that has the potential to achieve significant outcomes for Australian research and stimulate positive behaviours in universities and research organisations. We must help him get it right!
Presumably "get[ting] it right includes avoiding mediocrity and ameliorating the hysteresis engendered by the tendency to over value past performance, but how is that to be done...?
Professor Roberts went on to make the assertion,
The UK-RAE [Research assessment Exercise] is generally agreed to have had a significant positive impact. The exercise has driven a sustained improvement in the overall quality of the UK research base and has made a major contribution to maintaining national economic growth and international competitiveness.
And exhibited the following foil in support.
And added,
This has been fully recognised by the Treasury which has significantly increased its investment in research over the past 6 years.
Unfortunately it is not at all clear what contribution the RAE has had to improving British research as distinct from the government's increased investment in research per se.
As we've shown above, Professor Roberts himself, together with a number of others, has made the point that for the Australian government to institute an RQF without substantially increasing support for research will be an empty exercise.
In this context a paragraph from a July 14 Nature editorial which comments on a previously published Nature supplement on science in mainland China is pertinent in the light of observations made by Dr Nelson in his opening address to the forum and his decision to dismiss the ARC Board,
Some articles in the supplement are critical. One takes China to task for ploughing too much of its science budget into large, long-term, applied research projects at the expense of smaller-scale, basic science in which, the author argues, individual creativity can flower. Another, more pointed still, cites the lack of peer review in most grant allocation and calls on the Ministry of Science and Technology to surrender control of its science budget to organizations better able to distribute the funds where they are needed most. (In fact, the office of the Minister of Science and Technology turned down an offer to write a commentary for the supplement.)
As for Dr Nelson -- judging from the transcript provided by his department -- not all of his comments came from a prepared text. They are revealing of certain aspects of the Minister's makeup which appear to border on a siege mentality.
I notice quite a few of the vice-chancellors that are here today. I won't destroy your careers by saying many of you have become friends to me over the last three and half years...
...the RQF, in my opinion, is arguably the most important reform that we've got in front of you at the moment, and it's absolutely essential that we get it right. You will know much, much better than me what will be the consequences of not getting this right.
...it's difficult in this job finding time to reflect and think and have a sense, in the longer term, of where we're actually trying to go.
The real challenge that we've got... is to ensure that whatever reforms or change come from Research Quality Framework respect the finest traditions of research and science in this country whilst ensuring that, in every sense of the word, that we are well and truly in front of the world, at least on a per capita basis, in terms of the research we do and that which we fund.
It's worth, pointing out- my Department write a lot of speeches for me, and I don't think I've ever given one, - at least the ones they write for me. Some of you should be thankful for that, others probably disappointed.
I've worked out now that my job as your Minister, basically, for Education, Science and Training, is not to be an expert. I've got 1800 people that work for you in my Department who describe themselves as experts, to varying degrees.
My job, on the other hand, is to hopefully bring some intellectual rigour to the task of making decisions through the prism of what is it that we need in the long term in our country?
...every single dollar, every single dollar that we invest, whether in schools or universities, or research, or training, every dollar is a dollar that some Australian worked damned hard for, and we've got to make darn sure that every dollar that we invest delivers the very best outcomes for all Australians, ...at the moment, I cannot, with any confidence, tell the average Australian that every dollar we invest in research funds and supports the highest quality research in all circumstances.
Currently we are investing about $5.4 billion in research and development, which is 0.8 per cent of GDP, and the OECD average is 0.61. I think we rank
ninth in the OECD in that regard... $3.4 billion of it specifically is invested in research, and $1.8 billion of that goes directly into the higher education sector.
[But see the charts.]
[I was at an non research intensive university] and at the end of a meeting with the union representatives I was having a cup of tea and having a chat to them and I asked one of the fellows, I said, "Well, what do you do when you're not unionising?" And he said, "Well, I'm the Associate Professor of physics". And I said, "Well, what does that entail?" And he said, "I teach first year students and I do research".
I [thought] to myself, "How is it that we, as a relatively small country, can have obviously very high quality critical mass of research in [a] particular field of physics in one institution and then, I suspect, not necessarily of the same high quality in another?"
I am the first to accept that there can and there should always be an argument for increasing our research investment.
...before I can convince the people in central agencies in government [a persuasive case must be presented by you to support research] whether it is me, whether it is a person perhaps who might follow on from me when eventually, much to the joy of most of you, I'm moved on...
There's another thing... We had seven universities that received $83 million in competitive research funding [in 2003]. The licence return was $20,000 from one university, ...Not one of those universities employed a single person to work on the commercialisation of that research. ...The next seven universities received $178 million in competitive research funding for licence return of less than $475,000 ...each of them had employed, respectively, one person to work specifically on the commercialisation ...One university received $20,000 and three received less than half a million. ...basically the academic impact relates to a peer reviewed assessment of the impact of a piece of research on that particular field in terms of its originality or, indeed, the impact on a related field.
I know many of you are sitting there thinking, "Well, the last person we want telling us about efficiency is the government and particularly Nelson and his department". But, nonetheless, that's got to be at the heart of it as well, coming back to the average everyday person whose money it is that we are talking about investing.
My department was apoplectic because I'd actually said in my remarks [to FASTS Science Meets Parliament initiative] that the RQF will inform the distribution of money. I mean, for goodness sake, why on earth would we go through this exercise if we don't at least countenance the notion that it will inform the distribution of research funding.
[One of the purposes of this forum is] to do what might well be regarded as impossible and that is to see if we can develop some kind of broad consensus about the approaches which might be trialled between July and September this year.
...it is possible that the end of the RQF, when it is finally applied, that some universities may not attract research funding on the basis of quality, as defined by Australia's research community. ...It's also possible that some universities that would not currently be regarded
necessarily as being research intensive will attract more funding.
...one of the things I will not under any circumstances countenance... is anything... that does anything other than enhance and strengthen the role of humanities and social science research in this country. ...because what is done in applied science is extremely important but what is arguably more important and much more difficult is how we understand the consequences of knowledge. How we live in vast long term ignorance of its consequences.
Then there are the ten questions posed in the Research Quality Framework: Assessing the quality and impact of research in Australia Advanced Approaches paper:
What should be the unit of assessment?
What should be the Unit of Reporting (UoR)?
How should the broader impact of research be assessed?
How should research training (RT) be incorporated into an RQF?
Who should be included for assessment in an RQF?
How should an RQF address the situation of early career researchers?
What are the common elements of an RQF that could apply across universities and PFRAs?
How can an administratively efficient RQF be achieved?
Should an RQF include assessments of an institution’s research strategic planning (RSP) and, if so, how?
How should an RQF be applied to research funding distribution?
Following the Forum and the two workshops Professor Roberts closed the day with a summing up which he "found difficult". However, some of his comments ought to give the "stakeholders" pause.
I think when it comes to strategic planning it seems to be that we're not too far from agreeing there that we do need strategic planning statements to put a whole institutions work into context so that when a unit submits its proposal, there will be some sentences in the overall strategic plan of the institution relating to that particular unit.
I think, in order to get your academic colleagues on board, ...try and create a special group that will look at multidisciplinary activity to check other things that you could do which would could probably lead the world on this, because it's not an easy situation to look at.
The modelling is absolutely crucial and this is why I think it's the biggest message we have to give to the Minister. Those trials are going to make a difference.
On funding issues, I think we're more or less agreed. Certainly the AVCC agreed with EAG that IGS and RTS needs to be influenced by the RQF but not the Research Infrastructure Block Grant. Capability funding was mentioned and that really is, I think, a good thing to do. In Britain it's top sliced out of the quality money, but there may be a case to be made for extra money here in Australia.
To paraphrase Horace, The mountains are in labour but is a stillborn mouse being brought forth?
*The Federation of Australian Scientific and Technological Societies estimates that those societies number some 60,000 members. 370 individuals represents 0.62%.
Alex Reisner
The Funneled Web
