Brendan Nelson may well be best remembered for the necklace of millstones he bequeathed to Australia's higher education and research sectors at the behest of his Prime Minister, John Howard.

The object of Dr Nelson's "reforms" was never to improve learning and research at the nation's universities but rather to break them into submission and reduce the percentage of GDP the government expended on the higher education sector to a minimum.

Their utility is seen to be nothing more nor less than as training farms for the professions and a skilled workforce. And to that end they are to be teaching institutions foremost and research, such as must be tolerated, is to be focussed on applied research and development as tightly as possible.

Protestations that the government is perusing policies that are the surest way to strangle progress in Australian research, innovation and development are summarily dismissed.

The setting up of the Research Quality Framework, despite any denials to the contrary, is primarily an exercise to accrue greater control of university and research funding than is already available to the federal government. So for example the peer review system used to award research funding by the ARC and NH&MRC is not as easily controlled as the government would like. The noise and flack generated through Dr Nelson's vetoing of ARC research grants, his packing of the ARC's quality and scrutiny committee in order to hold the ARC's college of experts in check and the sacking of the ARC board are witness to a less than subtle approach to straight jacket the ARC.

And finally of course Dr Nelson made it known that he would be using the RQF to shape the ARC's and NH&MRC's  granting systems despite earlier precluding the possibility.

It requires only a short assessment of the "Preferred RQF Model" to understand the power it puts into the hands of the Minister for Education, Science and Training.

If the minister is a benevolent despot then you have an expensive exercise (direct costs are said to be ~$40M, indirect costs, e.g. researchers and university administrators time would be additional) which is in the process of being discarded by Great Britain, the only nation which has implemented a comparable system, the Research Assessment Exercise (RAE).

If the minister tends to have Nelsonian tendencies higher education and research are in serious difficulty.

Present at the press conference given by the Minister for Education, Science and Training, Julie Bishop and the Chairman of the Expert Advisory Group (EAG), Sir Gareth Roberts, where further steps to deal with the RQF were announced, was Bradley Smith, executive director of the Federation of Australian Scientific and Technological Societies (FASTS). He reported in advice to the federation:

At the media conference there was a number of questions about the UK dropping the RAE. The response from Minister Bishop and, in particular Sir Gareth Roberts, was that the RAE has operated for nearly 20 years and driven considerable cultural change. Thus, the implication being RQF should run for several iterations, drive the policy objectives/cultural change and later introduce a metrics system to takeover the RQF role.

Contrast this response to that of Michael Stirling, Chairman of the Russell Group, writing in The Guardian on March 23:

 Here we have an opportunity to revise an assessment system, which, for some subjects, may be unnecessary. Simplifying the process is long overdue as the RAE is well past its sell by date: it tells neither universities nor government anything we don't already know; it is hugely expensive and massively time consuming.

 

The direct costs of the RAE itself already stand at an acknowledged £40m [A$98m], while estimates suggest the real costs are up to 10 times that much - £400m - because of the amount of academics' time spent in the assessment process, related administration and unproductive gamesmanship.

 

It is my belief that the areas of science, technology, engineering and medicine would be keen to abandon the RAE and use research grants and contract income as a proxy. Government funding might then be awarded on a different basis: an assessment of how successful institutions had been over a track record of research grant and contract income in these key subjects.

 

For many years now, it has been obvious that there is consistently a direct correlation between research grants and contract income and excellence in research.

 

This review gives us the opportunity to stop wasting time and money on the RAE process, at least for some areas.

 

If you look at a league table of RAE ratings and match that with a league table of year on year achievements in research grant and contract income, institutions are mostly in the same place. A new system based on those existing measures is financially and academically robust for many subjects, has an established track record of accuracy, saves millions and saves time.

And see Gareth Roberts - the RAE - the RQF - and the Chancellor of the Exchequer, Mr Brown.

Repeated "leaks" emanating from the Expert Advisory Group indicated that there were strong, not so say irresolvable, differences among members of the EAG so that the preferred RQF model as finally presented to the then Minister (Brendan Nelson) was little more than an outline leaving much to be determined by the Minister.

Ms Bishop is far too canny to get lumbered with being solely responsible for filling in the gaping holes left by the EAG and has designated an RQF Implementation Group (RFQIG) which has now been renamed by her the RQF Development Advisory Group (RQFDAG). "The group will provide advice on the next phase of the RQF process, particularly how the model, if adopted by the Government, could be most effectively implemented." - Antonio could have done a worse than having Ms Bishop as his Portia.

The RFQIG / RFQDAG which will be Chaired by the Chief Scientist, Jim Peacock, will have as its duties if it sticks to the "Final Advice on the Preferred RQF Model":

But keep in mind it is now designated as an advisory group rather than an implementation group.

The Chairman of the Group of Eight, Melbourne University's Vice-Chancellor Glyn Davis says,

We remain concerned about a number of the EAG’s recommendations or lack thereof. These include the failure of the EAG to deal with crucial issues of the relative weighting to be given to research quality and research impact, the lack of detail about the amount of funding to be distributed on the basis of RQF performance, the funding mechanism to be applied, how universities will be compensated for the substantial administrative costs they will incur and the long lead time recommended for the introduction of any knowledge transfer initiative that might run alongside the RQF. We look forward to working with Dr Peacock’s committee to resolve issues such as these.

 

A prudent interim step before moving to a full RQF would be for the Government to conduct a detailed analysis of available research performance metrics - much like the UK Government has announced will run alongside the 2008 Research Assessment Exercise. It may be that Australia can move to a far simpler and cheaper, metrics-based approach much more quickly than has occurred in the UK.

And as Professor Davis points out, "Ultimately the decision to implement an RQF will be one for the Minister and the Government. It is important that the best possible advice is used as the basis for that decision."

The complexities of the preferred model really have to be seen to be believed and the monetary and temporal costs both immediate and ongoing ought to be considered prohibitive but then the real reasons for the government's interest in its implementation are anything but transparent.

The journal Nature in its featured March 30 editorial comments on UK Chancellor of the Exchequer Gordon Brown's budget statement regarding the merging of the Medical Research Council and the National Health Service as well as the scraping the two decades old Research Assessment Exercise.

In regard to the MRC - NHS merger Nature emphasises the point, "It is vital that the new biomedical-research body has an arrangement to protect it from political interference by ministers." The same would seem to be equally appropriate to Australia's ARC and NH&MRC.

With regard to the proposed termination of the RAE Nature has this to say:

 

Gordon Brown with the UK's 2006 Budget

    Thanks in part to Brown's conviction that research is key to economic growth, the Labour government has upped annual science spending by about 70% since it came to power in 1997, to more than £2.5 billion (US$4.4 billion). There were no spending increases this time. But Brown, who is expected to succeed Tony Blair as prime minister at some point in the next two years, has clearly been giving some thought to how British research could be strengthened.

 

Researchers will also be glad to see the back of the Research Assessment Exercise, a mechanism to measure university departments' performance, whose expiry after a final outing in 2008 was confirmed in Brown's budget statement. The exercise, which has taken place every seven years or so since 1985, has helped the government to determine the levels of fixed funding, on top of research grants, for Britain's universities. But it is vastly time-consuming for overseers and overseen alike, and its usefulness as a management tool has been gradually waning.

 

Brown is proposing to replace the Research Assessment Exercise with a system that rewards departments on the basis of performance metrics. One metric highlighted in the budget statement, external research income, seems a reasonable basis for departmental funding, as this income correlates very well with the peer review that was done in the assessment exercise.

 

But other types of research metrics — however attractive they might look to those who make funding choices — should be handled with great care. Citation statistics, for example, are a notoriously unreliable and inconstant guide to research quality. The top research agencies in the United States have successfully resisted periodic drives by bureaucrats to use them to measure the worth of the science that they should be supporting.

 

One way forward for the UK government would be to let reliable metrics, such as total external research income, replace much of the Research Assessment Exercise process, while retaining a slimmed-down version of the subject panels that currently oversee each discipline. All disciplines could then base their evaluations on research income, but the subject panels could tweak the process to reflect the needs of their particular fields of study. For example, they might choose to reward academics working on valuable long-term projects, such as the collection of environmental or astronomical data sets, the importance of which is not reflected in the metrics.

And according to Science's Elliott Marshall, "University leaders... seem delighted with... Brown's promise to overhaul the Research Assessment Exercise (RAE), a process that ranks departments by merit every 4 to 5 years and allocates funding for overhead costs of research. Critics say it has concentrated wealth in elite universities and destroyed some good departments elsewhere. Peter Cotgreave, head of the advocacy group Campaign for Science and Engineering in the U.K., says, 'abolishing the RAE is the best thing they could do.' Brown hasn't provided details of what might replace the RAE. But few will mourn its demise."

Below are the flow diagrams supplied in the preferred RQF Model followed by the members list of the RQFDAG together with an appendix referencing seventeen of TFW URLs which have dealt with the RQF since it was first proposed by Dr Nelson.

Alex Reisner

The Funneled Web