News & Views item - November 2006

 

 

Bishop Ignores Productivity Commission's Caveat, Flags Intent to Implement RQFDAG's Preferred Model. (November 15, 2006)

     In a media release issued yesterday the Minister for Education, Science and Training, Julie Bishop, announced, "The Australian Government will implement a Research Quality Framework (RQF) which will strengthen the assessment of research carried out in Australia."

 

The cover page of the document sent to the minister contains the statement

 

THE RECOMMENDED RQF
October 2006
Endorsed by the Development Advisory Group for the RQF

 

Ms Bishop was careful not to say that the Research Quality Framework model recommended by the RQF Development Advisory Group would be implemented but did publish the time frame the government intends to follow, "Preparatory work and trialling will continue in 2007, with data collection in 2008 and funding implementation in 2009."

 

In fact what is recommended by the RQFDAG is not so much a framework as a skeletal scaffold. According to page 16 of the recommendation, a great deal of work is still required (and much of it may well prove contentious).

 

It is also unclear how much of the final design will be in the hands of the Department of Education, Science and Training staff and how much of the effort will be allowed to be carried on at arms length from federal government intervention, but in addition, while a "complete set of overarching and panel-specific Guidelines is planned to be released by mid-2007,... A trial of the RQF is planned for the second quarter in 2007".

 

Those involved may enjoy their continuing dance in the dark.

 

Proposed Consultation for 2007

If the RQF is approved for implementation, work to develop Guidelines will commence in late 2006. In addition, the Chairs of the Assessment Panels will be recruited to assist in the development of panel-specific Guidelines and in the conduct of pre-implementation trials.
 

A draft of overarching Guidelines for the RQF will be developed for discipline workshops, planned for early 2007, to develop the panel-specific sections of draft RQF Guidelines. These workshops will build on the preliminary scoping workshop held in February 2006.
 

Each of the 13 panel areas will have a workshop, chaired by the Chair of its respective Assessment Panel. Workshop participants will have expertise in the relevant disciplines and will be representative of the relevant stakeholder institutions and groups.
 

The complete set of overarching and panel-specific Guidelines is planned to be released by mid-2007. This will provide the higher education and research sectors with a final opportunity to comment on the development process.
 

A trial of the RQF is planned for the second quarter in 2007 for selected institutions and disciplines so as to ensure an appropriate representation of different issues.
 

The trial will test the mechanisms for quality and impact assessments, the moderation process and the funding models. The outcome of these trials will also inform draft RQF Guidelines. Participating Research Groups will have six weeks to submit trial evidence portfolios. Assessment Panels will conduct quality and impact assessments and Chairs will moderate the process.

It is noteworthy that Ms Bishop's release of her intention's is contra to the caution suggested by the Productivity Commission's Draft Report on Public Support for Science and Innovation which was issued on November 2.

 

 

With regard to the prospective implementation of the RQF the Commission's concludes:

The arguments for discontinuing the existing formula-based approach to the allocation of block funding in favour of an approach based on the proposed RQF cannot be fully tested at this stage.

• Although formula-based approaches to funding do have deficiencies, there is no clear objective evidence pointing to deficiencies in the quality of research currently funded through block grants.

• However, there is evidence that the RQF will bring costs as well as benefits. But the full range of benefits and costs cannot be assessed until there are detailed criteria for quality and impact assessment, methodology for weighting and aggregation, and associated funding formulae. (Implementation aspects are currently being studied by the RQF Development Advisory Group.)

In this circumstance, the Commission would suggest that it is still too early to make a final decision about implementation of the RQF, one way or the other.

DRAFT FINDING 11.1
Consideration should be given to delaying the adoption of the RQF further, while undertaking the following investigations and analyses:

• continue with limited trials based on RQF peer-review principles, but focus them on providing indicators of the quality and impact of research dependent on block funding;

• systematically examine whether current procedures within institutions are sufficiently rigorous to promote quality and impact of block-funded research;

• examine what fine tuning of existing formulae, if any, might be advantageous in promoting incentives for continuing enhancement of quality and impact of research funded through block funding; and

• examine the merits of externally applied, risk-minimisation approaches to enhancing the quality and impact of block-funded research (applied in conjunction with formula-based funding).

Under the last approach an external auditor, for example, might identify areas of deficiency in institutions, which would then be encouraged to lift their game over the period ahead.

It would appear that Ms Bishop has a total disregard for the Commission's assessment and appears even to have rushed her announcement for whatever reasons best known to her.

 

Certainly it seems unlikely that either the minister or the RQFDAG were unaware of the Commission's draft  findings prior to its public release.

 

The Commission, having issued its Draft, has asked for submissions regarding the document by December 21, 2006 and intends to release its final report in March 2007. Whether or not it will retain reference to the RQF remains to be seen.