England's 2014 Research Excellence Framework Fumbles On. (July 26, 2011)

Like NASA's stated initiative under Dan Goldin, the Higher Education Funding Council for England's (HEFCE) reincarnation of its Research Assessment Exercise (RAE) the Research Excellence Framework (REF) for 2014 was to be Better, Faster, Cheaper.

Judging from the timeline and guidelines released by the HEFCE that seems unlikely:

Credit: Times Higher Education

And like its counterpart in Australia, the Excellence in Research for Australia (ERA) it remains an exercise in rear vision and second guessing the research councils' peer review systems without providing any impetus for improving the provision of resources for outstanding visionary researchers.

The one major difference between the REF and ERA, as well as its predecessor the RAE, is the counting of 20% for "impact" in the final scoring. The assessment of the impact of research is to be based on one case study for every 10 academics.

Initially, time, effort and costs were to be reduced through the use of bibliometric data, however, a test run by HEFCE satisfied the council that citation data were "not sufficiently robust to be used formulaically or as a primary indicator of quality" ( see Assessment framework and guidance on submissions). According to the document, assessment panels may not use citation data as a proxy for evaluating submissions through reading them. Furthermore, the ranking of journals for judging research quality is also deemed improper.

The guidelines also state that early career researchers whose circumstances "have significantly constrained their ability to... work productively throughout the assessment periods" won't be downgraded for submitting less that the normally required four items.

With regard to the evaluation of interdisciplinary research Graeme Rosenberg told Times Higher Education's Paul Jump that while "many of those already appointed to subpanels have interdisciplinary expertise... subpanels would also be permitted to ask other subpanels for help with cross-disciplinary submissions."

Meanwhile, attempts to directly improve the peer review system significantly in order to foster truly outstanding advancements in research remain neglected.