December 8, 2012
And It Took Only 2-Years for Brilliance to Burst Forth?
Brooks Atkinson's opening remark in his 1956 New York Times review of Samuel Beckett's 2 act tragicomedy Waiting for Godot. ... It is a mystery wrapped in an enigma. [E]xpect to be witness to the strange power this drama has to convey the impression of some melancholy truths about the hopeless destiny of the human race.
_______________________________
Vladimir: Second utterance of the ERA is out.
Estragon: You don't say and for the millions spent and thousands of hours of academics' time taken up with the exercise what has been deduced.
V: Well according to The Australian's Julie Hare: Just two years after the first Excellence in Research for Australia was conducted, almost every one of the 39 institutions audited showed marked improvement in the quality of research being assessed.
E: That's amazing, Vladimir, who would of ever thought it. In just Two years! You sure?
V: Would I lie to you, Gogo; we are truly punching above our weight.
E: no, no, forget that stuff, I'm asking you, do you really believe that in two years our university sector has increased its brilliance to this remarkable extent, because I think that's rubbish.
V: So what makes you the expert?
E: Not me, but how about the deputy vice-chancellor (research) at the University of NSW.
V: And what's he/she got to say?
E: In this case he's a he, Les Field, and he says that the upward drift from 10 universities classified as "at or above world standard" in 2010 to about half of the 39 this year, "can't be the result of dramatic improvement, but must reflect changes to assessment criteria and the assessment process".
V: So you're saying the ERA has cooked the books and is a waste of time?
E: Waste of time? How could you come to such a conclusion? Oh no, son, it's much worse, it's a pernicious waste of money and intellectual resources par excellence. The waste of time is a by-product. Tony Peacock, chief executive of the Cooperative Research Centres Association, told The Australian the ERA is too detailed, too time consuming, too expensive and comes too often and suggested the next one be put off until 2016 or 2017, which seems to me is really saying it ought to be junked.
V: I see, you'd also agree then with the V-C of Tassie, Peter Rathjen: We have to be very careful that ERA doesn’t constrain research by encouraging leaders to invest in things where they can do best in ERA. The worry is that at least some research leaders in this country seem to be saying that what they want to do for their university is to maximise their ERA outcome. I’m not alarmed about it, but I’m sounding a note of caution about the way we choose to use the data. The whole point of research is that it has to be nimble and it has to responsive to be able to move.
E: No, I think he ought to be alarmed because the ERA is a distraction if not a downright obfuscation. In Dr. Peacock's view the: "ERA doesn't save one baby, it doesn't make one species survive through environmental research, it doesn't improve the quality of water or anything." And then added that the money being spent on ERA, and any money for the proposed new measure of impact known as Excellence in Innovation for Australia, would be better spent on funding research.
V: And you'd go along with that?
E: As far as it goes, yes.
V: What does that mean?
E: It means that what are being avoided are the real issues.
V: Namely?
E: That university administrations don't do research and that the allocation of public funding for research should be determined both objectively and knowledgeably. In short what's needed is improvement of the system of peer review, not to place over it an additional bureaucratic and retrospective layer of compliance.
Look, Vladimir, there is a growing consensus that peer review within the study sections for research grant proposals is in need of significant renovation. It's true in the US, it's true in the UK, it's true in Canada and it's true in Australia just to name a few Anglophone nations. Yet all we've seen over the past half decade are some minor tweaks, and occasional hand wringing.
______________________________________
______________________________________
V: And you could fix it?
E: Me, No. But an international effort could and should. All the tools are available, what's lacking is the will, and yet the way is already being addressed.
V: By whom?
E: Subra Suresh.
V: Who he?
E: Don't play the fool, Vladimir, Subra Suresh is the director of the US National Science Foundation and he was and is the prime mover of the Global Research Council which got launched this past May. It sees itself as "a virtual organization of the heads of science and engineering funding agencies from around the world". And one of its 47 members is Australia's ARC.
V: And just what does it have to do with fixing peer review.
E: Well, as Suresh put it in his Science editorial of May 25, 2012: ...good science anywhere is good for science everywhere, provided that there exists an open flow of information with transparent processes to promote rigorous peer review and scientific integrity... One major barrier to successful international scientific collaboration is variation in what constitutes appropriate peer review of research proposals... [and] the most basic and essential ingredients of scientific merit review [are]: expert assessment, transparency of the evaluation process, impartiality, appropriateness, confidentiality, integrity and ethical consideration.
V: And you're saying that the GRC is a vehicle to drive this initiative?
E: I'm not he is: To ensure fair evaluation across countries, principles are needed to guarantee that the appropriate scientific expertise is employed for peer review, with a clear awareness of potential conflicts of interest. Agreement on such issues would help to foster more effective international research cooperation. The GRC represents a new model for discussing issues aimed at unifying and strengthening the global scientific enterprise.
V: So peer review of grant proposals should go international.
E: Precisely, the technology is at hand, pilot trials of study panels have already been undertaken by the NSF within a virtual-reality environment. For a nation such as Australia, with its restricted number of individuals available for peer review of research grant applications, which in turn can foster less than adequate peer review, such an approach could have a salubrious effect on ARC and NHMRC awards.
V: You're a dreamer, Gogo, Britain will have switched to the € before our Bureausaurs would go along with such a proposal.
Alex Reisner
The Funneled web