European Science Foundation* Releases the European Peer Review Guide. (May 6, 2011)

Although the attempt to reach agreement at a special general assembly of the European Science Foundation (ESF) to choose between alternative plans for the creation of a new body to represent European science through merging the ESF and the grouping representing the funding agencies of 24 countries, the European Heads of Research Councils. (EUROHORCs), ended in stalemate on May 4 the ESF did release on April 28 the European Peer Review Guide: Integrating Polices and Practices into Coherent Procedures.

The 88-page guide is the set of international guidelines for the peer review of research grants developed through the coordination by the ESF of the joint effort among the over 30 national funding and performing organisations from 23 of the countries, the European Research Council (ERC) and the European Commission and Research Executive Agency (REA).

Dr Cristina Marras from the Italian National Research Council (CNR) said: "By virtue of involving human judgement, even the same peer review procedures can have variable outcomes; peer review is the most widely used method for distributing research funding. So the five pillars for good practice described in the Guide can help us minimise this inherent variability as much as possible, furthermore, it fosters harmonisation in international peer review."

Dr Marc Heppener, Director of Science and Strategy Development at the European Science Foundation noted that: "Excellence in research depends on the quality of the procedures used to select the proposals for funding. We hope that this document can act as a central reference for all funding organisations, not just in Europe. It is a first step towards European level peer review, enabling the scientific community to operate in a global context."

In their forward to the guide, Professor Marja Makarow, ESF Chief Executive and Dr Marc Heppener make the point that:

"Excellence in research depends on the quality of the procedures used to select the proposals for funding. Public and private funding organisations at the national and international levels face the challenge of establishing and maintaining the best procedures to assess quality and potential. This is a demanding task as each proposal is scientifically unique and originates from varying research cultures. As a result, many different systems and criteria are currently in use in European countries. In order to address the issue of peer review collectively, the common needs have to be specified first. The needs then have to drive development of policies that are both convergent and complementary, where after coherent procedures can be conceived, promoted and implemented.

"The Peer Review Guide illustrates practices currently in use across the members of ESF and EUROHORCs, while also reflecting the experiences of the European Commission in its Framework Programmes. It describes good practices by setting a minimum core of basic principles on peer review processes commonly accepted at a European level.

"In addition to the quality of the basic procedures, peer reviewers and organisations face other challenges such as assessing multidisciplinary proposals and defining the appropriate level of risk inherent in frontier research.

"The Guide should serve to benchmark national peer review processes and to support their harmonisation, as well as to promote international peer review and sharing of resources. It should be considered as a rolling reference that can be updated and revised when necessary."

It was framed with the intention to present a minimum set of basic core principles commonly accepted at a European level, including those of the EU Framework Programme.

Among its specific recommendations:

• Invite non-European experts (it is also important to involve experts from emerging regions), both to ensure availability of scientific expertise and perspectives, and also to decrease the chances of conflicts of interest.
   

• Provide concise and clear instructions and guidance to the identified reviewers and panel members; this should cover all aspects of their involvement, including their tasks and contributions, requirements on conflicts of interest, confidentiality, language proficiency, etc.
   

• Provide as much advance notice to reviewers as possible, in order to increase chances of availability.
   

• Use dedicated and reliable information management systems including a reviewer database. The use of a common European database that would include its own quality assurance and possibly certification system would clearly help in promoting good practice.
   

• Provide equal playing fields. Effort should be made to consistently increase the number of representatives of the underrepresented gender in peer review activities where the percentage of the minority gender is less than 40% of the selected experts. For reviewers, it is therefore recommended that a gender ratio of at least 40% of women to men should be attained. Furthermore, individual “non-standard” career paths affected by changes or interruptions due to professional mobility and family-reasons should be considered when selecting experts.
   

• Incorporate the ‘right to reply’ in the process of peer review whenever possible. This step brings significant reliability and robustness to the decision making process and will increase the overall quality and equitability of the peer review.
   

• The rank-ordered or prioritised list must be consistently and thoroughly respected when funding decisions are made.
   

• The feedback from the review panel on the appropriateness of the requested budgets should be used if funding negotiations are to be included.
   

• Discretionary and ad hoc adjustments of the requested budgets by members of staff at the funding organisations should be avoided as much as possible.
   

• If negotiations and changes are to be included as part of the process, the expert assessors’ views must be used as much as possible as the basis for refining the funding allocations. Organisations’ dedicated scientific boards, councils and committees could also provide input as appropriate.
   

• Whether or not a programme is exclusively designed for pluri-disciplinary research, it is recommended to devote the necessary time, expertise and attention at an early stage of filtering or eligibility screening such that proposals that are genuinely of MICT [multi-, inter-, cross- and trans-disciplinary] type can be identified and undergo the most appropriate peer review process according to their disciplinary characters.
   

• For any instrument (whether or not explicitly devoted to pluri-disciplinary research), it is recommended to have proposals that are found to be genuinely of MICT character peer reviewed in a two-stage or three-stage evaluation process using individual expert assessments followed by appropriate review panel deliberations and decisions.
   

• Care should be taken in putting in the right context the assessments from the host and the complementing reviewers, especially when having large numbers of assessments, such that the chances of unduly penalising the proposals is minimised.

In addition there are a number of recommendations directed to specific situations such as:

• peer review approaches specific to Individual Research and Career Development proposals,

• peer review approaches specific to Collaborative Research proposals,

• peer review approaches specific to Scientific Network proposals,

• peer review approaches specific to Centre of Excellence proposals and

• peer review approaches specific to New Research Infrastructure proposals.

____________________________________________________

Below we reprint the Guide's table of contents.

_____________________________________

___________________________________________________

*The European Science Foundation (ESF) is an independent, non-governmental organisation, the members of which are 78 national funding agencies, research performing agencies, academies and learned societies from 30 countries.