The Hwang Affair -- Science Receives the Committee's Report Assessing How the Journal Dealt With the Fraudulent Research Papers. (November 29, 2006)

    On September 15, 2006 the Editor in Chief of the journal Science, Donald Kennedy, was sent the following letter.

Dear Don:
I submit a report of the committee... assembled to examine the process involved in publishing the paper by Woo Suk Hwang that was subsequently retracted: Woo Suk Hwang [et al.]... Science, 308, 1777 – 1783 (2005).

The committee examined the original submissions of Hwang from 2004 and 2005, and the editorial material (reviews, revisions, comments, editor’s notes, additional information). We discussed both by phone and email our impressions and analysis of the way in which the papers were handled at Science and possible changes in procedures that might be appropriate. We hope that the results of this study will be helpful to you and to Science.

Sincerely,

John I. Brauman (for the committee).

Accompanying the note was a concise four page assessment and list of recommendations put forward by the six members of the review committee.

Its findings in summary:

In the case of the Hwang papers, the editors followed the procedures in operation at Science, a process review similar to that at other top tier scientific journals. Despite this, a bad outcome occurred for the journal, and for science. The journal was intentionally deceived. This type of deception has occurred before, and, as a result of the visibility of the journal, could well be attempted again.

No realistic set of procedures can be completely immune to deliberate fraud. However, although infrequent, these cases have important consequences for the journal and for science broadly. Science should therefore modify and strengthen its procedures for review to provide additional scrutiny of the papers it publishes, especially the high-impact papers. Heightened scrutiny will also deter the submission of flawed or intentionally deceptive work that falls short of outright fraud.

Current procedures are based on an assumption of trust: Science assumes that the papers it receives are honestly conceived and written. In fact, it now receives a small number of papers that are either intentionally misleading (Hwang, Schoen) or substantially distorted by self-interest. Science must institutionalize a healthy level of concern in dealing with papers that it considers for publication, especially those likely to be very visible or influential.

The number of papers submitted to Science makes it essentially impossible in practice to examine each of the papers it receives with this heightened level of scrutiny. Fortunately, only a small fraction of the papers it selects for publication require such special attention – attention that will be time consuming and expensive, and may lead to conflict with authors.

New procedures would help Science improve its defenses against intentionally misleading work. We recommend that editors conduct audits of submissions at a level of detail that it does not currently use (for selected papers, and for occasional randomly chosen papers). Science should have substantially stricter requirements about reporting the primary data. Papers selected for possible publication should undergo a formal risk-assessment using a template that the editors should devise. Science should also rethink its requirements for coauthorship, its policies concerning treatment of digital images and biological samples, and its penalties for authors who knowingly submit distorted or faulty work.

In his editorial to be included in Science's December 1^st issue Donald Kennedy writes in part:

The good news for Science is that its editors and peer reviewers not only followed the procedures in place here and at other top-tier journals, but made a substantially greater effort than for most papers to ensure that the science was sound. The not-so-good news is that the report sends us some tough messages about what Science should do to confront a present reality and prepare for a more challenging future. It points out forcefully that the environment for science now presents increased incentives for the production of work that is intentionally misleading or distorted by self-interest.

Science has made available a Special Online Collection: Hwang et al. Controversy -- Committee Report, Response, and Background at http://www.sciencemag.org/sciext/hwang2005/