The Media, the Public and Scientific Hype Versus Peer Review. (July 7, 2004)

    They call themselves Sense About Science and describe themselves as "a charitable trust, founded in 2002, to promote an evidence-based approach to scientific issues in the public domain."
 

On June 24 they issued a sixty-one page report, Peer Review and the Acceptance of New Scientific Ideas. It is in fact a discussion paper on equipping the public and the popular media with an understanding of peer review.  Sense About Science set up a working party in November 2002 to consider how, indeed if, an understanding of the process of peer review might help the public to weigh the relative merits of different research claims. The report of the group's discussions is aimed at scientists and those groups who mediate and comment on scientific information.

The working party, chaired by Sir Brian Heap, immediate past vice-president of the Royal Society and including Colin Blakemore, head of the UK Medical Research Council and John Maddox, former editor of Nature,  first calls the scientific community to account saying more should be done to explain the system of peer review and its utility whereby "Many weak or flawed papers are rejected. Others are criticised, improved and published. This process of 'peer review' is a much more reliable guide to whether findings are plausible than who conducted the work or how it was funded. But scientists have rarely explained that peer review is used to decide which research is worthy of publication - and very few people outside the science world know to ask whether research has been peer reviewed [our emphasis]." According to the working party this situation must be rectified in order to assist  the public to evaluate confusing and contradictory claims about scientific research. So, for example, reporters, whether or not scientifically trained can and should ask more searching questions about how other experts have judged the work. "Scientists, commentators and educators who are committed to seeing public discussion informed by higher-quality research should use every opportunity to explain peer review so that more people can use that knowledge to question and weigh up different claims."

Tough questions should be asked about the information that is put before the media and the public: "questions like, 'have these research claims been peer reviewed? has the study been published in a recognised scientific journal?' and 'how many other research papers have reached the same conclusions?'"

The report doesn't claim that peer review is the be all and end all but it insists that "a culture of explaining and asking about peer review all along the line -- from radio phone-ins to ministerial briefings -- will put a lot more pressure on people bringing research claims to the public to explain exactly what the status of the work is." And media releases from universities, public and private research institutes as well as industry ought to be subject to the same scrutiny.

The report catalogues what might be termed scare sensationalisms that had their origin in publicized but unpublished claims such as health risks of radiation from mobile phones, the fear that the MMR (measles, mumps and rubella) vaccine could cause autism, and that acrylamide in fried foods could cause cancer. According to the Director of Sense About Science, Tracy Brown, such scares could be killed at birth if journalists and politicians paid more attention to the publication status of the original claims. One insidious example in the report warns, "There are also 'peer-reviewed' journals that are in reality just vanity presses. Most scientists know this, or discover it quickly. Sometimes organisations or individuals claim to have put their studies through peer review when, on inspection, they have only shown it to some colleagues. Such claims are usually made in the context of a campaign directed at the public or policy makers, as a way of trying to give scientific credibility to certain claims in the hope that a non-scientific audience will not know the difference."

Nature feels that Sense About Science has a hard row to hoe, "A poll commissioned this year by the Science Media Centre and Nature -- conducted by the London-based market-research company MORI -- showed that almost 75% of the public don't know what peer review is."