Is There an Alternative Way to Bring Scientists Together for Peer Reviews. (January 7, 2011)

John Bohannon contributed the "New Focus" article in this week's Science -- "Meeting for Peer Review at a Resort That's Virtually Free" which looks at the use of Second Life where the avatars of a group of peer reviewers for the NSF took part in experiments devised by William Sims Bainbridge, a sociologist and program director at the U.S. National Science Foundation (NSF). The question being asked: "Can the hard work of grant review be done without face-to-face meetings?"

As Mr Bohannon points out: "Over the past year, more than 19,000 scientists traveled to NSF headquarters in Arlington, Virginia, to take part in traditional review panels. Most of them worked for two solid days, huddled in groups of six to 10, carefully reading, discussing, and scoring dozens of research proposals competing for over $6 billion in grants. It's the scientific equivalent of jury duty. NSF covers expenses, but the small honorarium—typically $500—hardly covers a scientist's time, especially considering the days lost to travel. But how else can NSF evaluate the merits of all those proposals?"

Over the past nine months six NSF panels have met on its "island in Second Life, known as IISLand," and Dr Bainbridge says: "Real world panelists are provided with some resources so it was felt appropriate to provide them with the cost of a decent set of virtual clothes," assisting them to create their avatars, but aside from the gimmick Mr Bohannon writes: " the format of the meetings followed a traditional schedule, and all of the work was completed on time".

Dr Bainbridge estimates that $10,000 would be saved for every panel the worked in the virtual world. Currently NSF pays $3600 p/a rent to Linden Labs, the company that operates Second Life so: "just one normal-sized panel pays for the island more than twice over."

A couple of comments from those taking part in the experiment:

The work got done, but as soon as the computer turns off, you're gone. I much prefer the pain of going somewhere, hanging out, and having meals together. But at least you can sit around in your pajamas.

In  Douglas Fisher's view, a computer scientist at Vanderbilt University:

Is what is missing about physical presence relevant at all to making scientific recommendations? In my experience, it is not The real impediment to change is discomfort [with the virtual environment].

And while Mr Bohannon notes that that may be less of an issue for a younger generation of scientists, Dr. Bainbridge notes: "I just turned 70. So age is no barrier."