Open Reviews?

The PLoS Computational Biology journal has an editorial about open access biology papers. Most of it is the usual Web 2.0 aspirations: We need more structured data, more semantics, more killer apps taking advantage of it.

Not that I disagree, necessarily, but I've heard it all so often that it's not really registering anymore. Maybe the killer apps will come along, and maybe they won't. Web 2.0 thinking has potential for science publications, but not everything that has potential gets realised.

But towards the end of the article, there was something that caught my attention: Open review.

Certainly rating a paper would seem reasonable when done by the Faculty of 1000 (http://www.f1000biology.com), but it is not a generally accepted practice. We challenge you to rate this Editorial too. In some ways the reluctance to rate a scientific paper is strange since we suspect the same person may well rate a book on amazon.com. Another option would be to add a Digg or del.icio.us button (http://digg.com or http://del.icio.us) to incorporate conventional media ranking tools into an academic journal Web site. If one finds an interesting article, one could immediately flag it with these tools.

Now this is interesting, because peer-review is a nearly sacred notion in science. Your paper has not proven merit until it has passed the peer-reviewing process and been published. It basically says, "other scientists thought this was worth reading".

So what happens if the reviewing suddenly become open to all? Well, it essentially become a popularity contest. Digg is a perfect example: The stories that end up on the front page are the ones that a lot of people liked. Very democratic, isn't it? Only it means that today the stories included "Pranks to pull on your Co-workers" and "The 10 Most Mismatched Movie Couples".

Oh, there were plenty of interesting stories as well, but my point is that what's popular is not necessarily what's best. By all means, let people comment and review papers, but make sure we know which reviews come from scientists, and which come from your average schmoe. I know it sounds elitists, but it's not much more elitist than demanding that the person who takes out your appendix have a medical degree.