I think it is fair to say that even those of us most enamored of post-publication peer review would agree that its effectiveness remains to be demonstrated in a convincing fashion. Broadly speaking there are two reasons for this; the first is the problem of social norms for commenting. As in there aren’t any.

This post is both a follow up to last week’s post on the cost’s of peer review and a response to Duncan Hull‘s post of nine or so months ago proposing a game of “Fantasy Science Funding“. The game requires you to describe how you would distribute the funding of the BBSRC if you were a benign (or not so benign) dictator. The post and the discussion should be read bearing in mind my standard disclaimer. Peer review is in crisis.

Late last year the Research Information Network held a workshop in London to launch a report, and in many ways more importantly, a detailed economic model of the scholarly publishing industry.

A thought sparked off by a comment from Maxine Clarke at Nature Networks where she posted a link to a post by David Crotty. The thing that got me thinking was Maxine’ statement: Alright, in isolation this doesn’t look like much, read through both David’s post and the comments, and then come back to Maxine’s,Â

I hold no particular candle for traditional peer review. I think it is inefficient, poorly selective, self reinforcing, often poorly done, and above all, far too slow. However I also agree that it is the least worst system we have available to us.  Thus far, no other approaches have worked terribly well, at least in the communication of science research.

This post is an opinion piece and not a rigorous objective analysis. It is fair to say that I am on the record as and advocate of the principles behind PLoS ONE and am also in favour of post publication peer review and this should be read in that light. [ ed I’ve also modified this slightly from the original version because I got myself mixed up in an Excel spreadsheet] To me, anonymous peer review is, and always has been, broken.

The mainstream media has a lot of negative things to say about blogs and user based content on the web. Most of them can be discounted but there is one that I think does need to be taken seriously. The ability of communities to form and to some extent to close around themselves and to simply reinforce their own predjudices is a serious problem and one that we need to work against.

Another post prompted by an exchange of comments on Neil Saunder’s blog. The discussion here started about the somewhat arbitrary nature of what does and does not get counted as ‘worthy contributions’ in the research community.