Open peer review

Open peer review describes a scientific literature concept and process, central to which is the various transparency and disclosure of the identities of those reviewing scientific publications. The concept thus represents a departure from, and an alternative to, the incumbent anonymous peer review process, in which non-disclosure of these identities toward the public - and toward the authors of the work under review - is default practice. The open peer review concept appears to constitute a response to modern criticisms of the incumbent system; ergo, its emergence may be partially attributed to these phenomena.

Rationale

The traditional anonymous peer review has been criticized for its lack of accountability, possibility of abuse by reviewers, its possible bias and inconsistency,^[1] alongside other flaws.^[2][3] Both processes are intended to subject scholarly publications to the scrutiny of others who are experts in the same field.

The evidence of the effect of open peer review upon the quality of reviews, the tone and the time spent on reviewing is mixed, although it does seem that under open peer review, more of those who are invited to review decline to do so.^[4][5]

History

A number of reputable medical publishers have trialed the Open Peer Review concept, with many organizations making subsequent advances to the concept proper since its inception in c. ~1996.^[when?]

Early era: 1996-2000

In 1996, the Journal of Interactive Media in Education^[6] launched using open peer review.^[7] Reviewers' names are made public and they are therefore accountable for their review, but they also have their contribution acknowledged. Authors have the right of reply, and other researchers have the chance to comment prior to publication.

In 1997, the Electronic Transactions on Artificial Intelligence, [2], was launched as an open access journal by the European Coordinating Committee on Artificial Intelligence. This journal used a two-stage review process. In the first stage, papers that passed a quick screen by the editors were immediately published on the Transaction's discussion website fpr the purpose of on-line public discussion during a period of at least three months, where the contributors' names were made public except in exceptional cases. At the end of the discussion period, the authors were invited to submit a revised version of the article, and anonymous referees decided whether the revised manuscript would be accepted to the journal or not, but without any option for the referees to propose further changes. The last issue of this journal appeared in 2001.

In 1999, the open access journal Journal of Medical Internet Research^[8] was launched, which from its inception decided to publish the names of the reviewers at the bottom of each published article. Also in 1999, the British Medical Journal^[9] moved to an open peer review system, revealing reviewers' identities to the authors (but not the readers),^[10] and in 2000, the medical journals in the open access BMC series^[11] published by BioMed Central, launched using open peer review. As with the BMJ, the reviewers' names are included on the peer review reports. In addition, if the article is published the reports are made available online as part of the 'pre-publication history'.

Several of the other journals published by the BMJ Group^[12] allow optional open peer review,^[13][14][15] as do PLoS Medicine, published by the Public Library of Science.^[16][17] The BMJ's Rapid Responses^[18] allow ongoing debate and criticism following publication.^[19] By 2005, the editors found it necessary to more rigorously enforce the criteria for acceptance of Rapid Responses, to weed out the "bores".^[20]

Recent era: 2001-present

Atmospheric Chemistry and Physics, an open access journal launched in 2001 by the European Geosciences Union, has a two-stage publication process. In the first stage, papers that pass a quick screen by the editors are immediately published on the Atmospheric Chemistry and Physics Discussions website. They are then subject to interactive public discussion alongside formal peer review. Referees' comments (either anonymous or attributed), additional short comments by other members of the scientific community (which must be attributed) and the authors' replies are also published in ACPD. In the second stage, the peer-review process is completed and, if the article is formally accepted by the editors, the final revised papers are published in ACP. The success of this approach is shown by the ranking by Thomson Reuters of ACP as the top journal in the field of Meteorology & Atmospheric Sciences^[21]

In June 2006, Nature launched an experiment in parallel open peer review — some articles that had been submitted to the regular anonymous process were also available online for open, identified public comment.^[22] The results were less than encouraging — only 5% of authors agreed to participate in the experiment, and only 54% of those articles received comments.^[23][24] The editors have suggested that researchers may have been too busy to take part and were reluctant to make their names public. The knowledge that articles were simultaneously being subjected to anonymous peer review may also have affected the uptake.

In 2006, a group of UK academics launched the online journal Philica, which tries to redress many of the problems of traditional peer review. Unlike in a normal journal, all articles submitted to Philica are published immediately and the review process takes place afterwards. Reviews are still anonymous, but instead of reviewers being chosen by an editor, any researcher who wishes to review an article can do so. Reviews are displayed at the end of each article, and so are used to give the reader criticism or guidance about the work, rather than to decide whether it is published or not. This means that reviewers cannot suppress ideas if they disagree with them. Readers use reviews to guide what they read, and particularly popular or unpopular work is easy to identify.

Another approach that is similar in spirit to Philica is that of a dynamical peer review site, Naboj.^[25] Unlike Philica, Naboj is not a full-fledged online journal, but rather it provides an opportunity for users to write peer reviews of preprints at arXiv.org. The review system is modeled on Amazon and users have an opportunity to evaluate the reviews as well as the articles. That way, with a sufficient number of users and reviewers, there should be a convergence towards a higher quality review process.

In February 2006, the journal Biology Direct^[26] was launched by BioMed Central, providing another alternative to the traditional model of peer review. If authors can find three members of the Editorial Board who will each return a report or will themselves solicit an external review, then the article will be published. As with Philica, reviewers cannot suppress publication, but in contrast to Philica, no reviews are anonymous and no article is published without being reviewed. Authors have the opportunity to withdraw their article, to revise it in response to the reviews, or to publish it without revision. If the authors proceed with publication of their article despite critical comments, readers can clearly see any negative comments along with the names of the reviewers.^[27]

An extension of peer review beyond the date of publication is Open Peer Commentary, whereby expert commentaries are solicited on published articles, and the authors are encouraged to respond. In the summer of 2009, American academic Kathleen Fitzpatrick explored open peer review and commentary in her book, Planned Obsolescence, which was published by MediaCommons using Commentpress, a Wordpress plugin that enables readers to comment on and annotate book-length texts.

See also

• Open research

References

1. ^ Peter M. Rothwell, Christopher N. Martyn (2000). "Reproducibility of peer review in clinical neuroscience: Is agreement between reviewers any greater than would be expected by chance alone?". Brain 123 (9): 1964–1969. http://brain.oxfordjournals.org/cgi/content/full/123/9/1964. 
2. ^ The Peer Review Process
3. ^ Alison McCook (February 2006). "Is Peer Review Broken?". The Scientist. http://classic.the-scientist.com/article/display/23061/. 
4. ^ Effect of open peer review on quality of reviews and on reviewers' recommendations: a randomised trial - van Rooyen et al. 318 (7175): 23 - BMJ
5. ^ Elizabeth Walsh, Maeve Rooney, Louis Appleby, Greg Wilkinson (2000). "Open peer review: a randomised controlled trial". The British Journal of Psychiatry 176 (1): 47–51. doi:10.1192/bjp.176.1.47. PMID 10789326. 
6. ^ Journal of Interactive Media in Education
7. ^ About JIME
8. ^ JMIR Home
9. ^ bmj.com: BMJ - Helping doctors make better decisions
10. ^ Opening up BMJ peer review - Smith 318 (7175): 4 - BMJ
11. ^ BMC series
12. ^ BMJ Group website — BMJ Group website
13. ^ Peer review at ARD
14. ^ Peer review at JME
15. ^ Peer review at EMJ
16. ^ Public Library of Science
17. ^ PLoS Medicine: A Peer-Reviewed, Open-Access Journal
18. ^ bmj.com Rapid Responses published in the past day
19. ^ Twenty thousand conversations - Delamothe and Smith 324 (7347): 1171 - BMJ
20. ^ Revitalising rapid responses - Davies and Delamothe 330 (7503): 1284 - BMJ
21. ^ [1]
22. ^ Peer Review Trial
23. ^ Overview: Nature's trial of open peer review
24. ^ Peer review and fraud: Article: Nature
25. ^ Naboj Dynamical Peer Review
26. ^ Biology Direct | Home page
27. ^ Biology Direct |