GET THE APP

What Scientific Journals Can Do to Improve the Peer Review Proces
Journal of Nutrition & Food Sciences

Journal of Nutrition & Food Sciences
Open Access

ISSN: 2155-9600

Editorial - (2013) Volume 3, Issue 4

What Scientific Journals Can Do to Improve the Peer Review Process: Rewarding the Reviewer!

Massimo Bionaz*
Department of Animal and Rangeland Sciences, Oregon State University, Corvallis, Oregon 97331, USA
*Corresponding Author: Massimo Bionaz, Department of Animal and Rangeland Sciences, Oregon State University, Corvallis, Oregon 97331, USA

Since its first application back in the 17th century[1], peer-review of scientific publications has become an essential process to increase trustworthiness of scientific discoveries. In general, editors of scientific journals send the submitted manuscript to 2 or 3 reviewers. They are scientists with expertise in the subject of the scientific article and have the task of providing a thorough review. The report of the review should help the editor to decide if the article can be considered suitable for publication (based on scientific merit, proper literature review, adherence to objective(s), proper methods, and interpretation of results), but also should provide helpful suggestions to the authors to improve the manuscript. A peer review, when done well, should benefit the scientific journal by avoiding publication of articles with a low scientific standard, and the author(s) by helping to improve the manuscript and providing the peace of mind that what is published is scientifically sound.

Despite its importance, the peer review process presents several limitations, as recently reviewed [1]. The limitations are mostly associated with the anonymous or blind peer-review process. Among the limitations, it is worthwhile to mention the review performed by unfair reviewers with undisclosed competing interests (either economical or ideological) [2], the poor review performance due to indifferent reviewers or reviewers experiencing the so-called Dunning- Kruger effect (i.e., inability of the ignorant to recognize their ignorance) maybe together with the self-fulfilling prophecy (i.e., a prophecy that causes itself to become true due to series of positive feed-backs or feedforward processes such as the reviewer of a manuscript and an editor of a journal rejecting or accepting a paper for publications because of a shared wrong idea) [3], or the negative evaluation and rejection of authors that propose extreme/novel ideas [4]. Several experiments have been carried out to evaluate the peer-review process (reviewed in [4]). It is interesting to note that there has been vast disagreement among reviewers’ ratings on the same article [5], and, even though challenging the disagreement between reviewers, a recent paper confirmed a low correlation between reviewers’ recommendations and citation of the articles [6]. The authors, however, pointed out that this low correlation might be biased by the fact that papers were heavily changed as a consequence of the revision(s), implying a strong positive impact of the reviewers on the manuscript. In order to overcome such limitations, several mainstream journals have taken action to improve the peerreview process (e.g., [7,8]).

It has been reported that a review of a scientific article can take on average 3 hours of work, with, for instance, reviewers of nursing journals spending 5 hours and reviewing 7 to 8 manuscripts per year [4]. This was defined “considerable time” [4]. I do not know about other reviewers, but 3 or 5 hours and 7-8 papers seems to be a very conservative estimate. In my personal experience, a review of a scientific paper takes on average more than 5 hours. This includes reading the entire manuscript a minimum of two times, and providing a point-by-point commentary in order to point out limitations and offer suggestions as my expertise allows. As a rule of thumb, I review only one manuscript per month; however, it is not always the case, and often I find myself reviewing two or more papers in a month. The number of published articles has increased constantly in the last 50 years [1]. The number of scientists has not increased as much in the same time, and often the specialization and the advent of new disciplines require new expertise that takes time to be established and often few are the scientists with such expertise. Therefore, editors of journals often find themselves with the daunting challenge of finding more than 1 or 2 good reviewers.

Considering the time necessary for reviewing manuscripts, the high pressure on young scientists to get tenure, and the consistently higher competition for obtaining funding for research (which means spending more time to write grants as well as papers), it is not surprising that successful scientists (therefore potential good reviewers) frequently turn down the request of a journal to review a manuscript, as previously pointed out [9]. This is exacerbated considering that the reviewer does not get credit for the time spent in reviewing the paper, except being listed among the ad hoc reviewers of a journal or receiving an e-mail from the journal thanking them for their “invaluable” contribution. Both rewards are independent of the number and quality of the reviews performed. We also have to consider that very often the review of a paper is not performed only once. There are journals that allow performing a review of the revised version of the paper, and rightly so. In my experience, I have revised up to 5 drafts of the same manuscript for a journal. What was surprising was that among the 3 reviewers, I was the only one pointing out that the paper needed major changes, due to serious methodological flaws. The other two reviewers were happy with the manuscript. After all the revisions, the paper had a sound methodology and the authors thanked me all the way through the revisions, because the paper was getting more scientifically sound. I felt that I could have been considered an author of that paper due to the time and contribution to the final result, and I had asked the editor of the journal if he could suggest this to the authors. I got only silence. I spent more than a full week of work on that paper, and I have not received any more credit than the other two reviewers. Once, I told this story to an editor of a journal, and he told me that I should not worry about credits, because if I was a good reviewer I will be asked to be a section editor of the journal. This might be good for my career; however, it will require spending free time that I do not have, or taking time from other tasks, such as writing grants and papers, which is especially problematic since I am not an established scientist.

Why should I not be rewarded as a reviewer? I think it should be quite easy to reward a reviewer and I think it will benefit the whole peer review process. I suggest:

1) Disclose to the public the name of the reviewer(s) in a specific section of a published paper. This has been proposed previously by others (e.g., [2,10]). Several journals, especially the new open access ones, have embraced the so-called “open reviewer process”. This implies that reviewers will sign their report and authors know who has performed their review; however, the name of the reviewer is not present in the final published paper. For instance, GigaScience [11], PLoS ONE, and the BMJ journals opted for having the open peer review as a default. There is only one journal, the Journal of Medical Internet Research, which I know discloses to the public the name of the reviewers at the bottom of each published article. The fact that the reviewer is associated with the published paper has several advantages for the reviewer, the authors, and the journal. For the reviewer, this means more visibility and credit for specific expertise, particularly if the published paper is good (and this goodness can often be due to reviewer’s contribution). It will be a more measurable contribution to the scientific field and, very likely, can also be considered more important for tenure promotion if added in the CV [11]. Conversely, a bad paper with methodological flaws will also be associated with the reviewer; hence, this should prompt the reviewer to do a thorough review or decline it if the expertise is lacking (i.e., the Dunning-Kruger effectshould be reduced, likely). The authors will probably receive a more constructive review, as demonstrated by the open review process [11], and may additionally help to avoid having the manuscript rejected because of a bad reviewer and/ora reviewer with interest in halting the discovery of a competitor. If the manuscript will be sent to another journal and there accepted, other reviewers will get the credit (if the other journal has enacted the open reviewer process with disclosure of the reviewer names). Having credit for reviewing a novel paper would also be beneficial to the reviewer. Association with a new idea, especially if the reviewer ensures the methods are sound and interpreted correctly, would reflect well on the reviewer and keep them at the forefront, perhaps even leading to future collaborations. A journal that adopts this approach would likely see an increase in manuscript submissions due to the increased quality of reviews, which would lead to higher quality papers and a higher impact score. If the authors and the reviewers will be happier, so will the journal.

2) The reviewer’s report should be fully disclosed (except the confidential comments to the editor). This will allow seeing which reviewer contributed a more accurate assessment; again increasing the visibility of trustworthy scientists.

3) Numerical public scoring of the reviewers by the editor and by the authors (e.g., with a scale 1-5 or 1-10). This number can be used to evaluate the final performance of a reviewer for that journal, but can also be used to evaluate the yearly and total “impact” of that scientist as a reviewer. The evaluation of the reviewer is already performed by the OMICS group journal, for instance.

4) Using the evaluation score, the journal should provide a rank for each reviewer. The best reviewers should be awarded annually. This can be a good addition to a CV.

The proposed way to provide credit to the reviewers, as suggested above, can appear a little extreme considering the current diffuse blind reviewers system. I also realize that all the limitations of the peer review process are not likely to be solved by following the above suggestions. However, the disclosure of the reviewer and a credit recognized to her/him will likely make this activity more fair and attractive for busy scientists, especially if they are in their early career.

Acknowledgements

The author thanks Dr. Stephanie Bollmann for revising the manuscript and providing excellent suggestions.

References

  1. Larson BP, Chung KC (2012) A systematic review of peer review for scientific manuscripts. Hand (N Y) 7: 37-44.
  2. Horrobin DF (1974) Referees and research administrators: barriers to scientific research? Br Med J 2: 216-218.
  3. Huang S (2013) When peers are not peers and don't know it: The Dunning-Kruger effect and self-fulfilling prophecy in peer-review. Bioessays 35: 414-416.
  4. Thomas SP (2011) Conceptual Debates and Empirical Evidence about the Peer Review Process for Scholarly Journals. J Prof Nurs 27: 168-173.
  5. Bornmann L, Mutz R, Daniel HD (2010) A reliability-generalization study of journal peer reviews: a multilevel meta-analysis of inter-rater reliability and its determinants. PloS One 5: e14331.
  6. Baethge C, Franklin J, Mertens S (2013) Substantial agreement of referee recommendations at a general medical journal - a peer review evaluation at deutsches arzteblatt international. PLoS One 8: e61401.
  7. Kirby A (2005) Peering into the future of academic review. Cities 22: 85-87.
  8. Edmunds SC (2013) Peering into peer-review at GigaScience. Gigascience 2: 1.
Citation: Bionaz M (2013) What Scientific Journals Can Do to Improve the Peer Review Process: Rewarding the Reviewer!. J Nutr Food Sci 3:e120.

Copyright: © 2013 Bionaz M. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Top