Introduction to Peer Review

Why Peer Review?

Peer reviewing provides obvious benefits to multiple stakeholders:

For the author:

The author of the manuscript benefits from an objective and critical set of eyes. Though the process may seem fraught and daunting to some authors, their work is more effective in responding to the feedback they receive, making their findings stronger and more valuable. A reviewer can see holes in the research or documentation of the research that isn’t always clear when the writer is deep in the process.

For the editor:

The editors goal is to put out a strong reputable publication with engaging new ideas in a timely manner. They rely on reviewers to help them assess the quality of the manuscript submissions and help guide authors to strengthen their work. Though the editor is not bound by the recommendation of the peer reviewer, they are certainly guided by a thoughtful, quality review.

For the reviewer:

Reviewing other scholars’ work provides a way to stay current on research and knowledge within your discipline. It helps keeps your research skills sharp and may expose you to new methodologies or research approaches. Former editor of Educational Researcher, Donmoyer (2011) argues that “reviewing, when done well, has a great deal to do with becoming a better writer” (p. 240) by exposing the reviewer to different models of writing, some of which the reviewer may aim to emulate in their own work.

For the discipline:

The articles that ultimately achieve publication reflect the output and focus of the discipline. Consequently, the published work provides the foundation for future scholars and their understanding of the gaps within the discipline (Bedeian, 2003). In this way, reviewers assist editors in shaping the scholarly record of the field.

For the public:

With the ease of access to vast amounts of information available online 24-7, peer review is more important than ever to provide a useful way to sort reliable sources from information that has not been validated or may be inaccurate. Peer reviewed journals are one way to ensure that accuracy within various disciplines. Bakanic, McPhail, & Simon (1987) argue that reviewers help the disciplines “by improving the quality of what appears in the journals,” and, therefore, what is read by others in the field (p. 631) and, ultimately, disseminated to the public.

Terminology

The definition of some of these terms, particularly the types of reviews differ slightly depending of the field, the journal or even the editor. The meanings listed here are the most used and accepted ones but are not exclusive and may vary in meaning.

Scholarly versus peer-reviewed- Sources that are scholarly include research and are generally reputable. Scholarly sources that are reviewed critically by experts in the discipline are peer-reviewed. Peer-reviewed sources are scholarly but not all scholarly sources are peer-reviewed.

Single-blind review- in some journals, particularly in medical fields, a single blind review means that the authors’ names and affiliations are known to reviewers, but the reviewers remain anonymous to the author (Okike, Hug, Kocher, Leopold, 2016).

Blind or masked review- a review when the authors names are not included on the manuscript and the authors remain unknown to the reviewer.

Double blind review- Typically means that the authors are not revealed to the authors and the authors are not known to reviewers. In some publications or fields, double blind has the alternate meaning that in addition to author names being withheld from the reviewer, the author’s institutional affiliation is also withheld.

Refereed- Term sometimes used interchangeably with peer-reviewed.

Desk review- the stage after submission and prior to peer review. Often the editor will do a cursory read of a submission to ensure it fits the aims and scope of the journal and merits being sent out to reviewers. If there is not a fit or if the manuscript is glaring lacking in some way, an editor may reject a manuscript at the desk review stage.

As is typical within communities of thoughtful and passionate practitioners and scholars, there is sometimes discussion about the value of peer review. Studies have found that a majority of peer reviewers failed to detect significant methodological errors in studies (Baxt, Waeckerle, Berlin, & Callaham, 1998) and the process has also been identified as slow and inconsistent, prone to bias and abuse (Smith, 2006). Yet for the sometimes negative perceptions of the process on the part of the scholarly community, no better process has emerged. Peer-review remains the highly valued standard by which manuscripts must pass to join the work of scholars.

References

Bakanic, V., McPhail, C., & Simon, R. J. (1987). The manuscript review and decision-making process. American Sociological Review, 631-642.

Baxt, W. G., Waeckerle, J. F., Berlin, J. A., & Callaham, M. L. (1998). Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Annals of emergency medicine, 32(3), 310-317.

Donmoyer, R. (2011). Why writers should also be reviewers. In T. S. Rocco & T. Hatcher (Eds.), The Handbook of Scholarly Writing and Publishing. (pp. 239-250). San Francisco, CA: Jossey-Bass.

Okike K., Hug K. T., Kocher M. S., Leopold S. S. (2016). Single-blind vs double-blind peer review in the setting of author prestige. JAMA 316(12), 1315–1316. doi:10.1001/jama.2016.11014

Smith, R. (2006). Peer review: A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99(4), 178-182.

Spier, R. (2002). The history of the peer-review process. TRENDS in Biotechnology, 20(8), 357-358.

Svensson, L. (1984). Three Approaches to Descriptive Research.