TWe have all heard the phrase “peer review” used to credit research and academic work, but what does it really mean? How does it work?
Peer review is one of the gold standards of science. It is a process where scientists (“peers”) evaluate the quality of the work of other scientists. By doing this, they aim to ensure that the work is rigorous, coherent, uses past research and adds to what we already knew.
Most scientific journals, conferences, and grant applications have some type of peer review system. In most cases it is a “double-blind” peer review. This means that the reviewers do not know the author(s), and the author(s) do not know the identity of the reviewers. The intention behind this system is to ensure that the evaluation is not biased.
The more prestigious the journal, conference, or grant, the more demanding the review process and the more likely rejection will be. This prestige is the reason why these papers tend to be more read and more cited.
The peer review process
The peer review process for journals involves at least three stages.
If you value articles like this, consider supporting us by becoming a Pro subscriber. Subscribers enjoy access to members-only articles, materials, and webinars.
1. The desktop evaluation stage
Papers that reach scientific journals are sent to the editor-in-chief or associate editor with relevant experience for initial evaluation.
At this stage, anyone can “reject” the paper: that is, reject the paper without sending it to the blind reviewers. Generally, articles are rejected if the paper does not fit the scope of the journal or if there is a fundamental flaw that makes it unsuitable for publication.
The intention behind this system is to ensure that the evaluation is not biased
In this case, rejecting editors can write a letter summarizing their concerns. Some magazines, such as they reject up to two-thirds or more of the articles.
2. The blind review
If the editorial team judges that there are no fundamental defects, they send it for review to the blind reviewers. The number of reviewers depends on the field: in finance there may be only one reviewer, while journals in other social science fields may request up to four reviewers. These reviewers are selected by the editor based on their expert knowledge and their lack of a connection to the authors.
The reviewers will decide whether to reject the document, accept it as is (which rarely happens), or ask for the document to be reviewed. This means that the author needs to change the paper according to the reviewers’ concerns.
Some magazines, such as British Medical Journal they reject up to two-thirds or more of the articles
Typically, reviews address the validity and rigor of the empirical method, and the importance and originality of the findings (referred to as the “contribution” to the existing literature). The editor collects those comments, evaluates them, makes a decision, and writes a letter summarizing the reviewers’ comments and their own concerns.
Therefore, it may happen that despite the hostility of the reviewers, the editor could offer the journal a subsequent round of review. In the best social science journals, 10% to 20% of articles are offered a review and resubmission after the first round.
3. The reviews – if you’re lucky
If the paper has not been rejected after this first round of review, it is returned to the author(s) for review. The process is repeated as many times as necessary for the editor to reach a point of consensus on whether to accept or reject the document. In some cases this can last several years.
Ultimately, less than 10% of submitted articles are accepted into top social science journals. The renowned magazine publishes around 7% of the works presented.
Strengths and weaknesses of the peer review process
The peer review process is considered the gold standard in science because it ensures rigor, novelty, and consistency of academic results. Typically, through rounds of review, bad ideas are eliminated and good ideas are strengthened and improved. Peer review also ensures that science is relatively independent.
The reviews address the validity and rigor of the empirical method, and the importance and originality of the findings.
Because scientific ideas are judged by other scientists, the crucial criterion is scientific standards. If other people from outside the field participated in evaluating ideas, other criteria, such as political or economic gain, could be used to select ideas. Peer review is also considered a crucial way to remove personalities and biases from the process of judging knowledge.
Despite undoubted strengths, the peer review process as we know it has been . It involves a series of social interactions that can create biases; for example, reviewers can identify authors if they are in the same field and desk rejections are not blind.
It might also favor incremental research (added to past research) rather than innovative (new) research. Finally, reviewers are human after all and can make mistakes, misinterpret elements, or miss errors.
Is there any alternative?
Proponents of the peer review system say that while there are flaws, we have not yet found a better system for evaluating research. However, several innovations have been introduced in the academic review system to improve its objectivity and efficiency.
Some new open access journals (such as ) publish articles with very little evaluation (they verify that the work does not have methodological flaws). The focus is on the post-publication peer review system: all readers can comment and criticize the article.
Reviewers are human after all and can make mistakes, misinterpret elements, or miss errors
Some magazines like Naturehave done (“open” review), offering a hybrid system in which peer review plays a role as primary gatekeepers, but the public community of academics judges in parallel (or subsequently in some other journals) the value of the investigation.
Another idea is to have a set of reviewers who rate the document each time it is reviewed. In this case, authors will be able to choose whether they want to invest more time in a review to obtain a better rating and obtain public recognition of their work.
Article written by: professor of organizational behavior at the University of London and , researcher at the University of Oxford.
Article published in and transferred for publication in .com