Thoughts from AMS 2024 Editor’s Award Recipients
Understanding the role of peer review in science is vital not only for scientists themselves, but also for all of us who live in a society that relies on scientific research. Each September during Peer Review Week, AMS and other scholarly publishers highlight the essential role that peer review plays in scholarly communication.
In this first of two Peer Review Week posts, we’re hearing from some of AMS’s outstanding peer reviewers, recipients of the 2024 Editor’s Award, about what they think non-researchers need to understand about peer review.
At a basic level it is a check on, “do I believe the results presented here and the implications that are claimed?” The check is made by other researchers working independently in the field. The checking of a single paper isn’t exhaustive, but there is an ongoing process—results and ideas established in one paper will, if they are of any significance, be re-examined and developed further in subsequent papers, which will themselves be peer reviewed.
Dr. Peter Haynes, Cambridge University
For non-researchers, I think it’s important to understand that peer review requires substantial effort … by volunteer reviewers, editors, and [the] scientists submitting manuscripts. A well-coordinated review process by all involved improves the quality and ensures the integrity of scientific research.
Dr. David Bodine, University of Oklahoma
The peer review process is the only barrier standing between the writing of a scientific study and its publication. It is very easy for an unsubstantiated or erroneous paper to set an entire field back by years. In my opinion, the peer review process is as important as the research that goes into a given paper. … A given scientist may have one perspective and associated biases. Through peer review, the results are assessed by other scientists and are judged against the state of knowledge of the field. To push our boundary with the unknown forward, scientists must rely on both the historical backbone of their field as well as thorough review by their modern-day peers.
Dr. Elizabeth Yankovsky, New York University
Just because a paper was published after undergoing “peer review” does not make it absolutely correct or perfect, nor is it the final message on that idea. Unfortunately I feel the phrase “peer reviewed” is often used to imply some absolute consensus on a subject has been reached, when in reality it’s an ongoing, necessary criticism of the science that we do. As scientists we are constantly peer-reviewing each other’s work, and this may spark new, contrary ideas to be published that refute earlier findings.
Dr. Luke Madaus, Jupiter Intelligence
Popular culture oftentimes misconstrues science in either the extreme of being purely political/agenda-driven or being the ultimate source of truth without question. The truth is, science is not ultimate, and understandings of processes and concepts are dynamic, and, as is especially evident in interdisciplinary research, scientists do have perspectives shaped by experiences. While I also cannot deny that bias exists in scientific fields and that contrasting perspectives may filter out at times, authentically anonymous and double-anonymous peer review processes (i.e., the reviewer not knowing the authors’ name(s)), act as guardrails to significantly reduce bias. [If] these processes remain clean and the selected reviewers are indeed experts in the field of the papers that they review, this significantly aids in ensuring that the end product is of the highest quality. The existence of these processes is why academic journals are deemed among the most credible sources of scientific information.
Dr. Sarah Buckland, University of the West Indies
I think what I wish non-researchers understood about the scientific review process is how many eyes are on these papers before they’re published, and how that regularly improves the science and writing of the end-product. Most reviewers take the responsibility very seriously, and indeed are usually hesitant when they haven’t “found enough to fix” in a paper they review. It is typically a truly rigorous process.
Dr. Mimi Hughes, NOAA Physical Sciences Laboratory
I think non-researchers should know that peer review is only good and valuable when it is conducted from an unbiased position. It is vitally important that authors receive unbiased, external perspectives on their work in order to ensure that any gaps or misunderstandings can be addressed, and that the science is technically sound. Peer review is just ONE step in the scientific process as well, and sometimes bad work slips through the cracks of review. But peer review is a critical component to upholding and advancing science.
Dr. Aaron Hill, Colorado State University
Peer review serves as a crucial method for the scientific community to uphold the quality and credibility of scientific information accessible to the public. A paper passing peer review doesn’t guarantee absolute perfection, it indicates a level of examination and approval by experts in the field to some extent at the current time.
Dr. Qiaohong Sun, Nanjing University of Information Science and Technology
Peer review is a critical control mechanism in the scientific process. Mistakes can happen and may still get through the process. However, the collective nature of peer review and subsequent scrutiny by the scientific community help correct errors over time. This in particular highlights the importance of making research reproducible by publishing data and code.
Dr. Sebastian Lerch, Karlsruhe Institute of Technology
It works! It is the main mechanism that keeps science reliable and transparent. Scientists respect and cite published work. In order to get science published, it needs to be read by 2-4 anonymous colleagues and editors and then revised. Even when it goes wrong and a paper is published with an error or not-well-supported argument, researchers are good at detecting this after the fact. It is a robust process that keeps the advancement of knowledge at a high-quality and transparent level.
Dr. Andrew Feldman, NASA Goddard Space Flight Center