As a career scientist, I review scientific papers regularly, at least once every month. For instance, I reviewed a total of 18 papers in 2021, and many of them 2 ~ 3 times for revisions. I think I can speak for almost everyone that nobody really enjoys reviewing papers. We do it because it is a duty. Well, someone has to review our papers. So, we will need to do the same for others, right? In my earlier days, I used to spend several days reviewing one paper. Now, I usually finish my review within 3 ~ 4 hours. I first read the manuscript briefly, abstract, plots, results, and conclusion in that order to get some ideas about the paper, before reading the entire paper. Obviously, well-written papers are very easy to follow. So, I can finish my review and make a recommendation quickly. But, a large number of the manuscripts I have reviewed so far were not so well-written. So, I usually ended up spending too much time correcting sentences after sentences before I could evaluate the scientific content and contribution of an article. That’s the part that I hate the most with a paper review. When that happens, I get exhausted after completing my review, and my productivity goes down very quickly afterward. So, I usually start my review in the afternoon around 2 pm. Of course, the review often continues until 10 pm or to the next day.
In some journals including all American Meteorological Society (AMS) journals, after submitting my review, I can read reviews from other reviewers and the decision letter. I often find myself being more critical than other reviewers. When that happens, I feel guilty and ask myself if I am turning into a harsh and bitter senior scientist. Probably, I am. But, before submitting my review, I always ask myself if it would be acceptable to me if someone judges my own work with the same level of scrutiny. I see only one thing before I make the final recommendation for a paper. What is the new contribution of this paper, and does it advance our understanding? If the answer is “no” or “unclear”, my recommendation is a firm “rejection” regardless of how well a paper is written.
Sometimes, I am asked to review a paper written by my former colleagues or collaborators. I have reviewed some of those papers because I felt confident that I could serve as an impartial reviewer. But, rejecting a paper written by my friends or close colleagues sometimes negatively affects my mental health. So, I tend to decline such review requests with a “conflict of interest” note.
I found many useful tips about paper review in the following articles and web links. One thing that caught my eye is that writing something like “expect the revised manuscript to be proofread by native English speakers” in a review letter is inappropriate and no longer acceptable (Schultz, 2022). So, be careful out there my fellow reviewers!
Image Credit: https://syncedreview.com/2020/04/10/its-time-to-improve-the-scientific-paper-review-process-but-how/
Benos, D. J., K. L. Kirk, and J. E. Hall. 2003: How to review a paper. Advances in physiology education, 27, 2, 47-52. https://doi.org/10.1152/advan.00057.2002
Schultz, D. M. 2022: How to be a more effective reviewer, Monthly Weather Review, 150, 6 1201-1205. https://doi.org/10.1175/MWR-D-22-0102.1
Ocean to Climate 
Comment from Renellys Perez:
Hi Sang-Ki
Thank you for sharing. My approach is similar. I try to read the paper and write up all of my comments/suggestions in one day. The next day, I read over and polish my comments. This could all be done in the course of one day, but I find that pausing and reflecting on my review prior to submitting it allows me to smooth out the rough edges of my comments, remove repetitive suggestions, and perhaps reword the suggestions so that they are as helpful as possible. On average, I try to review three papers for every paper that I submit in a year.
Best,
Renellys
Comment from Rik Wanninkhof:
Hi Sang-Ki
Many thanks for sharing. Reviewing papers is a key requirement to maintain a high level of scholarship and scientific excellence at AOML and in the community at large. Your helpful hints in your blog are very useful. A few additional notes:
The number of requests for review is getting a bit daunting and your suggestion as to # of reviews one should commit to seems on point
Accepting a variety of types of articles for review is useful and keeps things interesting.
Communicating with the editor, either in the dedicated section of a review or directly by e-mail is good when there are issues or ambiguities.
Read reviewer instructions and check if there is a reviewer template, as different journals have different requirements
Specify in the review if there are aspects or sections of the paper that you do not feel qualified to assess
One of the reasons I review is to improve my knowledge (“learn something new”) on a particular topic and/or keep up to date with the latest developments
To decrease the chance of providing an unduly harsh review, I always include my name
Rik W.