Table 1. Definitions of the five Essential Areas of peer review.
The team described best practice in each Essential Area in detail, using the case studies and findings from the literature review, and developed a checklist of questions to help journal teams. The work was shared as a preprint in April 2018 (Allen et al ., 2018) and published after peer review and revision (Allen et al., 2019).
At the same time a smaller group of colleagues created an online tool for journals to use, the ‘Better Peer Review Self-Assessment, Version 1.0’, developed from the checklist presented in Appendix 1 of the preprint. The Self-Assessment was built on Microsoft FormsTM and Microsoft FlowTM, with a dashboard built using Microsoft ExcelTM that partially automates the creation of further feedback in the form of a Better Peer Review Self-Assessment Quartile, Badge, and Data Visualization. Sixteen colleagues attended three workshops in September 2018 in Wiley offices in the USA (Hoboken) and UK (Chichester and Oxford). They completed the Self-Assessment and shared their feedback, which led to improvements to the Self-Assessment. This was then made available to Wiley colleagues as Version 2.0 and is now accessible to everyone, including individuals outside Wiley (https://wiley.com/go/betterpeerreview).
The Better Peer Review Self-Assessment comprises three steps. First, in the ‘Think and Reflect’ step, journals answer 48 questions focused on the five Essential Areas. Journals decide how to approach the Self-Assessment: individual team members can look at the questions prior to undertaking the Self-Assessment as a group or, if preferred, an individual journal team member such as a Managing Editor can first complete the Self-Assessment, then discuss the results with the rest of the team, revise practices accordingly, and then repeat. Journals must answer the question and also briefly explain the rationale for their answer with a free text summary. Next, in the ‘Immediate Feedback’ step, the journal receives instant on-screen feedback as well as an immediate record by email of their answers. Finally, in the ‘Summary Feedback’ step, journals receive another follow-up email with detailed information on their Quartile compared with other journals that have completed the Self-Assessment, Badge, Data Visualization, and some hints and tips for the journals to use, if they wish, to improve their processes. The ‘Badge’ is a radar plot illustrating performance (Figure 1). The Data Visualization breakdown is a histogram comparing a journal’s scores with the mean scores reported by all other journals (Figure 2). The Better Peer Review Self-Assessment therefore enables journal teams to identify their strengths and weaknesses; to find out how their practices across the Essential Areas compare with those of their peers; and to receive guidance about how they might improve their processes.