Reviewing Registered Reports

Registered Reports are a form of empirical article offered by a number of journals at Wiley in which the methods and proposed analyses are pre-registered and reviewed prior to research being conducted. High- quality protocols are then provisionally accepted for publication before data collection commences. This format is designed to minimize publication bias and research bias in hypothesis-driven research, while also allowing the flexibility to conduct exploratory (unregistered) analyses and report serendipitous findings. The review process has two stages:

  1. Stage 1 (study design): Reviewers assess study proposals before any data collection.
  2. Stage 2 (completed study): Reviewers evaluate the full study, including results and interpretation.

Guidelines for reviewers

Stage 1: Study design

In Stage 1, manuscripts should contain the following sections: Introduction, Methods (with proposed analyses), and Pilot Data (if applicable). As a reviewer, you will be asked to evaluate the following:

  1. Research significance: Is the study’s focus relevant and impactful in its field?
  2. Hypothesis and measurement: Are the hypotheses clearly defined, logical, and aligned with theory, methodology, and analysis? Are they measurable and coherent?
  3. Methodological soundness: Is the study design sound and feasible? Does it include clear methods, defined variables, participant/sample details, conditions, and statistical power analyses?
  4. Replicability: Is there sufficient methodological detail to allow exact replication of the study and analyses?
  5. Outcome-neutral tests: Are there pre-specified quality checks and controls to ensure results can genuinely test the hypotheses?

After Stage 1 peer review, manuscripts may be accepted, offered the opportunity to revise, or rejected outright. Accepted manuscripts receive an in-principle acceptance (IPA), meaning the article will be published pending successful completion of the study according to the pre-registered methods and analytic procedures, as well as a defensible and evidence-based interpretation of the results. Some journals may publish the Stage 1: study design after it has received an IPA, in addition to the final Stage 2 completed study; others may publish only the final Stage 2 study. Check your journal’s specific Registered Reports guidelines for details.


Stage 2: Completed study

Following completion of the study, authors will complete the manuscript, including Results and Discussion sections. Manuscripts will more closely resemble a regular article format. Reviewers then assess the manuscript’s adherence to the pre-registered methods and evaluate the findings. The key points for review include:

  1. Data validity and clarity: Assess if the data collected align with the hypotheses and satisfy the outcome-neutral conditions (e.g., quality checks and positive controls). Ensure data presentation is clear and transparent, and results appear robust enough to test the original hypotheses.
  2. Consistency with Stage 1 submission: Confirm that the introduction, rationale, and hypotheses remain consistent with the approved Stage 1 submission (required).
  3. Protocol adherence: Review adherence to the pre-registered methods. Where appropriate, floor and ceiling effects, positive controls, or quality checks should be reported. Note any major deviations from the pre-registration.
  4. Impact of findings: Do the results support the study’s conclusions?
  5. Post hoc analyses: If the authors include unregistered (post hoc) analyses, check if these are justified, methodologically sound, and enhance the study’s findings.
  6. Justification of conclusions: Ensure that the authors’ conclusions are well-supported by the data presented.
  7. Overall clarity and coherence: Ensure the manuscript is clearly written, with well-organized sections and a logical flow that effectively communicates the research process, findings, and insights.

In Stage 2, reviewers may suggest further post hoc tests, but authors are only required to perform those needed to satisfy one or more of the Stage 2 review criteria. Editorial decisions will be based on adherence to the approved protocols and experimental design in Stage 1 and conclusions supported by data (even if they are negative findings) as opposed to novelty and perceived importance of results.