Preregistration

Preregistration is the practice of registering your detailed research plan before conducting a study. The preregistered report format requires researchers to submit a description of the confirmatory hypotheses, variables, study methods, and analysis plan prior to data collection.

This practice allows researchers to circumvent the publication bias toward significant findings and prevent the data from taking you hostage. Preregistration also makes the distinction between hypothesis testing and exploratory (hypothesis generating) research more clear. As a result, the obtained results won’t affect the hypothesis and vice versa.

Click here to see some examples of preregistrations segregated by discipline and study type.

Types of preregistration

1. Unreviewed registration report - contains a detailed description of the researcher’s plans for a study as possible, and the researcher saves those plans in a time-stamped, uneditable archive; can be shared with reviewers, editors, and other researchers.

2. Reviewed registration report - researcher submits a detailed proposal for a study to a journal before conducting the study. Reviewed registration reports have the same virtues as preregistration, but they also address the problem of publication bias: results of the study are published regardless of their outcomes.

3. Registered replication reports (RRR) - researchers conduct replication of one or more original findings. Many labs follow the same preregistered plan, and the results from all of these independent studies are published collectively regardless of the outcomes of individual studies.

How to make a preregistration?

Domain-agnostic template

Follow a template from Open Science Framework at Word / GoogleDocs or AsPredicted and fill it with your own research ideas.

To illustrate the preregistration process, we created a simple example of the whole process:

fMRI research template

To be more specific and for the purposes of this documentation, we extract a preregistration report specifically for fMRI research, by clicking here (Flannery, 2020).

Any solution put in following template selected by the author is a suggestion of practices for reported fMRI methods put forth by Nichols et al., 2016 & Poldrack et al., 2008. The whole documentation is available on GitHub so everyone can contribute and improve the whole statement.

Rules of preregistration

1. Once registered, your preregistration will have a short URL for citation. Remember to include a link to your registration report.

2. Report the results of ALL preregistered analyses regardless of outcome.

3. ANY unregistered analyses must be transparently reported as exploratory finding. Do not stop yourself before making transparent changes to the analysis plan but remember to report the change and its substantive justification.

Why all researchers should preregister their studies?

In 2016, Eric-Jan Wagenmakers and Gilles Dutilh published the article where they highlight the personal benefits of pre-registration for increasing the potential of success in an academic career. In the paper, authors presented seven selfish reasons for preregistration. In particular, that reasons included:

1. Preregistration allows you to get credit for having been able to anticipate.

   By sharing your research plan you prove yourself that you are able to predict your data processing flow and research outputs before conducting a study.

2. Preregistration is exciting.

   The excitement comes from complex project planning and answering the big question - if the theory will be confirmed?

3. Preregistration prevents you from being taken hostage by your own data.

   Without preregistration you might think that the results you obtained based on particular data, determine your future success in publishing and science in general. As a consequence, you might feel urged to work on your data as long as necessary to get something out of it. After preregistration, you reduce the number of possible methods and questions to only those that are solely related to your research interest.

4. Preregistration is easy.

   When a scientist has an idea for his work, the pre-registration process is straightforward, mostly thanks to existing preregistration templates (see subsection How to make a preregistration?).

5. Preregistration increases your reputation.

   Open sharing of the hypotheses and methods for a study is the best determinant of researchers’ confidence. It also shows that scientists wish to transparently conduct science and have done everything to shield themselves from hindsight bias and confirmation bias. Scientists who openly share their work are cited more often because their research results can be widely replicated and trusted.

6. Preregistration allows you to have manuscripts accepted “in principle“ regardless of the results.

   It is possible to make a preregistration proposal to a journal that offers Registered Report format and after approval, the (see subsection Types of preregistration). After the Registered Report’s approval, the journal ensures publication of the results regardless of whether they confirmed the hypothesis. It also allows you to improve the registered report, thanks to the reviewer’s comments before the data collection process begins.

7. Preregistration can shield you from post hoc critique.

   When you preregister your research, you might receive a constructive critique to your study before the start of data collection. Thus, it protects you from being criticized based on obtained results, as your research plan was already approved by the scientific community.