Preregistration

Version: 0.1, status: concept. Last modified by GtR on May 18, 2020. Version control see here.

Preregistration is a time-stamped, non-modifiable public record of detailed research and analysis plans and helps to avoid selective revealing or suppression of results.

What is preregistration?

Ideally, preregistration of research plans is a procedure that creates a time-stamped, non-modifiable public record of detailed research and analysis plans before the data collection starts. In the 1990s, preregistration of clinical trials was invented to enable patients to find a trial they might want to participate in. Pre-registration is no cure-all. There is excellent research that has not been preregistered and some preregistered research is quite poor. Nevertheless, we believe that with correct use, timely pre-registration in the public domain will help making the total scientific record more reliable.

Preregistration in qualitative research?

One may argue that in qualitative research preregistration of the research design and plan is less relevant, because in qualitative research plans cannot strictly be followed. Qualitative research design needs to be flexible. The process is iterative e.g. in many qualitative research designs the number of participants cannot be predetermined. Still, we propose to preregister the research plan, because making the plan available for the research community will enhance transparency in qualitative research.

Why is preregistration important?

Here we define preregistration as a publicly available building block in a wider system that helps counteract suppression (or selective revealing) of results and publication bias. It thus helps provide an unbiased public record of all research findings worldwide. Preregistration also prevents data dredging and finally, you provide examples for others who need to write a protocol or a SAP.

How might preregistration of the study protocol and SAP benefit you?

First, preregistration (making public your research plan and SAP) creates a strong incentive to carefully write these documents. It also serves as proof (DOI, trial registration number) to editors, reviewers and others that you stuck to your (last amended and time-stamped) plan. Second, it creates a stronger incentive to do as you promised and to stick to your (analysis) plans. Third, preregistration can prove priority of ideas, which still is important to many scholars. Fourth, preregistration increases your readers’ confidence that you are reporting honestly. Fifth, you contribute to the reliability of systematic reviews in which your research may end up.

How might preregistration of the study protocol and SAP hurt you?

First, if you make your plans public, others may see what you are planning and scoop you (i.e. copy the idea and publish (results) before you). Second, preregistration, especially if made public, is like voluntarily tying your hands and restricts your flexibility (during data analysis). Sadly, finding fewer statistically significant results might hurt your research career. Let us put these seemingly strong arguments into perspective. First, only seldom will you have truly unique plans. Ideas are in the air most of the time and colleagues are working on very similar stuff. Second, we already explained that voluntarily reducing your data-analytic flexibility is not a bad thing (you gain time and there is less opportunity for data-dredging). Finally, the focus on statistically significant findings is a very poor focus. If you want to understand more about the misinterpretations of p-values and the behaviors that it brings about read what the American Statistical Association had to say on this.

What, when and where?

What should you preregister?

  • For quantitative research: preregister a research protocol and the plan for data analysis (SAP).
  • For qualitative research: preregister the research plan including the design.

When should you preregister?

Preregister before the first participant (study unit) enters the study. Do it after you have obtained ethics approval by an ethics committee to incorporate any changes that such assessments may bring about.

Where should you preregister?

If your research is a trial or falls under the Dutch law (WMO) and needs a Trial Registration Number (TRN), then you should preregister in the Netherlands Trial Register (NTR). Mind you, NTR does not allow registering retrospectively. In all other cases, we advise preregistering in UvA/HvA figshare. If you really dislike using UvA/HvA figshare for preregistration, consider using the Open Science Framework (OSF). Here are examples of preregistrations on OSF.

Are there any alternatives to preregistration via UvA/HvA figshare (and NTR)?

Yes, you may submit your protocol (and SAP) to:

  1. an Open Access journal as a formal publication (amendments cannot easily be made, but this approach can be combined with the use of UvA/HvA figshare)
  2. a preprint server such as medRχiv. Here you can check if your target journal will accept your manuscript after it has been out on a preprint server.
  3. a journal accepting registered reports.
  4. a journal that is between a preprint server and a traditional journal, such as F1000.