How to develop Inclusion and Exclusion Criteria for your scientific reviews

When developing a plan of action for tackling a systematic review or a meta-analysis, one of the most important steps is developing a standardised and meaningful way of deciding which pieces of work to include and which not to. Doing this is so critical because first and foremost, it’s impossible to include absolutely every study in your review, due to the sheer size of the task. Secondly, it would likely produce irrelevant results. Several factors can influence a study outcome, which can bias your review. Here, we’ll cover what to think about when developing these criteria.

Purpose of inclusion and exclusion criteria

The main reason inclusion/exclusion criteria exist is because factors that are outside of a study organiser’s control can influence the outcome of the study. Moreover, your particular research question may only apply to a handful of studies, and including certain other studies could influence your analysis and therefore weaken it. 

Process of developing inclusion and exclusion criteria

1. Start by compiling a broad range of studies

This is the pool of literature you will begin narrowing down from. Indeed, many researchers prefer to narrow down from a large set of studies, rather than start with inclusion criteria. 

2. Consider your research question

Broadly speaking, you’ll be focusing on a specific part of a study, usually the methods or outcome measures. Scan your literature for the aspect you are focusing on, and start to exclude papers that don’t apply to your research question.

Tip: There are great tools available to automate this process, one example being NVivo software. 

3. Iteratively start excluding literature based on specific criteria you set for yourself

As leading factors that could be easy starting points, consider factors such as:

• Publication date
• Geographic location
• Peer review: Determine whether peer review is an important factor, and if so, think about excluding preprints.
• Setting: For example, medical studies can take place in a variety of settings, such as hospitals or within patients’ homes.

Setting inclusion and exclusion: Some points to note

a. Choose only factors that are relevant to your question

For example, the study date may not actually factor into what you are assessing. However, if there is a very large body of studies, you can set yourself a cut-off year, where you won’t consider anything earlier than this date, in order to make the study manageable for yourself. Similarly, think about whether a previous review already covered a specific span of time, so that your research could pick up where they left off.

b. There is no right or wrong way – it’s an iterative process

When it comes to mammoth tasks such as systematic reviews or meta-analyses, the main thing to remember is:

The more you take into consideration in your study, the more convoluted things become.

It is therefore good practice to keep your pool as limited as possible, while including everything you can that ticks the right boxes for your study. This may mean that you initially set yourself inclusion and exclusion criteria, but find that you need to go back and iterate, and start over again.

c. There is no set protocol for the selection

Overall, this process does not have a set protocol. The PRISMA checklist does refer to inclusion and exclusion criteria, but only specifies that a justification for each is required. So, as long as you have a reasonable justification, your criteria are acceptable.

End notes

Whatever your research question, it is imperative that you screen available literature for quality, and to identify and collect what applies to your specific question. Think about what could significantly ‘sway your results’, and base your criteria on determining ‘signal over noise’ within the abundant wealth of published data.

Maximise your publication success with Charlesworth Author Services.

Charlesworth Author Services, a trusted brand supporting the world’s leading academic publishers, institutions and authors since 1928. 

To know more about our services, visit: Our Services