Significance and use of Post-hoc Analysis studies

Post hoc in Latin means ‘after this’. Simply put, a post-hoc analysis refers to a statistical analysis specified after a study has been concluded and the data collected. A post-hoc test is done to identify exactly which groups differ from each other. Therefore, such tests are also called multiple comparison tests.

Purpose of post-hoc test

A priori comparisons are performed before the data are collected, and post-hoc (or a posteriori) comparisons are done after the data have been collected. When the null hypothesis of an analysis of variance (ANOVA) model is rejected, post-hoc tests are used to identify the population means that are different.

When the null hypothesis is rejected in an omnibus test (a test that provides overall results for study data, e.g. ANOVA), it means that at least one parameter is significant. The question to be asked at this stage is:

Which groups significantly differ from the others in terms of the mean?

Post-hoc tests: Examples

Some post-hoc tests are:

• Bonferroni’s test
• Tukey’s honest significant difference test
• Scheffe’s test

Types of studies that commonly use post-hoc analysis

A post-hoc analysis can be conducted for proportions and frequencies, but it is mostly used for testing mean differences. The following types of research involve post-hoc analyses.

A. In any discipline, studies investigating differences between groups will use post-hoc tests when the null hypothesis of an ANOVA model is rejected. 

Here is an example.

A researcher wants to investigate differences in the effectiveness of TikTok, Instagram and Facebook influencers in promoting a nutraceutical brand. Let’s say that, by ANOVA, the null hypothesis (that all three influencer types have similar effectiveness) is rejected. A post-hoc pairwise comparison may then reveal that Instagram influencers have a significantly higher effectiveness in promoting the brand than TikTok and Facebook influencers, while the latter two are similar.

B. In medicine, post-hoc analyses may be used in clinical trials if the original hypothesis does not hold (e.g. the primary outcome being the antidiabetic effect of a drug). Triallists then re-examine the dataset for other outcomes (not originally planned, e.g. improvement in renal outcomes in diabetes patients) and perform statistical analysis to determine other valuable results from the trial. 

Note: For most clinical trials, the research questions and statistical tests must be defined before observing the research outcomes, even before the first patient is enrolled. Primary, secondary and exploratory outcome measures should be established beforehand, while post-hoc outcome measures can be specified after the trial has started. This ‘pre-registration’ avoids the practice of outcome switching (reporting something different from what was originally planned). Pre-specified and post-hoc outcome measures must be clearly indicated in the analysis section, in a way that makes it possible to readily distinguish between them.

C. Analyses of pooled data from completed trials comprise a type of post-hoc study as well.

End note

Remember that post-hoc analysis differs from post-hoc theorising or ‘hypothesising after results are known’ (HARKing). To avoid revising or formulating the hypothesis after data collection, begin with your preferred theoretical framework and develop the hypotheses that logically follow from within that framework.

Maximise your publication success with Charlesworth Author Services.

Charlesworth Author Services, a trusted brand supporting the world’s leading academic publishers, institutions and authors since 1928. 

To know more about our services, visit: Our Services