Cochran–Mantel–Haenszel (CMH) test

A non parametric test, it does not require normal distribution or variance assumptions about the population from which the samples are drawn.

Purpose:
It allows the comparison of two groups on a categorical response after controlling for variation in response due to a control variable.
CMH test statistic was introduced to extend the chi square test of independence in a 2×2 table to multiple 2×2 tables where each table corresponds to a different level of a control variable

Scope:
The Cochran–Mantel–Haenszel (CMH) test is widely used to measure the strength of the association between an exposure and disease or response, after stratifying on the observed control variable. Thus, observed confounders are accounted for in the analysis.
CMH statistics have low power for detecting an association when the patterns of association for some of the strata are in the opposite direction of the patterns displayed by other strata. As a common practice, direction of association is infered by the value of odds ratio. Odds ratio (significantly) less than 1 is considered as negative association as odds of success in 1st category is less than odds of success in 2nd category while odds ratio (significantly) greater than 1 is considered as positive association. Odds ratio equal to 1 is considered as no association.

For example, consider the following odds ratio plot where a, b, c and d are levels of control variable. CMH statistics for this case will be insignificant because odds ratio for levels a and b are in opposite directions of association.

Note: Here odds ratios of levels a and b are nearly equidistant from the value 1, the no association odds ratio. hence We got insignficant CMH statistic i.e. p-value > 0.05

So by stating that CMH statistics have low power, we mean CMH lacks power to detect signficance in scenario like the above mentioned one.”
The stratification of the subjects into groups (according to the values of controlled variables – e.g. “Age group”) increases the power of the test to detect association. This increase in power comes from comparing like subjects to like subjects.

If the number of groups is large, but the sample sizes within the groups are small, then the CMH test can be recommended.

Basic assumptions and requirements :

  • The sample is drawn randomly from the population.
  • Data are reported in raw frequencies (not percentages).
  • Observed variables are independent
  • Breslow-Day Test shows signficant p-value (<0.5) for the data considered

Test for assumptions
One of the main assumption of CMH test is that the association between two categorical test variables are significant different for various levels of control variable. If this assumtion is not satisfied then it means control variable has no significant effect on association between test variables and there is no point doing CMH test because it will simply complitcated the procedure. So for such cases, a simple chi-square test is recommended.

Breslow-Day Test (test for significance of control variable):
The test helps to check whther there is neeed to do CMH test by checking for significance of effect of control variable on the association between test variables. It checks for singificance of magnitude difference of odds ratios. If at least magnitude of one of the odds ratio is different others then we will get a signficant p-value indicating the need for CMH test.
Note: No additional options are required to perform Berslow-day’s test. By default we get this test results in the output whenever we perform CMH test in SAS.

Methodology