I performed a Kruskal-Wallis test in the NPAR TESTS procedure and now wish to perform a post hoc analysis to see which pairs of groups differ significantly. How can this be done?
Resolving the problem
As of Release 18, the new NPTESTS procedure offers post hoc tests for the Kruskal-Wallis omnibus test. In the menus, select Analyze>Nonparametric Tests>Independent Samples. You will get a Kruskal-Wallis test and post hoc tests automatically if the omnibus test is significant if you specify a grouping variable with more than two levels and one or more test or response variables that are defined as having measurement scale levels of Scale. Note that the full test results for the K-W test and the post-hoc tests are contained in the Model Viewer in the output. You need to double-click on this object in the output to see the omnibus test results, which will be on the right-hand side of the Viewer when it opens. Select the View drop down at the bottom of the screen and Pairwise Comparisons to see the post-hoc results.
Post hoc testing is not offered in the NPAR TESTS procedure, but a series of Mann-Whitney tests can be performed to ascertain which pairs of groups differ significantly from one another. For example, if the test variable is called Y and the grouping variable GROUP has four levels numbered 1 to 4, run the syntax:
/M-W= y BY group(1 2).
/M-W= y BY group(1 3).
/M-W= y BY group(1 4).
/M-W= y BY group(2 3).
/M-W= y BY group(2 4).
/M-W= y BY group(3 4).
In post hoc analyses, one wishes to control the experiment-wise type-1 error rate in order to be assured, for example, of having no more than a five percent chance of making a type-1 error for ANY of the hypotheses being tested. To accomplish this in the present situation, one can apply either a Bonferroni or Dunn-Sidak correction to the significances obtained from the series of Mann-Whitney test results, and, if a corrected significance remains under .05, declare that pair of groups to be significantly different on the test variable Y.
The Dunn-Sidak adjustment is a bit more powerful than the Bonferroni, but a little more difficult to compute. Both will be described here. Let px denote the experiment-wise type-1 error rate, ps a single test type-1 error rate, and k be the number of pairwise comparisons that can be made. The Bonferroni adjustment calculates
px = k * ps ,
and declares that pair of groups to be significantly different at the .05 level if
px < .05. The Dunn-Sidak adjustment calculates
px = 1 - (1 - ps)**k .
Both expressions are upper bounds for the chances of making a type-1 error on any of the k comparisons. Px calculated using the Dunn-Sidak expression will always be a little smaller than if it is calculated using the Bonferroni adjustment.
Note that the analysis described here does not produce results identical to those obtained in Release 18 or later in NPTESTS. The reason for this is that the method used in NPTESTS compares pairs of groups based on rankings created using data from all groups, as opposed to just the two groups being compared. The tests in NPTESTS are known as Dunn-Bonferroni tests and are based on:
Dunn, O. J. 1964. Multiple comparisons using rank sums. Technometrics, 6, 241-252.