17 Nonparametric Hypothesis Testing

17.1 Nonparametric Procedures in General

Ultimately, hypothesis testing procedures for the mean that rely on \(Z\) or \(t\) statistics have theory that is founded upon normality assumptions. The standardized test statistic, \[T = \frac{\overline{X} - \mu_0}{S/\sqrt{n}},\] has a \(t_{n-1}\) distribution under the null whenever \(\overline{X}\) is normally distributed. This is typically justified in one of two ways. First, many quantities that arise in practice are very close to being normally distributed. As a result, many physical quantities that we are interested in will have a population that is conducive to this testing in this manner. Alternatively, by appealing to the Central Limit Theorem, whenever the sample size is large enough, regardless of the population distribution, \(\overline{X}\) should be approximately normally distributed. As a result, in large samples, these \(t\) and \(Z\) procedures can be justified. Depending on how strongly you desire rigorous justifications for mathematical procedures, these justifications may feel inadequate to support the widespread use of \(t\)-tests. In both cases, the theory that has been developed is only approximately justified. This may lead one to question whether there are alternative procedures to perform the same types of hypothesis testing, but which do not rely on assuming that the estimator follows a normal distribution.

This question goes beyond the setting of hypothesis testing. Many statistical procedures rely either explicitly or implicitly on assumptions regarding the population distribution, at least for their theoretical justifications. These procedures are referred to as parametric methods since they rely on assumptions regarding the parametric form of the distribution.

Definition 17.1 (Parametric Method) A parametric method or parametric technique is a statistical procedure that relies on assumptions regarding the specific structure of the distributions of the random variables of interest. Typically, this will come in the form of assuming that the data follow some specific distribution, even if the values of the parameters are left unspecified.

Parametric methods are incredibly useful, and largely serve as the foundation for the most widely applied statistical methods. The concern with parametric methods, as discussed for the specific case of hypothesis testing, is that these methods are theoretically justified only when the parametric assumptions hold. If a population is assumed to follow a normal distribution, and it does not in fact follow the normal distribution, the theory can no longer provide guarantees as to the performance of the procedure. This does not necessarily mean that the procedure will not perform well, however, it does mean that justification for the application of the procedure is lacking. There are secondary approaches in statistics that remedy this, broadly referred to as nonparametric methods.

Definition 17.2 (Nonparametric Method) A nonparametric method or nonparametric technique is a statistical procedure that does not rely upon strict assumptions regarding the structure of the distribution of the underlying random variables. Nonparametric techniques may make minimal assumptions regarding the shape of the distribution, such as assuming that the population distribution is symmetric. These assumptions are typically far less restrictive or easier to assess in practice.

Nonparametric techniques are designed to be flexible, and to apply well in a wide range of scenarios. This way, there can be confidence in the results obtained by applying the procedure, even when little is known or can be assumed about the underlying population distribution. This makes nonparametric methods attractive in a variety of settings. In the previous context, where we desire hypothesis tests for the mean that are not dependent on assumptions of normality, we can state this more plainly as desiring a nonparametric hypothesis testing procedure.

17.2 Nonparametric Hypothesis Tests for Location

It is frequently the case that scientific questions of interest center on the location of a population. Often this will be questions regarding the mean of a distribution, though, in certain contexts, the median or similar measures may be of particular interest. As a result, commonly used procedures for statistical inference are commonly motivated by questions relating to population locations. This is true for both parametric and nonparametric techniques.

When considering the application of nonparametric statistical inference, it is important to recognize some intricacies that arise when we are unwilling to make concrete assumptions regarding the population distribution. Notably, if we are willing to assume that the population is well-behaved,¹ then we are able to talk concretely about the parameters that specify these distributions. If instead we want to use a nonparametric technique, we forego the capacity to make these nice generalizations. By leaving the distributions we are considering unrestricted, it can be challenging to even specify the null or alternative hypothesis that we are interested in testing.

There are many distributions that are particularly poorly behaved. For instance, there are distributions that do not have means, or do not have variances. This is a strange concept, to have a distribution that is well-defined, and may very well describe data arising from some population, but which does not possess a mean or a variance. Still, if we want to consider truly nonparametric techniques, it is important that our methods recognize and account for this reality.

When approaching hypothesis testing nonparametrically, instead of discussing hypothesis tests for the mean of a distribution, we more commonly discuss hypothesis testing for the location of a distribution. The location of the distribution (Definition 6.1) may correspond to the mean, when it exists, or the median, or similar related measurements. Despite this linguistic difference, nonparametric tests for the location of a distribution are analogous to parametric tests for the mean of a distribution.

Just as is the case with parametric hypothesis tests for the mean, there are multiple nonparametric hypothesis tests for the location of a population. These tests apply in different settings, depending on the available data, the specific parameter of interest, or the assumptions that the analyst is willing to make.

17.3 The Wilcoxon Signed-Rank Test

The simplest setup for a nonparametric test of location takes a single sample, denoted \(X_1, \dots, X_n\), from an unknown distribution. To begin, assume that there are no zeros in the data, and that no two observations have the same absolute value. While we do not assume any parametric form for the distribution, we will assume that the distribution is symmetric. In this setting, we are concerned with testing a null hypothesis of the form \(H_0: \mu = 0\) versus the alternative \(H_A: \mu \neq 0\). Here, \(\mu\) represents the location of the distribution. Because we assume that the distribution is symmetric, the mean² will coincide with the median, and as such, there is a candidate for a single measure of location. We may also wish to consider one-tail alternatives, either taking \(H_0: \mu \geq 0\) versus \(H_A: \mu < 0\), or \(H_0: \mu \leq 0\) versus \(H_A: \mu > 0\).

To test the hypothesis in this setting, we can make use of Wilcoxon’s Signed-Rank Test. In order to perform this test we need to consider the signed-rank of every observation, and then use these to compute a test statistic. Consider first the magnitude of each observed data point. That is, take \(|X_1|, |X_2|, |X_3|, \dots, |X_n|\). Each of these will produce a value that is (at least) zero. Further, by assumption, none of these values repeat. We can then consider ranking these magnitudes from smallest to largest, and assigning them their corresponding rank, \(R_1, R_2, \dots, R_n\).

For instance, suppose that we observe the sample \(\{-2, 1, -7, 9, 6\}\). Then, we first consider the magnitudes of each term in the sample, giving \(\{2, 1, 7, 9, 6\}\). Next, we sort this set from smallest to largest, giving \(\{1, 2, 6, 7, 9\}\). Thus, the observation \(1\) will receive a rank of \(1\), the observation \(-2\) will receive a rank of \(2\), the observation \(6\) will receive a rank of \(3\), the observation \(-7\) will receive a rank of \(4\), and the observation \(9\) will receive a rank of \(5\). We can summarize this, noting: \[(x_1, x_2, x_3, x_4, x_5) = (-2, 1, -7, 9, 6) \quad\text{and}\quad (r_1, r_2, r_3, r_4, r_5) = (2, 1, 4, 5, 3).\]

With the ranks for each observation recorded, then we compute the signed-rank sum, denoted \(T\). Namely, taking \(\text{sgn}(x)\) to represent the sign of \(x\), such that \(\text{sgn}(x) = -1\) if \(x < 0\) and \(\text{sgn}(x) = 1\) if \(x > 0\), then \[T = \sum_{i=1}^N \text{sgn}(X_i)R_i.\] Under the null hypothesis, \(H_0: \mu = 0\), the signed-rank sum will be distributed according to the signed-rank distribution, parameterized by \(n\), the total sample size.

Definition 17.3 (Signed-Rank Sum) The signed-rank sum, denoted \(T\), is a test statistic used for nonparametric testing of a population location. Given data \((X_1, \dots, X_n)\), the data are first transformed into a set of ranks, \((R_1, \dots, R_n)\), determined by ranking the absolute values, \((|X_1|, \dots, |X_n|)\). Then, \[T = \sum_{i=1}^n \text{sgn}(X_i)R_i.\]

The \(p\)-value can then be determined on the basis of the signed-rank distribution. Namely, taking \(F_n\) to be the cumulative distribution function for the signed-rank distribution based on a sample of size \(n\), then \[p = P(T \leq -|t|) + P(T \geq |t|) = F_n(-|t|) + 1 - F_n(|t|-1) = 2F_n(-|t|).\] Note that the signed-rank distribution is a discrete distribution. As a result, it is not true to say \(P(T \geq t) = 1 - F_n(t)\). Instead, \(P(T > t) = 1 - F_n(t)\), and \(P(T \geq t) = P(T > t-1)\). Thus, when computing the \(p\)-value, we must take \(F_n(-|t|) + 1 - F_n(|t|-1)\), or the more simple \(2F_n(t)\). Computing probabilities for the signed rank distribution is feasible, using a computer, for small values of \(n\). For larger values of \(n\), however, this is not the case. Instead, if \(n\) is sufficiently large, then the signed-rank distribution is approximately normal, with a mean of \(0\) and a variance of \(\dfrac{n(n+1)(2n+1)}{6}\). Thus, for large \(n\), the \(p\)-value can be approximated as \[p \approx 2\times\Phi\left(-\frac{|t|}{\sqrt{n(n+1)(2n+1)/6}}\right).\]

Wilcoxon Signed-Rank Tests for Single Populations (Without Ties)

If data are selected from a symmetric population, and are such that the absolute values of all observed data points are unique, then the Wilcoxon Signed-Rank test can be used to test \(H_0: \mu = 0\) versus \(H_0: \mu \neq 0\), or the one-tailed alternatives, where \(\mu\) is the population location. To do so, the signed-rank sum test statistic is computed, \[T = \sum_{i=1}^n \text{sgn}(X_i)R_i,\] where \(R_i\) is the rank order (smallest to largest) of \(|X_i|\).

Under the null hypothesis, \(T\) follows a signed-rank distribution with size parameter \(n\). Taking \(T = t\) based on the observed sample, the \(p\)-value can be computed as:

If \(H_0: \mu = 0\) versus \(H_A: \mu \neq 0\), then \[p = F_n(-|t|) + 1 - F_n(|t|) \approx 2\Phi\left(-|t|/\sqrt{n(n+1)(2n+1)/6}\right).\]
If \(H_0: \mu \leq 0\) versus \(H_A: \mu > 0\), then \[p = 1 - F_n(t) \approx 1 - \Phi\left(t/\sqrt{n(n+1)(2n+1)/6}\right).\]
If \(H_0: \mu \geq 0\) versus \(H_A: \mu < 0\), then \[p = F_n(t) \approx \Phi\left(t/\sqrt{n(n+1)(2n+1)/6}\right).\]

17.3.1 Non-Centered Data, Ties, and Zeroes

The Wilcoxon signed-rank test can only test the null hypothesis \(H_0: \mu = 0\).³ This is useful whenever you expect that data are already centered, or if \(0\) is truly the location of interest. Frequently, non-zero values for the location will be of interest. If there is a desire to test \(H_0: \mu = \mu_0\), for \(\mu_0 \neq 0\), then before the test can be run, the data need to first be centered. To do so, the observations \((X_1,\dots,X_n)\) can be transformed into \((Z_1, \dots, Z_n) = (X_1 - \mu_0, \dots, X_n - \mu_0)\).

If the null hypothesis holds then \(X_i\) is symmetric with location \(\mu_0\). Thus, by shifting the entire distribution, it must be the case that \(Z_i\) is centered with location \(0\). As such, the Wilcoxon signed-rank test can then be applied to the computed \(Z_i\) testing \(H_0: \mu_Z = 0\) versus \(H_A: \mu_Z \neq 0\). This is equivalent to testing \(H_0: \mu = \mu_0\) versus \(H_A: \mu \neq \mu_0\).

A secondary concern is the accommodation of tied ranks. Specifically, if there are two observations with the same magnitude then the procedure cannot proceed as described, as there is not a unique rank for each observed point. This presents challenges, particularly when analyzing discrete data, where ties may be prevalent. While several tie-breaking procedures exist, two common approaches are the average rank procedure and the random rank procedure. In the average rank procedure, the rank of any observation is the average of all possible ranks it could take on, if ties were broken in any possible way. Suppose that two points, \(X_2\) and \(X_8\), have the same magnitude, and should be ranked as \(2\) or \(3\). Then, in the average rank procedure, both would receive a rank of \(r = \dfrac{2+3}{2} = 2.5\). The random rank procedure, on the other hand, breaks ties completely at random. For any tied points, the order within the ties is randomly selected, and test proceeds as though those are the true ranks.

If the average rank procedure is used, the distribution under the null hypothesis changes slightly. It is still referred to as a signed-rank distribution, but it is a signed-rank distribution with average ranks. This is not a problem, computationally, but it does render the null distribution contingent on the observed data. There are further theoretical concerns with the average rank procedure that, in certain settings, can lead to paradoxical conclusions.⁴ The random rank procedure, on the other hand, produces a test statistic with the same null distribution regardless of whether ties exist or not. The primary drawback in this case is that the specific test statistic, and as such \(p\)-value and conclusion, is dependent on both the observed data and the random rankings. To limit the impact that this has on final results, some researchers will run random ranking many times, and either collect the set of observed \(p\)-values or else average the set of \(p\)-values.

There is no single, universally preferred procedure for addressing ties. The most common approach in statistical software is⁵ the average rank procedure. However, the drawbacks with paradoxical conclusions can be severe and limiting. Random ranking avoids these issues, but at the cost of a more arbitrary test statistic, depending on additional randomness. Some researchers have proposed variations on these techniques, or new tie-breaking procedures all together. While any approach is likely to produce satisfactory results in the event of a few ties within the data, should there be a very large number of ties it is likely preferable to consider alternative test procedures that directly accommodate the data.

A final concern in the application of the Wilcoxon signed-rank test is the presence of zeroes. There are two common approaches for addressing zeros: the reduced sample procedure and the signed-rank zero procedure. In the reduced sample procedure, zeroes are simply dropped from the sample. Then, the test procedure runs exactly as before. This is an easy approach, and does not require the null distribution to change, but has similar drawbacks to the average rank procedure described above. In particular, the reduced sample procedure may produce paradoxical conclusions. Alternatively, the signed-rank zero procedure still ranks the zeros, and defines \(\text{sgn}(0) = 0\). This way, all data are still ranked and included within the test statistic. This has several theoretical benefits over the reduced sample procedure, including resolving the paradoxical conclusions that can arise, but has the disadvantage of changing the null distribution in the presence of zeroes.

Between the two techniques, there is no statistically optimal approach. In certain cases the reduced sample procedure will outperform the signed-rank zero procedure, and vice versa. Just as with ties, most software implementations will have a technique for accounting for zeroes in the data. These techniques will likely perform suitably well for data with minimal zeroes, however, in the event of data with an abundance of zeroes, it is likely worthwhile to consider alternative test procedure all together.

17.4 The Mann-Whitney \(U\)-Test

The signed-rank test allowed for nonparametric tests regarding a single population. A simple extension to this setting is the desire to run a hypothesis test comparing two independent populations. Suppose that \(X_{1},\dots,X_n\) are drawn from the first population and \(Y_1,\dots,Y_m\) are drawn from the second. General interest may be in whether the distribution of \(X\) and \(Y\) are equivalent to one another, or not.

Considering the equivalence of the distributions tends to be a stronger requirement than in the parametric case of comparing equality of means. However, it is worth revisiting why this may be reasonable. Specifically, in the nonparametric case, it is not guaranteed that any two distributions can be characterized by the same parameters. One distribution may have a mean, while the other does not. In this setting, it would not be reasonable to compare population means, since there are not two means to compare. If, when running the test comparing the populations, you are willing to make stronger assumptions regarding the distribution, it is possible to convert a test of equality of distributions into a direct test of location measures. For instance, suppose you are willing to assume that the population distributions of \(X\) and \(Y\) are equivalent except for, perhaps, their location. In this case testing whether the distributions are equivalent is the same as testing whether the locations are equal. In the standard \(t\)-test, this is essentially the assumption that is made. For nonparametric testing, it is useful to leave this more broad.

Denoting the cumulative distribution function of the first population, \(F_X\), and of the second population, \(F_Y\), the process of testing for equality of distributions can be expressed as testing the null hypothesis \(H_0: F_X = F_Y\) versus the alternative \(H_A: F_X \neq F_Y\). To do this nonparametrically, we can apply the Mann-Whitney \(U\)-test. The Mann-Whitney \(U\)-test is also referred to as the rank-sum test, the Mann-Whitney-Wilcoxon test, or the Wilcoxon rank-sum test. To ensure that the test remains clearly distinguished from the signed-rank test, it will be referred to as the \(U\)-test. The \(U\)-test, like the signed-rank test, relies on the ranks of the data. Unlike the signed-rank test, however, these are not the ranks of the absolute values, but rather the ranks of the data directly. Specifically, consider the combined sample of \(\{X_1,X_2,\dots,X_n,Y_1,Y_2,\dots,Y_m\}\). Within this sample, rank all the observations from smallest to largest, assigning the ranks as \(\{R_1,R_2,\dots,R_{n+m}\}\). If there are any ties, the average-rank procedure is used. That is to say that each observation gets the average rank that would be given if we resolved ties according to every possible reordering.

With the ranks for each data point computed, we can then compute the rank-sum for each sample. Namely, take \(R_X\) to be the sum of all the ranks from the observations corresponding to \(\{X_1,\dots,X_n\}\) and \(R_Y\) to be the sum of all the ranks from the observations corresponding to \(\{Y_1,\dots,Y_m\}\). That is, \[R_X = \sum_{i=1}^n R_i \quad\text{and}\quad R_Y = \sum_{i=1}^m R_{n+i}.\] Once the rank-sum is calculated for both samples, the \(U\)-statistics can be as well. Specifically take \[U_X = nm + \frac{n(n+1)}{2} - R_X \quad\text{and}\quad U_Y = nm + \frac{m(m+1)}{2} - R_Y.\]

The Mann-Whitney \(U\)-Statistic is then given by \(U = \min\{U_X, U_Y\}\). Under the null hypothesis, the \(U\)-statistic will follow the Wilcoxon rank-sum distribution. This distribution is characterized by both sample sizes, \(n\) and \(m\). Thus, taking \(F_{n,m}\) to be the cumulative distribution function for the Wilcoxon rank-sum distribution, then the \(p\)-value for the \(U\)-test will be \[p = 2\times\min\{F_{n,m}(u), 1 - F_{n,m}(u-1)\}.\] The form of this \(p\)-value arises since the distribution will not be symmetric around \(0\).⁶ When both \(n\) and \(m\) are large, then the null distribution is well-approximated by a normal distribution. Specifically, for large \(n\) and \(m\), \[U \stackrel{\dot H_0}{\sim} N\left(\frac{nm}{2}, \frac{nm(n+m+1)}{12}\right).\] Thus, for the \(p\)-value, \[p \approx 2\times\Phi\left(-\frac{|U - nm/2|}{\sqrt{nm(n+m+1)/12}}\right).\]

The Mann-Whitney \(U\)-Test (General)

Suppose that two independent samples are drawn from two separate populations, denoted \(X_1,\dots,X_n\) from the first and \(Y_1,\dots,Y_m\) from the second. The distribution function for \(X_i\) is \(F_X\) and the distribution function for \(Y_i\) is \(F_Y\). To test the null hypothesis that the distributions are equivalent, \(H_0: F_X = F_Y\), versus the alternative that they differ, \(H_A: F_X \neq F_Y\), the Mann-Whitney \(U\)-test can be run. First, all data from the two samples are combined and ranked. Then, the rank-sum statistics are computed for both samples, giving \(R_X\) and \(R_Y\). Using these statistics, the \(U\)-statistics for each sample are computed as \[U_X = nm + \frac{n(n+1)}{2} - R_X \quad\text{and}\quad U_Y = nm + \frac{m(m+1)}{2} - R_Y.\] Finally, \(U = \min\{U_X, U_Y\}\) is calculated as the \(U\) statistic.

Under the null hypothesis, \(U\) follows a rank-sum distribution with size parameters \(n\) and \(m\). Taking \(U = u\) based on the observed sample, the \(p\)-value can be computed as: \[p = 2\times\min\{F_{n,m}(u), 1 - F_{n,m}(u-1)\} \approx 2\times\Phi\left(-\frac{|U - nm/2|}{\sqrt{nm(n+m+1)/12}}\right).\]

Formality Under the Alternative

In order for the \(U\)-test to give consistent,⁷ accurate results, then a further assumption is required under the alternative hypothesis. Namely, it must be the case that \(P(X > Y) \neq P(Y > X)\) if the alternative hypothesis holds. Thus, if \(X\) and \(Y\) both have symmetric distributions, centered on the same value, then the \(U\)-test will not reliably conclude that the distributions are different, even when sample sizes grow indefinitely. This is a fairly technical consideration, but it is important when applying the \(U\)-test in practice. If you suspect that the two distributions are symmetric, and centered on the same value, the hypothesis test is no longer correctly regarded as testing \(F_X = F_Y\). To do so, alternative test procedures would be required.⁸

Direct Calculation of the \(U\) statistic

While the formal procedure for computing the \(U\) statistic gives rise to an easier theoretical analysis of the test statistic, there is a more practically applicable procedure for computing the statistic. The direct method for calculation circumvents the need to directly rank the data, or keep track of which observations came from which samples.

Instead, to compute \(U\), we first consider one of the samples, point-by-point. For each point in the sample, we count the number of observations from the second sample that are smaller than the presently considered point. If there are any ties, we count these as \(0.5\). We do this for every point, in turn, summing up the total results. The resulting sum is denoted \(U'\). Then, to find \(U\), we take \[U = \min\{U', nm-U'\}.\]

Suppose, for instance, that we observe \(\{4, 21, 32, 64\}\) as the first sample, and \(\{32, 31, 44, 45, 15,36\}\) as the second sample. To find \(U'\), we first consider the first sample. We start with \(4\), and note that there are \(0\) points in the second sample smaller than \(4\). We then consider \(21\), and note that there is exactly \(1\) point (\(15\)) in the second sample smaller than \(21\). We then take \(32\) and note that there are \(2\) points that are smaller than \(32\) (\(31\) and \(15\)) and \(1\) point equal to \(32\), giving a total of \(2.5\). Finally, for \(64\), we note that all \(6\) points are smaller than \(64\). Thus, \(U' = 0 + 1 + 2.5 + 6 = 9.5\). This gives \(U = \min\{9.5, 24-9.5\} = 9.5\).

17.4.1 Correction for Ties

When the data have a substantial number of observations that are tied in rank, just as in the signed-rank test, the null distribution is no longer represented by a Wilcoxon rank-sum distribution. Because of the altered null distribution, it is typically preferable - in the presence of many ties - to use the normal approximation. However, when the number of ties is large, the variance of the approximate normal distribution should be modified to better capture the impact of the tied observations. Specifically, if there are many ties, the variance will be overestimated relative to what it should be, leading to a less powerful test.

Suppose that there are \(k\) unique data points observed. For each of the these points, enumerated from \(1\) to \(k\), we can label the total number of observations of that value, between both samples, as \(t_i\). Then we must have that \(n + m = \sum_{i=1}^{k} t_i\). For any \(t_i \neq 1\), there are multiple points tied with the given value. Using this notation, we define \[\sigma_U^2 = \frac{nm(n+m+1)}{12} - \frac{nm\sum_{i=1}^k (t_i^3 - t_i)}{12(n+m)(n+m-1)}.\] Notice that, if \(t_i = 1\) for all \(i=1,\dots,k\), then \(t_i^3 - t_i = 0\), and as a result, \(\sigma_U^2\) equals the previously established variance term for \(U\). However, if \(t_i \neq 1\) for any \(i=1,\dots,k\), then \(t_i^3 - t_i > 0\), and as a result, \(\sigma_U^2\) will be strictly less than the previously established variance term.

The normal approximation for \(U\) can then be stated as \(U \stackrel{H_0}{\sim} N(\dfrac{nm}{2}, \sigma_U^2)\). To compute an approximate \(p\)-value, based on an observed \(U=u\), we can then take \[p \approx 2\Phi\left(-\frac{|u - nm/2|}{\sigma_U}\right).\] When there are not many ties, this should be approximately equal to the previously established approximation. However, when there are many ties, this can change substantially from the approximation without the correction for ties.

17.4.2 The Mann-Whitney \(U\)-Test for Location Shift Distributions

Because the null hypothesis is that the two distributions are equivalent, it is possible to reject the null hypothesis when the location of the two samples is equivalent. This is because two distributions that share a location are not necessarily equivalent. Consider, for instance, the \(N(0,1)\) and \(t_3\) distributions. Both have a location of \(0\), but otherwise behave quite differently. This is a desirable property of the \(U\)-test, that makes it broadly applicable. However, on occasion it may be desirable to test whether \(\mu_X = \mu_Y\), or more broadly, \(H_0: \mu_X - \mu_Y = \Delta_0\), for some fixed constant. The \(U\)-test can be extended to this setting, under stricter assumptions regarding the null and alternative distributions.

Namely, if it is assumed that \(F_X(x) = F_Y(x + \delta)\), for some value \(\delta\), then we are in effect saying that \(X\) and \(Y\) have equivalent distributions, except they are shifted away from one another by \(\delta\). In this context, testing whether \(F_X\) and \(F_Y\) are equivalent is the same as testing whether \(\delta = 0\), or alternatively, whether \(\mu_X = \mu_Y\). Thus, if we are willing to assume that it is impossible for the distributions of \(X\) and \(Y\) to differ, except in location, then the \(U\)-test becomes a test about the equality of those locations. Note that it is not possible to test the assumption that the distributions are otherwise equivalent, and as a result, using the \(U\)-test in this way is only valid when this assumption is justified via subject-matter arguments.

When this assumption is made, in order to test \(H_0: \mu_X - \mu_Y = \Delta_0\), the \(U\)-test can be directly applied to \(X_1,\dots,X_n\) for the first sample, and \(Y_1 + \Delta_0, \dots, Y_n + \Delta_0\) for the second. Note that if the null hypothesis holds, then \(\mu_X = \mu_Y + \Delta_0\), and so \(Y + \Delta_0\) will have location \(\mu_X\). Then testing \(F_X = F_{Y+\Delta_0}\) is equivalent to testing whether their locations are equal.

17.5 The Wilcoxon Signed-Rank Test (for Paired Data)

The Wilcoxon signed-rank test can be applied to test hypotheses regarding the location of a single population distribution. The Mann-Whitney \(U\)-test applies to test⁹ for the equality of locations of two independent populations. A third scenario that is useful to consider is testing of the location of two dependent distributions, specifically when observations from these distributions are paired.

Suppose that \(X_1,\dots,X_n\) are observed from the first population, with some location measure, \(\mu_X\). Then, \(Y_1, \dots, Y_n\) are observed from the second population, with some (possibly different) location measure, \(\mu_Y\). Further, suppose that the data are paired such that \(X_1\) and \(Y_1\) are naturally linked, \(X_2\) and \(Y_2\) are naturally linked, and so forth. Then, this sample can be converted into a single sample of paired differences by taking, \(D_i = X_i - Y_i\) giving \[(D_1, D_2, \dots, D_n) = (X_1 - Y_1, X_2 - Y_2, \dots, X_n - Y_n).\]

The location of the population \(D\), \(\mu_D\), will be equal to the difference in locations for the \(X\) and \(Y\) populations. That is, \(\mu_D = \mu_X - \mu_Y\). As a result, should we wish to test \(H_0: \mu_X - \mu_Y = \Delta_0\), this is equivalent to testing \(H_0: \mu_D = \Delta_0\). Further, supposing that the distributions for both \(X\) and \(Y\) are symmetric, then the distribution of \(D\) must also be symmetric. As a result, in this case, the Wilcoxon signed-rank test can be conducted on \((D_1, \dots, D_n)\). Specifically, the signed-rank sum can be computed based on the ranks of \((|D_1|, |D_2|, \dots, |D_n|)\), and the resulting statistic can be compared to the signed-rank distribution with size parameter \(n\).¹⁰

Wilcoxon Signed-Rank Tests for Paired Populations (Without Ties)

Suppose that data are selected from two symmetric populations, such that the observations are paired. First, compute the paired differences, taking \[(D_1, D_2, \dots, D_n) = (X_1 - Y_1, X_2 - Y_2, \dots, X_n - Y_n).\]

Further, suppose that the paired differences are such that the absolute values of all differences are unique. Then the Wilcoxon Signed-Rank test can be used to test \(H_0: \mu_D = 0\) versus \(H_A: \mu_D \neq 0\), or the one-tailed alternatives, where \(\mu_D = \mu_X - \mu_Y\) is the difference in population locations. To do so, the signed-rank sum test statistic is computed, \[T = \sum_{i=1}^n \text{sgn}(D_i)R_i,\] where \(R_i\) is the rank order (smallest to largest) of \(|D_i|\).

Under the null hypothesis, \(T\) follows a signed-rank distribution with size parameter \(n\). Taking \(T = t\) based on the observed sample, the \(p\)-value can be computed as:

If \(H_0: \mu_D = 0\) versus \(H_A: \mu_D \neq 0\), then \[p = F_n(-|t|) + 1 - F_n(|t|-1) \approx 2\Phi\left(-|t|/\sqrt{n(n+1)(2n+1)/6}\right).\]
If \(H_0: \mu_D \leq 0\) versus \(H_A: \mu_D > 0\), then \[p = 1 - F_n(t-1) \approx 1 - \Phi\left(t/\sqrt{n(n+1)(2n+1)/6}\right).\]
If \(H_0: \mu_D \geq 0\) versus \(H_A: \mu_D < 0\), then \[p = F_n(t) \approx \Phi\left(t/\sqrt{n(n+1)(2n+1)/6}\right).\]

17.5.1 Non-Centered Data, Ties, and Zeroes

Just as in the single sample case, the signed-rank test for paired data can be adjusted to accommodate ties, zeroes, and non-centered data. The same considerations can be applied to the single sample of paired differences, \((D_1,\dots,D_n)\). Shifting the observed differences by a constant allows for testing for non-zero differences. The average rank or random rank procedures can be used to accommodate ties in the differences. The reduced sample procedure or signed-rank zero procedure can both be used to accommodate zeroes in the sample. Just as was the case with the one sample procedure, any of these choices will likely produce satisfactory results when data have only a few ties or a few zeroes. When ties or zeroes are abundant, it is unlikely that the signed-rank test is the best choice for testing the location.

17.6 The Intuition of Rank-Based Tests

Both the Wilcoxon signed-rank test, and the Mann-Whitney \(U\)-test, often called the rank-sum test, are known as rank-based test procedures. The rationale is that these are tests on statistics derived from the ranks of the data. It is worth considering the intuition between how the test statistics connect to the hypotheses of interest.

Consider first the signed-rank sum. For this, the data (from a single sample) are ranked from smallest to largest, and then these ranks are signed. If we add up all the signed ranks then we can think about this equivalently as adding all the ranks for points observed above \(0\) and subtracting from that all ranks that were below \(0\). Under the null hypothesis, the location of the distribution is \(0\). In order for this to hold, it must be the case that it is about as likely for observations below zero to occur as it is for observations above zero. In connecting this to ranks, we may suggest that there should be roughly as many small values above zero and below zero, and the same with large values. As a result, in a distribution with \(\mu = 0\), we should expect that \(T\) will be approximately \(0\). If the values of this statistic are much larger or much smaller than zero, then it is likely that the true location is above or below zero as well.

For the Mann-Whitney \(U\)-statistic, a similar intuition takes place. Here, all the data are ranked together. If it is the case that the distributions are the same as one another, we should expect that the data will all be mixed in together in terms of ranks. If the distributions differ substantially from one another then there will likely be a biasing of one distribution being more likely to be large or small compared to the other. Reflected in ranks this means that, if the null hypothesis holds, the ranks should be roughly evenly distributed between the two populations and the rank-sums should be approximately equal. Otherwise, we would expect that one of the rank sums would be substantially larger than the other. The \(U\)-statistic is then computed simply by shifting the rank sums so that it sits between \(0\) and \(nm\). Values near the middle of this range are likely, and occur when the rank-sums are nearly equal.

In this way, whether we are considering a single population or multiple populations, the ranks of the data can be used to connect to the location of those populations.

17.7 Non-Parametric Tests Procedures in `R`

In R there is an implementation of both the signed-rank distribution for the signed-rank test and the rank-sum distribution for the \(U\)-test. Moreover, both procedures are available through a single function call on data.

The signed-rank distribution is accessible via the {r, p, d, q}signrank calls. These are analogous to the rnorm, pnorm, dnorm, and qnorm calls. An important caveat on the signed-rank distribution in R, however, is that it is computed not for the full signed-rank sum (as presented in this chapter), but instead, for the positive-rank sum. That is, in the test procedures built into R, the relevant test statistic is given by \[T^{+} = \sum_{i=1}^n \psi(X_i)R_i,\] where \(\psi(X_i) = 1\) if \(X_i > 0\) and \(\psi(X_i) = 0\) otherwise. This positive-rank sum statistic is connected to the signed-rank sum statistic by noting that \[T = 2T^{+} - \frac{n(n+1)}{2}.\]

This presents two important considerations. First, if we wish to compute \(P(T \leq t)\) then we can make the translation to using \[P(T \leq t) = P(2T^{+} - \frac{n(n+1)}{2} \leq t) = P(T^{+} \leq \frac{t}{2} + \frac{n(n+1)}{4}).\] Thus, to directly compute \(p\)-values in R, we need to modify the values we are passing to psignrank. Second, if we use the direct test function, the reported statistic will correspond to the positive ranks only. The resulting \(p\)-values and conclusions will be equivalent, but it is important to recognize that the output is not identical to what we would compute by hand.

Self-Assessment

Note: the following questions are still experimental. Please contact me if you have any issues with these components. This can be if there are incorrect answers, or if there are any technical concerns. Each question currently has an ID with it, randomized for each version. If you have issues, reporting the specific ID will allow for easier checking!

For each question, you can check your answer using the checkmark button. You can cycle through variants of the question by pressing the arrow icon.

For instance, that it follows a normal distribution, a binomial distribution, or similar.↩︎
If it exists.↩︎
Or, the one-tailed equivalents.↩︎
Specifically, it is possible that the average rank procedure rejects the null hypothesis in the one tail test, concluding (for instance) that \(\mu > 0\). However, using the same data, if you add some amount to the tied data points, making them more positive, then the null hypothesis may not be rejected any longer. These two conclusions are incompatible with one another, and arise as a natural consequence of the average rank procedure.↩︎
Very likely.↩︎
In the case of \(Z\)-tests, \(t\)-tests, and the presented version of the signed-rank test, the null distribution has always been symmetric around \(0\). In this case we must have that \(F(-|t|) = 1-F(|t|)\) and as such, we can express the \(p\)-value as the sum of the two tails. For non-symmetric null distributions, we still wish to understand the probability of observing something at least as extreme as what was actually observed. Here, however, that is not as simple as saying less than \(-|t|\) or above \(|t|\), since the distribution is not centered on \(0\). In fact, in the case of the \(U\)-statistic, the null distribution is strictly positive. This form, \(2\times\min\{F(u), 1-F(u-1)\}\) is broadly applicable across many hypothesis tests, however, it tends to be less informative compared to the sum of the tail probabilities.↩︎
Note that here, consistency refers to a specific statistical idea. Namely, consistency represents the idea that, as sample sizes increase larger and larger, the resulting statistical procedure will approach the correct results.↩︎
For instance, the Kolmogorov-Smirnov test is a nonparametric test that considers the equality (or lack thereof) of continuous distributions, without making the same assumptions under the null hypothesis.↩︎
Under somewhat rigorous assumptions↩︎
This procedure is analogous to the \(t\)-test for paired data. In the \(t\)-test for paired data we first consider the single sample that arises from differencing the observations. Then, a single sample \(t\)-test is conducted on the sample of paired differences. To do so nonparametrically, we first consider the single sample of paired differences, and then conduct a Wilcoxon signed-rank test on this sample of paired differences.↩︎