3.5.1 Mann-Whitney

Mann-Whitney Introduction The Mann-Whitney test (also called Mann-Whitney U or Wilcoxon rank-sum test) is a nonparametric statistical test used to compare two independent groups when: - The outcome is at least ordinal (rankable). - Normality or equal-variance assumptions for the two-sample t-test are questionable. - Sample sizes may be small, skewed, or contain outliers. It tests whether the distribution of one group tends to be shifted higher or lower than the other. It is widely used when data are not well modeled by a normal distribution but a comparison of central tendency (often “medians”) between two groups is needed. --- Purpose and When to Use Mann-Whitney Appropriate Use Cases Use the Mann-Whitney test when comparing two independent groups on a single continuous or ordinal outcome, and at least one of these conditions holds: - Data are skewed or have outliers. - Data are ordinal (e.g., Likert scales) rather than truly continuous. - Normality assumptions of the two-sample t-test are clearly violated. - Group sample sizes are small and distributional form is uncertain. Typical examples: - Comparing customer satisfaction ratings (1–5 scale) between two process designs. - Comparing cycle times between two machines when times are strongly skewed. - Comparing defect discovery counts per batch for two different inspection methods. When Not to Use Mann-Whitney Avoid Mann-Whitney when: - Groups are not independent (e.g., before/after measurements on the same units). In that case, a paired nonparametric test (Wilcoxon signed-rank) is more appropriate. - There are more than two independent groups (then consider a nonparametric ANOVA approach such as Kruskal-Wallis). - Data are nominal (categories without order); a chi-square type test is more suitable. - Exact equality of distributions, including shape and spread, is not a sensible question (for example, when data are not rankable). --- Hypotheses and Interpretation Test Hypotheses Conceptually, the Mann-Whitney test evaluates whether one group tends to yield larger values than the other. Let: - Group 1: population A - Group 2: population B The hypotheses are often expressed as: - Null hypothesis (H₀): The distributions of A and B are identical (no systematic shift). - Alternative hypothesis (H₁): The distributions differ such that one group tends to have larger (or smaller) values. For a two-sided test: - H₀: Distributions of A and B are the same. - H₁: Distributions of A and B are different. For a one-sided test (example, A greater than B): - H₀: A is not greater than B (distributions equal or A tends to be smaller). - H₁: A tends to have larger values than B. If distributions are assumed to have the same shape and spread, the test can be interpreted as a test of equality of medians. Without that assumption, it is safer to say it tests for a difference in distributions or in central tendency. Practical Interpretation Key interpretation points: - A small p-value (below chosen alpha, often 0.05) suggests a statistically significant difference between groups. - A large p-value suggests insufficient evidence to claim a difference. - The test result alone does not quantify the size of the effect; that requires an effect size or confidence interval. Useful interpretation statements: - “Data provide statistical evidence that distribution of outcome in group A is shifted higher than in group B.” - “No statistical evidence of a difference between the distributions of group A and group B.” --- Data Requirements and Assumptions Data Requirements Mann-Whitney requires: - Two groups: Exactly two, independent of each other. - Independent observations: No subject appears in both groups. - Ordinal or higher-scale outcome: Data can be ranked (at least). - Meaningful ordering: Higher values correspond to more of the measured attribute. Sample sizes: - Works with small or large samples. - Exact p-values are typically used for small samples; normal approximation for larger samples. Statistical Assumptions The main assumptions are: - Independence within and between groups - Each observation is independent of others. - Group membership is unrelated to measurement error structure. - Ordinal measurement - The ranking of observations is valid and meaningful. - Shape assumption (for median comparison) - If the distributions of the two groups have similar shape and spread, the test can be interpreted as comparing medians. - If shapes differ, the result indicates general distributional difference, not strictly median difference. --- Ranking Procedure and Test Statistic Ranking the Data The test is based on ranks, not raw values. Steps to rank: - Combine all observations from both groups into a single list. - Sort from smallest to largest. - Assign ranks starting at 1 for the smallest value: - If there are ties, assign each tied value the average of the ranks they would have occupied. - Sum the ranks separately for each group: - R₁ = sum of ranks for group 1. - R₂ = sum of ranks for group 2. These rank sums reflect whether one group tends to have larger or smaller values than the other. U Statistic Computation Let: - n₁ = sample size of group 1. - n₂ = sample size of group 2. - R₁ = sum of ranks for group 1. - R₂ = sum of ranks for group 2. Compute: - U₁ = n₁n₂ + [n₁(n₁ + 1)] / 2 − R₁ - U₂ = n₁n₂ + [n₂(n₂ + 1)] / 2 − R₂ These are two equivalent forms; they satisfy: - U₁ + U₂ = n₁n₂ The Mann-Whitney U statistic used for testing is: - U = min(U₁, U₂) The smaller U reflects greater separation between rank distributions. --- Sampling Distribution and p-Value Exact and Approximate Methods The distribution of U under the null hypothesis is known. - Exact distribution - Available for small sample sizes. - Tables or statistical software can give exact p-values. - Normal approximation - For larger samples, U is approximated by a normal distribution with: - Mean: μᵤ = n₁n₂ / 2 - Variance (no ties): σᵤ² = n₁n₂(n₁ + n₂ + 1) / 12 - A continuity correction is often used in calculating Z. The test statistic under normal approximation: - Z = (U − μᵤ) / σᵤ (with optional continuity correction) The p-value is derived from Z using the standard normal distribution. Tie Corrections When data contain tied values, the variance must be adjusted. General approach: - Compute a tie correction factor based on groups of tied ranks. - Adjust σᵤ² downward to reflect reduced information due to ties. Software typically applies tie corrections automatically, but for understanding: - More ties → slightly less power (larger p-values for the same effect). --- Effect Size and Practical Significance Common Effect Size Measures To complement the p-value, effect size helps quantify the magnitude of difference. Common metrics: - Rank-biserial correlation (rᵣᵦ) - Based on difference in rank sums or proportion of “wins” by one group over the other. - Interpretable as strength and direction of association between group membership and outcome rank. - Probability of superiority (also A measure) - P(X > Y) + 0.5·P(X = Y) - Interpreted as the probability that a randomly chosen observation from one group exceeds a randomly chosen observation from the other. - Standardized Z-based effect size (r) - r = |Z| / √N, where N = total number of observations. - Interpreted similarly to a correlation coefficient (magnitude only). Confidence Intervals Some software provides confidence intervals for: - Difference in medians (under similar-shape assumption). - Probability of superiority or related effect size measures. Use confidence intervals to assess: - Range of plausible differences between the groups. - Whether differences are practically meaningful, not just statistically detectable. --- One-Sided vs Two-Sided Mann-Whitney Tests Two-Sided Testing Use a two-sided test when: - The direction of difference is not specified in advance. - Any difference (higher or lower) is of interest. Interpretation: - H₀: Distributions are equal. - H₁: Distributions differ (in any direction). One-Sided Testing Use a one-sided test only when: - A clear, justified directional expectation is specified before seeing the data. - You care only about evidence in that one direction. Example: - H₀: Group A is not greater than Group B. - H₁: Group A tends to have higher values than Group B. Selecting one-sided versus two-sided: - Must be decided before analyzing data. - Affects p-value (one-sided p is half of two-sided p if data favor the tested direction). --- Comparison with the Two-Sample t-Test Relationship to the t-Test Mann-Whitney is often seen as a nonparametric alternative to the two-sample t-test. Key differences: - t-test - Compares means, assuming approximate normality and often equal variances. - Uses raw values directly. - Mann-Whitney - Compares distributions (often interpreted as medians under equal-shape assumption). - Uses ranked data, reducing sensitivity to extreme values and non-normality. Choosing Between Them Preference for Mann-Whitney when: - Data are skewed or contain pronounced outliers. - Data are ordinal (ratings). - Distributional assumptions for t-test clearly fail and transformation is not desired or not effective. Preference for t-test when: - Data are approximately normal with similar variances. - Interest is specifically in comparing means. - Power is a priority and assumptions are reasonably met. --- Typical Output and Interpretation Steps Typical Software Output Common outputs from statistical software for Mann-Whitney include: - Group sizes (n₁, n₂). - Sum of ranks and average rank for each group. - U statistic (often reported as U or Mann-Whitney U). - Z value (for normal approximation). - p-value (one-sided and two-sided). - Optionally, effect size and confidence intervals. Interpretation Flow A systematic approach: - Check assumptions - Two groups, independent observations. - At least ordinal data, reasonable rankability. - Review rank information - Which group has higher average rank? - Indicates which group tends to have larger values. - Assess p-value - Compare to significance level (e.g., 0.05). - Decide whether to reject or fail to reject H₀. - Examine effect size - Evaluate magnitude of difference, not just existence. - Consider whether effect is practically important. - State clear conclusion - Specify group direction (“Group A tends to have higher [metric] than Group B”). - Note statistical significance and, where relevant, practical significance. --- Common Pitfalls and Misinterpretations Misinterpreting as a Mean Comparison Mann-Whitney does not directly compare means. It compares: - Distributions, or - Central tendency via ranks (often interpreted as medians under equal-shape assumption). Avoid claiming: - “The mean of group A is different from the mean of group B” as a direct Mann-Whitney conclusion, unless you have additional justification. Ignoring Distribution Shape The interpretation as a median comparison assumes similar distribution shapes. When shapes differ: - A significant result may reflect differences in spread, skewness, or other distributional features—not only location. - Graphical checks (e.g., boxplots) can help understand the nature of the difference. Misusing for Paired Data Mann-Whitney is for independent groups. Using it with paired or matched data: - Violates independence. - Can yield misleading p-values. In paired contexts, a different nonparametric test is appropriate. Overreliance on p-Values Relying only on p-values: - Ignores effect magnitude. - Can mark trivial differences as significant with large samples. Balance: - p-values (statistical evidence). - Effect sizes and confidence intervals (practical meaning). --- Summary The Mann-Whitney test is a nonparametric method for comparing two independent groups when the outcome is at least ordinal and normality assumptions for parametric tests are doubtful. It operates by ranking all observations, computing a U statistic from rank sums, and evaluating that statistic against its sampling distribution to obtain a p-value. Key points: - Compares overall distributions and, under similar-shape assumptions, acts as a test of median difference. - Requires independent groups and rankable data. - Handles skewed data and outliers more robustly than the two-sample t-test. - Provides direction of difference via relative ranks and magnitude of difference via effect sizes. Mastery of Mann-Whitney involves understanding when to use it, how its rank-based U statistic is constructed, how to interpret its p-values and effect sizes, and how to avoid misinterpretations about means, medians, and distributional differences.