a. Including Tests of Equal Variance, Normality Testing and Sample Size calculation, performing tests and interpreting results.

Including Tests of Equal Variance, Normality Testing and Sample Size calculation, performing tests and interpreting results.. Introduction This article explains how to: - Check normality of data - Check equality of variances across groups - Calculate sample size for hypothesis tests and confidence intervals - Perform tests and interpret results correctly All content is focused on the level of rigor needed to correctly select, run, and interpret these statistical tools in improvement projects. --- Why Normality, Equal Variance, and Sample Size Matter Before applying statistical tests, three questions must be answered: - Is the data approximately normal? This affects whether to use parametric tests (t-test, ANOVA, regression) or nonparametric alternatives. - Are variances approximately equal across groups? Many parametric tests assume equal variance; violating this can distort p-values and conclusions. - Is the sample size adequate? Too small: high risk of not detecting real effects. Too large: detects trivial differences and wastes resources. The goal is to choose appropriate tests, ensure assumptions are reasonable, and interpret p-values and confidence intervals in context. --- Normality Testing Understanding Normality in Practice Normality refers to whether data follow a bell-shaped, symmetric distribution. Many parametric tests assume: - The population of residuals (or errors) is normal - Or, for simple tests, each group’s data is approximately normal In real applications: - Moderate deviations from normality are often acceptable, especially with larger samples (Central Limit Theorem). - Severe skewness, heavy tails, or outliers may invalidate parametric test results. Normality matters most when: - Sample sizes are small (commonly n < 30 per group) - Data are highly skewed or zero-inflated - Using interval estimates or hypothesis tests that rely on t-distributions or F-distributions Graphical Assessment of Normality Graphical checks are essential before formal tests. Key plots: - Histogram - Look for approximate bell shape, symmetry, and single peak. - Check for strong skew, multiple peaks, or extreme outliers. - Boxplot - Visualize median, spread, skewness, and outliers. - Long whiskers on one side and many outliers indicate skewness. - Normal probability plot (Q-Q plot) - If data are normal, points align roughly along a straight line. - Systematic curvature indicates non-normality: - S-shaped: heavy tails - Convex/concave: skewness Interpretation principle: - Use plots first to understand the nature and severity of departures from normality. - Combine plots with formal tests, not replace them. Formal Normality Tests Common tests include: - Anderson–Darling test - Sensitive to deviations in both center and tails. - Often used in statistical software for normality testing. - Shapiro–Wilk test - Powerful for small to moderate sample sizes. - Frequently used in research applications. - Kolmogorov–Smirnov test (with Lilliefors correction) - Compares empirical distribution to theoretical normal. - Less powerful than Anderson–Darling and Shapiro–Wilk in many practical cases. Each test uses: - Null hypothesis (H₀): Data follow a normal distribution. - Alternative hypothesis (H₁): Data do not follow a normal distribution. Decision rule: - If p-value < α (often 0.05): Reject H₀, conclude data are not normal. - If p-value ≥ α: Do not reject H₀; no evidence against normality. Important cautions: - With large samples, even tiny, unimportant deviations can yield significant p-values. - With very small samples, tests may lack power; reliance on plots and process knowledge becomes more important. Practical Interpretation and Response When normality tests indicate non-normality: - Review plots to understand the pattern: - Skewed right: long right tail (e.g., time to complete, defect counts). - Skewed left: long left tail. - Bimodal: possibly mixed populations or stratification issues. Common responses: - Use a transformation: - Log, square root, or Box–Cox transformations can stabilize variance and improve normality of residuals. - Interpret final results in original units when reporting. - Use nonparametric tests: - For example, Mann–Whitney instead of 2-sample t-test, Kruskal–Wallis instead of one-way ANOVA. - Especially useful when sample sizes are small and deviation from normality is severe. - Increase sample size: - Reduces sensitivity to modest non-normality due to the Central Limit Theorem, especially for mean-based tests. - Model at the right level: - For counts, consider Poisson/binomial models rather than forcing normality. The key is to match the test to the data and clearly state any limitations in conclusions. --- Tests of Equal Variance Why Equality of Variance Matters Many parametric tests assume homogeneity of variance (equal variances across groups), including: - 2-sample t-test (pooled-variance version) - One-way ANOVA - Classical linear regression with constant error variance If variances are not equal: - Type I error (false positive) and power can be distorted. - Tests may favor one group, especially when sample sizes are unequal. Before applying these tests, check: - Are group spreads similar? - Are outliers or extreme values inflating variance? Graphical Assessment of Variance Use simple visuals first: - Side-by-side boxplots - Compare IQR (box height), whisker length, and spread. - Large differences in box height suggest unequal variances. - Residuals vs fitted values plots (for models) - Look for patterns in spread as the mean changes. - Funnel shapes indicate non-constant variance. Interpretation: - Approximate similarity is often sufficient. - Major differences or clear patterns suggest using robust methods or adjustments. Formal Tests for Equal Variance Common tests include: - F-test (two groups) - H₀: σ₁² = σ₂², H₁: σ₁² ≠ σ₂². - Highly sensitive to non-normality; best only when normality is reasonable. - Levene’s test - Based on absolute deviations from group medians or means. - Less sensitive to non-normality than the F-test. - Bartlett’s test - Sensitive to non-normality; best with normal distributions. - Used for more than two groups. - Brown–Forsythe test - A robust variant of Levene’s test using medians. - Handles skewed data better. General decision rule: - Null hypothesis (H₀): All group variances are equal. - Alternative (H₁): At least one group variance differs. - If p-value < α: Evidence of unequal variances. - If p-value ≥ α: No evidence against equal variances. Use both tests and graphics to judge whether the equal variance assumption is adequate. What to Do When Variances Are Unequal When equal variance is questionable: - Use a test that does not assume equal variance: - For 2 groups: Welch’s t-test instead of pooled t-test. - For multiple groups: Welch’s ANOVA. - Transform the data: - Some transformations (e.g., log) can both stabilize variance and improve normality. - Use nonparametric methods: - Mann–Whitney, Kruskal–Wallis, or rank-based regression when appropriate. - Use robust standard errors (for regression): - Adjusts inference without changing the model form. When reporting: - State which equal-variance test was used. - Indicate whether pooled or nonpooled (Welch) methods were applied. - Clarify that conclusions are based on methods robust to unequal variance when needed. --- Sample Size Calculation: Foundations Key Concepts: Effect Size, Alpha, Power, and Variability Sample size determination depends on four main components: - Effect size (Δ) The smallest difference that is practically important to detect. - Alpha (α) Probability of a Type I error (rejecting a true null). Commonly α = 0.05. - Power (1 – β) Probability of detecting a true effect of size Δ. Common targets: - 0.80 (80%) - 0.90 (90%) when missing a difference is costly. - Variability (σ or s) Standard deviation from historical data, a pilot study, or prior experience. Trade-offs: - Smaller α or larger power → larger required sample. - Smaller effect size to detect → larger required sample. - Higher variability → larger required sample. General Steps in Sample Size Planning A systematic approach: - Clarify the objective - Estimate a mean with a given margin of error. - Compare two means. - Estimate or compare proportions. - Detect an improvement of at least a specified amount. - Specify statistical parameters - Choose α (often 0.05). - Choose desired power (e.g., 0.80 or 0.90). - Define the minimum practically important effect size Δ. - Estimate standard deviation or baseline proportion. - Choose the appropriate formula or software tool - Match the design: - 1-sample, 2-sample, paired, ANOVA, regression, etc. - Adjust for practical constraints - Integer sample sizes, balance across groups, resource limits. - Plan for attrition or unusable data - Increase required sample size to account for dropouts, measurement issues, or defects. --- Sample Size for Means One-Sample Mean (Confidence Interval or Test) Objective examples: - Estimate a mean within a margin of error E at confidence level (1 – α). - Detect a difference from a target value μ₀ of at least Δ. Key elements: - Known or estimated standard deviation s. - Z or t quantile depending on sample size and assumptions. Interpretation: - Larger confidence levels or smaller margins of error require larger n. - For hypothesis tests, smaller detectable Δ requires larger n. Two-Sample Means: Independent Samples Common scenarios: - Comparing process A vs B. - Comparing before vs after when data are not naturally paired. Inputs: - Effect size (Δ): minimum meaningful difference between μ₁ and μ₂. - Standard deviation in each group, often assumed equal initially. - Allocation ratio (often 1:1, but may differ). Considerations: - If variances are likely unequal, use methods based on Welch’s t-test for planning. - Balanced designs (equal n per group) maximize power for a given total sample. Power and sample size are highly sensitive to: - The accuracy of the standard deviation estimate. - The realism of the effect size Δ. --- Sample Size for Proportions One-Sample Proportion Example objectives: - Estimate defect rate within ±E with confidence (1 – α). - Detect a change from baseline p₀ to p₁ = p₀ + Δ. Inputs: - Baseline proportion (p₀) or best estimate. - Desired margin of error or minimum detectable change. Observations: - Proportions near 0.5 require the largest sample for a given margin of error. - Rare events (very small p) may need large samples or alternative designs. Two-Sample Proportions Typical use: - Compare defect rates of two processes or time periods. - Compare conversion rates across two treatments. Inputs: - Baseline proportion p₁. - Target proportion p₂ = p₁ + Δ or an absolute difference of interest. - Alpha and power. Planning decisions: - Whether to use pooled or unpooled methods in calculations. - Whether group sizes will be equal or different due to practical constraints. --- Linking Normality, Variance, and Sample Size to Test Selection Choosing the Appropriate Test After planning sample size and collecting data: - Check normality - If acceptable and n is moderate to large: - Mean-based parametric tests (t-test, ANOVA, regression) are typically valid. - If non-normal with small n or severe skew: - Consider transformations or nonparametric tests. - Check equality of variances - If equal variances are plausible: - Use pooled-variance methods (2-sample t-test with pooled variance, classical ANOVA). - If unequal: - Use Welch’s methods or robust approaches. - Match test to measurement level - Continuous data: tests on means (t-tests, ANOVA, regression). - Categorical data: tests on proportions or counts (z-tests, chi-square). Interpreting p-Values and Confidence Intervals For any test: - p-value - Probability, under H₀, of observing a result at least as extreme as the one obtained. - Compare to α: - p < α: Evidence against H₀. - p ≥ α: Insufficient evidence to reject H₀. - Confidence interval - Range of plausible values for a parameter (mean difference, proportion difference, etc.). - If the interval: - Includes 0 (for mean difference) or includes the null value: consistent with no effect. - Excludes 0 or null value: consistent with a significant effect. Relationship: - A two-sided test at α corresponds to a (1 – α) confidence interval: - If the interval does not contain the null value, p < α. Decision-making: - Use both p-values and confidence intervals: - p-value: whether the evidence is statistically significant. - Confidence interval: how large the effect might be, in practical units. --- Performing and Interpreting Normality and Equal Variance Tests Stepwise Approach in Analysis A practical workflow: - Step 1: Plot the data - Histograms, boxplots, normal probability plots. - Residual vs fitted plots for model-based analyses. - Step 2: Conduct normality tests - Use Anderson–Darling or Shapiro–Wilk. - Interpret p-values with sample size in mind. - Reconcile with visual evidence and process knowledge. - Step 3: Conduct equal-variance tests - Levene’s or Brown–Forsythe for multiple groups. - F-test for two groups only when normality is reasonable. - Step 4: Choose the inference method - If normality and equal variance are acceptable: - Use standard parametric methods. - If issues arise: - Consider transformations, nonparametric tests, or robust methods. - Step 5: Interpret results comprehensively - Combine: - Assumption diagnostics. - p-values. - Confidence intervals. - Practical significance. Common Pitfalls and How to Avoid Them Typical issues: - Over-reliance on p-values from diagnostic tests - Large samples produce significant p-values for trivial deviations. - Avoid automatic rejection of normality based solely on a small p-value. - Ignoring variance inequality in unbalanced designs - When group sizes differ, unequal variances can seriously bias results. - Always check both assumptions when sample sizes are very different. - Using parametric tests on extreme outliers without investigation - Outliers may indicate data issues or a different process regime. - Investigate causes, consider robust or transformed methods. - Planning sample size without realistic variance or effect size estimates - Use pilot data, historical data, or subject-matter input. - Conduct sensitivity analyses (e.g., how required n changes with s or Δ). - Interpreting non-significant results as evidence of no effect - Non-significant may mean: - No effect, or - Insufficient power or poor sample size. - Examine the width of the confidence interval and the actual power. --- Summary Normality tests, tests for equal variance, and sample size calculations work together to ensure valid, interpretable statistical conclusions. - Normality testing (via plots and tests like Anderson–Darling or Shapiro–Wilk) informs whether mean-based parametric methods are appropriate or whether transformations or nonparametric tests are needed. - Tests of equal variance (such as Levene’s or Brown–Forsythe) guide the choice between pooled and nonpooled methods for comparing groups and help detect when robust approaches are necessary. - Sample size calculation balances effect size, alpha, power, and variability to ensure data are sufficient to detect practically important differences without wasting effort. - Performing tests and interpreting results requires: - Checking assumptions with both diagnostics and graphics, - Choosing appropriate tests based on data characteristics, - Interpreting p-values and confidence intervals in the context of practical significance and study design. When these elements are applied systematically, statistical conclusions become reliable, transparent, and aligned with the real objectives of process analysis and improvement.