3.2.3 Central Limit Theorem

Central Limit Theorem Introduction The Central Limit Theorem (CLT) is one of the most important results in statistics. It explains why sample means tend to follow a normal distribution, even when the original data are not normal. This makes the CLT the foundation for: - Confidence intervals for means - Hypothesis tests for means - Many capability and performance analyses that rely on normality Understanding the CLT allows you to use normal-based methods in real-world processes with non-normal data, as long as certain conditions are met. --- Core Idea of the Central Limit Theorem Informal Statement The Central Limit Theorem says: - When you take many random samples from any population (with finite mean and variance), - And calculate the mean of each sample, - The distribution of those sample means tends to be normal as sample size becomes large. This remains true even if the original population is skewed or otherwise non-normal. Formal Statement Let a population have: - Mean: μ - Standard deviation: σ - Finite variance: σ² < ∞ Draw all possible random samples of size n from this population and compute the sample mean for each. As n increases: - The distribution of sample means becomes approximately normal - The mean of the sample means = μ - The standard deviation of the sample means (standard error) = σ / √n If n is sufficiently large, the distribution of sample means is approximately: - X̄ ~ Normal(μ, σ/√n) --- Why the Central Limit Theorem Matters Bridge from Any Distribution to Normal Real process data often are not normal. The CLT gives a way to legitimately apply normal-based tools: - The process data may be non-normal - The distribution of the sample mean can still be approximately normal - This lets you use z or t distributions for inference about the mean Foundation for Statistical Inference Most statistical procedures for means depend on the CLT: - Confidence intervals for μ - Tests such as: - One-sample t test - Two-sample t test - Paired t test - Many control charts based on averages (for example, X̄ charts) The CLT justifies using these tools as long as sample size and assumptions are appropriate. --- Sample Mean and Standard Error Sampling Distribution of the Mean The sampling distribution of the mean is the distribution you would get if: - You repeatedly take samples of size n from the same population - Compute the sample mean for each - Plot all those means Key results: - Mean of X̄ = μ - Variance of X̄ = σ² / n - Standard deviation of X̄ = σ / √n (called the standard error of the mean) Standard Error and Sample Size The standard error quantifies how much sample means vary from sample to sample: - Larger n → smaller standard error - Smaller n → larger standard error Formula: - Standard error (when σ is known): SE = σ / √n - Standard error (in practice, σ unknown): SE ≈ s / √n, where s is sample standard deviation Implications: - To reduce the variability of the sample mean by half, you must quadruple the sample size - Sample size reduction of variability has diminishing returns --- Conditions and Assumptions Independence The CLT assumes samples are independent: - Each observation should not influence another - Common violations: - Time-ordered data with autocorrelation - Data taken from the same item repeatedly when treating them as separate items To support the CLT: - Use random sampling - Avoid strong dependence between observations Identically Distributed The CLT assumes that all observations are drawn from the same distribution: - Same mean and variance across observations - No systematic shifts during sampling This means: - Do not mix data from fundamentally different conditions and treat them as one homogeneous sample - Use stratification or segmentation when underlying processes differ Finite Variance The original population must have finite variance: - Real process data almost always meet this requirement - Extremely heavy-tailed theoretical distributions (rare in practice) can violate it --- Sample Size Guidelines How Large is “Large Enough”? The exact sample size needed for the CLT approximation to be good depends on the shape of the original distribution: - Approximately symmetric distribution: - n around 30 is usually sufficient - Moderately skewed distribution: - May need larger n (for example, 40–60) - Strongly skewed or with extreme outliers: - May require much larger n for the sample mean to be nearly normal These are guidelines, not rigid rules. Actual adequacy depends on: - Degree of skewness - Presence of outliers - Desired accuracy of the normal approximation Practical Considerations When deciding whether to rely on the CLT: - Examine data for strong skewness and outliers - Use larger sample sizes for more skewed processes - Consider data transformations if normal-based methods are important --- Central Limit Theorem and Normality of the Mean Population vs. Sample Mean Distributions It is crucial to distinguish: - Distribution of individual data (X) - Distribution of sample means (X̄) Key points: - CLT refers to the distribution of X̄, not X - X may be non-normal, but X̄ will tend to normal as n grows - For very small n, the distribution of X̄ often still reflects the shape of the original data Standardization of the Sample Mean When σ is known and n is large, the CLT implies: - Z = (X̄ − μ) / (σ / √n) is approximately standard normal: Z ~ Normal(0, 1) This standardization is the basis for: - z tests for means (large samples or known σ) - Normal-based confidence intervals for μ When σ is unknown and n is small, the t distribution is used instead, but the underlying justification still comes from the CLT. --- Central Limit Theorem in Confidence Intervals Confidence Interval for the Mean For large n, using the CLT, a two-sided confidence interval for μ is: - X̄ ± z* × (s / √n) Where: - X̄ = sample mean - s = sample standard deviation - n = sample size - z* = critical value from standard normal for the desired confidence - For 95% confidence: z* ≈ 1.96 - For 99% confidence: z* ≈ 2.576 Why the CLT matters here: - Even if the original data are non-normal, the sampling distribution of X̄ is approximately normal for large n - This justifies using z* and a normal-based interval for μ Width of the Interval The CLT helps understand how interval width depends on n: - Width ∝ 1 / √n - Larger n → narrower confidence interval (more precise estimate of μ) - Doubling n does not halve the width; it reduces width by factor 1/√2 --- Central Limit Theorem in Hypothesis Testing Tests for a Population Mean For large samples or known σ, the test statistic for a mean is: - Z = (X̄ − μ₀) / (σ / √n) when σ is known - Z ≈ (X̄ − μ₀) / (s / √n) for large n and unknown σ Where μ₀ is the hypothesized population mean. The CLT ensures: - Under the null hypothesis, the distribution of Z is approximately standard normal - p-values and critical values from the normal distribution are valid approximations t Tests and the CLT For small samples with unknown σ: - The t distribution is used: - t = (X̄ − μ₀) / (s / √n) The t distribution itself is derived assuming: - Underlying observations are (ideally) normal, or - The CLT provides approximate normality of X̄ as n increases In practice, for moderately large n (often n ≥ 30): - The t and normal distributions are very close - The CLT still underlies the validity of using t-based inference --- Aggregation, Subgrouping, and the CLT Rationale for Using Means in Analysis Many analytical tools use subgroup averages rather than individual data. The CLT explains why: - Averages are more stable than individual data - Distribution of subgroup means is more nearly normal than the underlying data - Normal-based methods for means can be valid even with non-normal individual data Subgroup Size and Approximate Normality When forming subgroups of size n and analyzing their means: - Larger subgroup size → subgroup means closer to normal - Too large subgroup size → may hide process shifts within subgroups - Too small subgroup size → subgroup means may still reflect non-normality Balance is needed: - Choose subgroup sizes that both: - Respect process dynamics (time order, shift detection) - Provide reasonable basis for near-normal means --- Limitations and Misconceptions CLT Does Not Guarantee Normal Data Common misconceptions: - Misconception: “CLT says any data set becomes normal for large samples.” - Reality: - CLT refers to the distribution of sample means, not raw data - Raw data can remain highly skewed even for very large n CLT Does Not Fix Bad Sampling The CLT does not correct: - Biased sampling - Systematic measurement errors - Non-independence created by poor data collection methods If sampling is biased or the process is not stable, the CLT does not restore validity to inferences. Extreme Non-Normality For distributions that are: - Extremely skewed - Very heavy-tailed - Containing frequent extreme outliers The sample size needed for CLT-based approximations to be accurate can be very large. In such cases: - Examine the data carefully - Consider whether transformations or alternative models are appropriate before relying on normal-based methods --- Practical Checklist for Applying the CLT Before Using Normal-Based Methods for Means Check the following: - Sampling: - Are observations reasonably independent? - Is the process stable during data collection? - Data Shape: - Is the distribution extremely skewed or heavy-tailed? - Are there influential outliers? - Sample Size: - Is n large enough for the level of non-normality observed? - For moderate skewness, is n at least around 30 or higher? Interpreting Results When using CLT-based methods: - Remember that all results are approximations - For borderline sample sizes and obvious skewness: - Treat conclusions with appropriate caution - Consider sensitivity of results to assumptions --- Summary The Central Limit Theorem states that, for sufficiently large samples from any population with finite variance, the distribution of sample means is approximately normal with: - Mean equal to the population mean (μ) - Standard deviation equal to σ/√n (the standard error) This result: - Justifies using normal or t distributions to: - Construct confidence intervals for the mean - Perform hypothesis tests on the mean - Allows valid analysis of means even when individual data are not normal, provided: - Observations are independent and identically distributed - Sample size is adequate relative to the degree of non-normality Understanding the CLT, its conditions, and its practical limitations is essential for correct application of normal-based statistical methods to real process data.