3.2.1 Understanding Inference

Understanding Inference What Statistical Inference Is Statistical inference is the process of using sample data to draw conclusions about a wider population, while quantifying uncertainty. - Population: the entire group of interest - Sample: a subset of the population that is actually measured - Parameter: a numerical characteristic of a population (for example, μ, σ, p) - Statistic: a numerical characteristic calculated from a sample (for example, x̄, s, p̂) Because data come from a sample, every estimate and decision includes uncertainty. Inference provides a disciplined way to: - Estimate unknown parameters - Quantify the accuracy of estimates - Test claims about populations - Support decisions with a known risk of being wrong --- Populations, Samples, and Sampling Error Key Concepts of Sampling Inference always begins with sampling. How the data are collected affects the validity of any conclusion. - Random sample: each population unit has a known, non-zero chance of selection - Independence: each observation does not influence the others - Representativeness: the sample reflects the population of interest Without reasonably random and independent samples, standard inference methods may become unreliable. Sampling Error vs Bias Two distinct issues can affect inference: - Sampling error - Natural variability from taking one sample instead of another - Decreases as sample size increases - Quantified through standard errors and confidence intervals - Bias - Systematic deviation from the true population value - Does not disappear simply by increasing sample size - Examples: non-random sampling, measurement bias Inference tools primarily address sampling error, not bias. Good study design seeks to minimize bias before inference begins. --- Point Estimation and Sampling Distributions Point Estimates A point estimate is a single best guess of a population parameter based on sample data. - Population mean μ → estimate x̄ - Population proportion p → estimate p̂ - Population variance σ² → estimate s² - Population standard deviation σ → estimate s A good estimator is: - Unbiased: on average equals the true parameter - Efficient: has small variability among all unbiased estimators - Consistent: gets closer to the true parameter as sample size grows Sampling Distributions A sampling distribution is the probability distribution of a statistic over all possible random samples of the same size from the same population. Key ideas: - x̄ has its own distribution, with mean μ and standard error σ/√n - p̂ has its own distribution, with mean p and standard error √[p(1−p)/n] - As sample size increases, sampling distributions become more concentrated around the true parameter Inference methods are based on the properties of these sampling distributions. Central Limit Theorem The Central Limit Theorem (CLT) explains why many inference procedures work: - For sufficiently large n, the sampling distribution of x̄ is approximately normal, regardless of the population distribution - Approximate normality improves as: - n increases - the population distribution is not extremely skewed with extreme outliers This allows normal-based methods to be used widely, even with non-normal raw data, when sample sizes are large. --- Confidence Intervals Conceptual Meaning A confidence interval (CI) is a range of plausible values for a population parameter, constructed from sample data, with an associated confidence level. - A 95% CI for μ might be [4.5, 5.3] - Interpretation: if the same procedure were repeated many times, about 95% of the intervals constructed would contain the true μ Important: - The parameter is fixed; the interval is random - Confidence level refers to the long-run performance of the procedure, not to a specific interval “containing μ with 95% probability” General Structure Most confidence intervals have the form: - Estimate ± (Critical value × Standard error) Where: - Estimate: statistic (x̄, p̂, difference of means, etc.) - Critical value: z or t value corresponding to the desired confidence level - Standard error: estimated variability of the statistic As sample size increases: - Standard error decreases - Interval width shrinks - Estimate precision increases Choosing Confidence Level Common choices: - 90%: narrower intervals, less confidence - 95%: widely used compromise - 99%: wider intervals, more confidence Trade-off: - Higher confidence → wider intervals - Lower confidence → narrower intervals Choice should reflect the seriousness of making a wrong inference and the need for precision. --- Hypothesis Testing Framework Purpose and Logic Hypothesis testing uses sample data to evaluate a claim about a population parameter. It is a structured decision process under uncertainty. Two competing statements: - Null hypothesis (H₀): a status quo, equality, or no-effect statement - Alternative hypothesis (H₁ or Hₐ): a competing claim, representing difference or effect Inference assesses whether sample evidence is sufficiently inconsistent with H₀ to justify rejecting it in favor of Hₐ. Null and Alternative Hypotheses Typical forms: - Equality in H₀: - Mean: H₀: μ = μ₀ - Proportion: H₀: p = p₀ - Difference of means: H₀: μ₁ = μ₂ (or μ₁ − μ₂ = 0) - Difference of proportions: H₀: p₁ = p₂ (or p₁ − p₂ = 0) - Alternative directions: - Two-sided: Hₐ: parameter ≠ hypothesized value - One-sided upper: Hₐ: parameter > hypothesized value - One-sided lower: Hₐ: parameter < hypothesized value The choice of one-sided or two-sided must be made before looking at the data and should reflect the real decision objective. Test Statistic, p-Value, and Decision Test statistic: - Standardized measure of how far the sample result is from H₀ in standard error units - Common forms: z-statistic, t-statistic, chi-square, F p-value: - Probability, assuming H₀ is true, of obtaining a test statistic at least as extreme as the observed one in the direction of Hₐ - Smaller p-values indicate stronger evidence against H₀ Significance level (α): - Pre-chosen threshold probability for rejecting H₀, often 0.05 or 0.01 Decision rule: - If p-value ≤ α → Reject H₀, conclude results are statistically significant - If p-value > α → Fail to reject H₀, conclude evidence is insufficient Important: “Fail to reject H₀” is not the same as “prove H₀ is true.” --- Types of Errors and Power Type I and Type II Errors Hypothesis tests can be wrong in two ways: - Type I error (α) - Rejecting H₀ when H₀ is actually true - Controlled directly by choice of significance level α - Type II error (β) - Failing to reject H₀ when H₀ is actually false - Inversely related to the power of the test Trade-off: - Lower α (more conservative) usually increases β unless sample size is increased - Higher α (less conservative) usually lowers β but increases risk of false alarms Power of a Test Power = 1 − β - Probability of correctly rejecting H₀ when a specific alternative is true - High power means the test is sensitive to meaningful differences Power is increased by: - Larger sample size - Larger true effect size (bigger difference from H₀) - Lower data variability - Using a higher α (at the cost of more Type I errors) Inference planning often uses power analysis to determine a sample size that balances error risks and resource constraints. --- Common Inference Procedures Inference for Means Typical scenarios: - One-sample mean: comparing a single mean to a target - Two-sample independent means: comparing two groups - Paired means: comparing before–after or matched pairs Key choices: - Use t-distribution when the population standard deviation is unknown and sample size is moderate or small - Use z-approximation for large samples or when population standard deviation is known Assumptions often include: - Approximate normality of the population or large sample (CLT) - Independence of observations - Random or representative sampling Inference for Proportions Typical scenarios: - One-sample proportion: comparing a proportion to a target - Two-sample proportions: comparing two proportions Approach: - Use normal approximation to the sampling distribution of p̂ when sample size is sufficiently large (for example, np and n(1−p) both reasonably large) Assumptions: - Independent Bernoulli trials (two outcomes: success/failure) - Random or representative samples --- Practical Interpretation of Inference Results Statistical vs Practical Significance Statistical inference may detect very small effects that have little practical impact. - Statistical significance: p-value ≤ α; unlikely result under H₀ - Practical significance: effect size is large enough to matter in practice Key checks: - Examine effect size (difference in means or proportions, ratio of variances, etc.) - Use confidence intervals to understand the plausible range of the effect - Consider cost, benefit, and feasibility of acting on the result Using Confidence Intervals to Support Decisions Confidence intervals provide richer information than a binary hypothesis test decision. - If a CI for difference in means excludes 0, the difference is statistically significant at the corresponding confidence level - The width of the CI indicates estimate precision - CI endpoints help answer: - What is the smallest plausible improvement? - What is the largest plausible improvement? In many cases, it is more informative to report: - Point estimate - Associated confidence interval - p-value - Contextual interpretation --- Assumptions and Robustness Checking Assumptions Inference procedures rely on assumptions about data and sampling. Common assumptions include: - Independence of observations - Approximate normality of the underlying distribution (for certain tests) - Equal variances for some two-sample procedures - Random or at least representative sampling Violations can: - Inflate Type I error rate - Reduce power - Distort p-values and confidence intervals Robustness Concepts A procedure is robust if mild violations of assumptions do not seriously affect its performance. Examples of robustness: - t-tests are fairly robust to moderate non-normality with larger sample sizes - Nonparametric alternatives are sometimes used when assumptions are strongly violated Before trusting inference results, consider: - Data patterns (skewness, outliers) - Sample size - Whether assumptions are approximately met --- Summary Understanding inference involves mastering how sample data are used to make reasoned statements about populations under uncertainty. Key elements include: - Using random, representative samples to estimate population parameters - Recognizing sampling distributions and the role of the Central Limit Theorem - Constructing and interpreting confidence intervals as ranges of plausible parameter values - Formulating and testing hypotheses with clearly defined null and alternative statements - Managing Type I and Type II errors and understanding test power - Distinguishing statistical significance from practical significance - Evaluating and respecting the assumptions behind each inference procedure With these concepts integrated, inference becomes a structured, quantitative way to support decisions using data rather than relying on judgment alone.