4.4.4 Fit, Diagnose Model and Center Points

Fit, Diagnose Model and Center Points Introduction This article explains how to fit regression and designed experiment models, diagnose their adequacy, and use center points in designed experiments. The focus is practical: how to build and verify a good model of the process response and how to use center points to detect curvature and improve the model. --- Fitting the Model Choosing the Model Form In improvement projects, models are usually based on: - Simple linear regression – one predictor, straight-line relationship. - Multiple linear regression – several predictors, straight-line approximation. - Designed experiment (DOE) models – main effects and interaction terms from factorial or fractional factorial designs. - Quadratic models – linear terms plus squares and sometimes interactions, often arising from center points or response surface designs. The model form must reflect: - The experimental design (what factors and levels were run). - Prior process understanding (suspected curvature or interactions). - The goal: prediction, explanation, or factor screening. Fitting Regression and DOE Models A typical model for k predictors is: - General form: Y = β₀ + β₁X₁ + β₂X₂ + … + βₖXₖ + error For DOE with two-level factors: - Coded units: -1 for low, +1 for high, 0 for center (if used). - Main effects: each factor alone. - Interactions: product terms (e.g., X₁X₂). Steps to fit: - Define response Y and predictors X’s. - Code factors if using DOE (−1, +1, and possibly 0). - Fit the model using least squares. - Examine the initial full model including relevant main effects and interactions. Selecting Terms: Main Effects and Interactions In factorial or regression modeling: - Main effects: impact of each factor individually. - Two-factor interactions: joint effect when two factors change together. - Higher-order interactions: three or more factors together, usually smaller and often omitted unless strongly indicated. Term selection typically uses: - Practical knowledge – which interactions are plausible in the process. - Statistical evidence – p-values, standardized effect plots, Pareto charts of effects. - Design resolution – which interactions can be estimated separately from main effects (in fractional factorials). A useful strategy: - Start with a hierarchical model: - Include significant interactions. - Keep the corresponding main effects even if not individually significant. - Remove clearly non-significant higher-order terms while maintaining hierarchy. --- Model Adequacy and Goodness of Fit Key Model Statistics After fitting, check indicators of fit: - R² (coefficient of determination) - Proportion of variation in Y explained by the model. - Higher is better, but very high R² alone does not guarantee a good model. - Adjusted R² - Adjusts for number of predictors. - Useful when comparing models with different numbers of terms. - Helps avoid overfitting. - Predicted R² - Based on cross-validation or prediction errors. - Assesses how well the model predicts new data. - Large gap between adjusted R² and predicted R² suggests overfitting or problems in the model. - Standard error of the estimate (s) - Typical size of residuals. - Smaller s indicates tighter fit around the regression line. ANOVA and F-test Analysis of variance (ANOVA) summarizes model fit: - Mean square for regression (MSR) – variation explained by the model. - Mean square error (MSE) – residual variation not explained by the model. - F-statistic = MSR / MSE - Tests the null hypothesis that all model coefficients (except intercept) are zero. Interpretation: - Large F and small p-value for the overall model: - Evidence that at least one predictor is significantly related to the response. - p-values for individual terms: - Indicate which factors or interactions are statistically significant. - Use in combination with practical importance and design structure. Lack-of-Fit Test The lack-of-fit test checks whether the chosen model form is adequate: - Requires replicated runs at the same factor settings (often at center points or other design points). - Partitioning of error: - Pure error – variation between replicates at identical settings. - Lack-of-fit – leftover variation when the model does not match the true mean response. Key ideas: - If lack-of-fit is statistically significant (small p-value): - The model form is missing important structure (e.g., curvature, interactions, nonlinearity). - Reconsider adding terms such as quadratic or omitted interactions. - If lack-of-fit is not significant: - No strong evidence that the model form is wrong, given the data. - Still verify assumptions and practical reasonableness. --- Model Diagnostics Residual Analysis Residuals are the differences between observed and fitted values: - Residual = Observed Y − Predicted Y They provide a way to test the assumptions of linear modeling: - Linearity – correct functional form. - Constant variance (homoscedasticity) – similar spread across predicted values and factors. - Independence – errors are not correlated. - Normality – residuals roughly normal (primarily important for inference). Common residual plots: - Residuals vs fitted values - Random scatter: supports linearity and constant variance. - Funnel shape: suggests non-constant variance (may need transformation). - Curved pattern: suggests nonlinearity or missing terms (e.g., squared terms). - Residuals vs each predictor - Helps detect patterns tied to specific factors. - Non-random patterns: consider additional terms (e.g., interactions, nonlinear terms). - Normal probability plot (or histogram) of residuals - Approximately straight line: supports normality. - Strong curvature or heavy tails: consider transformation or robust methods. - Residuals vs order (run sequence) - Detects time trends or drifts in process conditions. Influential Points and Leverage Not all points have equal impact on the model: - Leverage – numeric measure of how far a point’s predictor values are from the center of the design. - High leverage points can strongly influence the fitted line or surface. - Influence – combination of leverage and residual size. - Common measures: Cook’s distance and DFFITS (names only; interpretation depends on software cutoffs). Practical actions: - Investigate influential points: - Check for data errors, special causes, or unusual conditions. - Do not automatically delete them: - Decide based on process knowledge and data validity. - If a valid influential point reveals important behavior: - Consider redesigning the experiment to better cover that region. Transformations and Model Refinement If diagnostics show problems, possible fixes include: - Transform response: - Log, square root, or Box–Cox transformation to stabilize variance or improve normality. - Add missing terms: - Interactions or quadratic terms to capture curvature. - Simplify model: - Remove non-essential terms to reduce overfitting, especially if predicted R² is poor. Always refit and recheck residuals after any change. --- Center Points in Designed Experiments Purpose of Center Points Center points are runs at the mid-level of all quantitative factors in a factorial design: - For each factor: - Low = coded −1 - High = coded +1 - Center = coded 0 (midpoint between low and high) They serve key purposes: - Detect curvature – reveal departures from linearity between low and high settings. - Estimate pure error – provide replicate data at a consistent location in the design space. - Improve robustness – often represent a practically reasonable operating region. Center points are meaningful for quantitative factors with ordered levels. They are not applicable for categorical-only factors. Detecting Curvature with Center Points In a two-level factorial design, main effects are assumed linear between low and high. If the true response curve is curved, the straight-line model is inadequate. Center points help by comparing: - The average response at factorial points (all combinations of low and high). - The average response at center points. Logic: - If the true relationship is linear: - The mean at the center equals the mean of predictions from the linear model at the center. - If there is curvature: - The center-point mean differs systematically from the linear prediction. Statistical test: - Compare average response at center points to average of the factorial vertices. - A significant difference indicates overall curvature in at least one factor. Interpretation: - Significant curvature: - Add quadratic terms (e.g., X₁²) and possibly consider a response surface design. - No significant curvature: - A first-order (linear) model may be adequate within the studied range. Center Points and Pure Error When center points are replicated: - Replication at same settings gives: - An unbiased estimate of pure error (random variation in the process). - This pure error is crucial for: - Lack-of-fit tests – separating modeling error from random noise. - More accurate confidence intervals and significance tests on effects. Guidelines: - Use multiple center point runs (not just one) to: - Average out random noise. - Ensure a stable estimate of pure error. Planning and Using Center Points When designing experiments: - When to include center points: - When factors are quantitative and ranges are not extremely narrow. - When linearity across the studied range is uncertain. - When an estimate of pure error is needed for lack-of-fit tests. - How many center points: - Enough to provide a reasonable estimate of pure error (often 3–6 scattered through the run order). - Balance against overall experiment size. - Placement in the run order: - Randomly distributed or deliberately spread across the sequence (early, middle, late) to check stability over time. Practical uses after running: - If curvature is detected: - Consider transitioning from screening (two-level factorial) to a response surface design that includes axial and center runs. - If no curvature is detected: - Retain a simpler first-order model and focus on optimizing within the linear region. --- Integrating Fitting, Diagnosis, and Center Points Bringing these elements together: - Fit: - Build an initial linear (or linear–interaction) model using regression or DOE results. - Use coded variables when analyzing factorial designs. - Use center points: - Assess curvature and estimate pure error. - Run a curvature test: if significant, enrich the model with quadratic terms or extend the design. - Diagnose: - Check ANOVA, R² statistics, and lack-of-fit test. - Examine residual diagnostics for assumption violations. - Investigate influential points and re-evaluate suspicious data. - Refine: - Add or remove terms while maintaining model hierarchy. - Apply transformations when needed. - Reassess fit and diagnostics after each adjustment. The goal is a model that: - Accurately describes the response within the studied factor ranges. - Satisfies statistical assumptions reasonably well. - Provides reliable predictions and insights for process improvement and control. --- Summary A sound model arises from a disciplined cycle of fitting, diagnosing, and refining: - Fit the model using regression or DOE methods with appropriate main effects and interactions. - Assess adequacy through R² measures, ANOVA, and lack-of-fit tests. - Diagnose assumptions with residuals, normality checks, and influence measures. - Use center points to detect curvature and estimate pure error, enabling meaningful lack-of-fit testing. - Refine the model by adding curvature terms, adjusting the design, or simplifying the structure until the model both fits the data and makes practical sense. Mastering these steps ensures that conclusions about factor effects and process behavior rest on statistically valid, well-diagnosed models.