Diagnostics for Linear Regression

1
Diagnostics for Linear Regression

• STAT120C

2
Introduction

• Purpose introduce a few simple methods to
  examine assumptions of linear regression when
  there is one covariate
• Methods are visual and subjective
• Focus on residuals
• Methods introduced are very limited. Learn more
  from STAT110.

3
The Model

• (4) normality
• Diagnostics also focus on these two components

Systematic component
random component
4
Possible Departures from Model

• Nonlinearity (systematic)
• Presence of outliers the model fits all but a
  few points (systematic)
• Nonconstancy of variance (random)
• Non-independent variances (random)
• Major violation from normality (random)

5
Linearity?

• Are X and Y in a linear relationship?
• Plot y v.s. x residuals v.s. x or residuals
  v.s.

6
Outliers?

• Does any observation standout in the y space?

7
Outliers
8
Random Component

• Constant variance?
• Plot residuals v.s. x or fitted
• Independence?
• Plot residuals v.s. x or fitted
• Normality?
• Boxplot of residuals. Other tools such as
  histogram
• Quantile-quantile plot residuals (normal
  probability plot)

9
Random Component
10
What to do if violations are detected

• Nonlinearity change regression terms. E.g., add
  a quadratic term
• Outliers remove outliers
• Nonconstancy of variance transformation
• Nonindependence consider models allow correlated
  errors
• Nonnormality transformation

11
Example fixes for nonconstancy

• Two methods will be introduced
• Weighted least square (WLS)
• Transformations

12
Transformations

• Example Poisson example.
• See class notes
• Other useful transformations log, power,
  box-cox, exponential