Simple Regression

1
Simple Regression
Mary M. Whiteside, PhD
2
Overview

• Model
• Data
• Least Squares Criterion
• Interpreting b0, b1, and R2
• Partitioning of the Sum of Squares
• LINE Assumptions
• Inferences
• Diagnostics
• Caveats

3
Model

• Y b0 b1X e
• m(Yx) b0 b1X
• where Y and e are random variables
• X is a variable with fixed values
• b0, b1 are unknown parameters

4
Data

• n independent observations of Y for fixed values
  of X
• bi-variate pairs (xi,yi) i 1, 2, , n.
• Data comprise a random sample from a finite
  population or independent observations of a
  random variable
• Example (female literacy, infant mortality rate)

5
Least Squares Criterion

• Minimize sum of squared error from the point (y)
  to the line (yhat)
• Unique analytical solution obtained by
  differentiating the sum of squared error with
  respect to b0 and b1

6
Interpreting b0, b1 and R2

• bo is only interpretable if zero, (0) is in the
  range of the X data. Then, it is the estimated
  expected value of Y when x 0.
• b1 is the estimated change in the expected value
  of Y when X increases by 1 (change scale for
  context)
• R2 is the reduction in the squared error of
  Y associated with the linear relationship with X

7
Partitioning of the total sum of squares TSS of Y

• (mean adjusted)TSS MSS SSE
• Total (mean adjusted) sum of squares model sum
  of squares sum of squared error
• Sy - ?2 Sy ?2 Sy y2
• R2 1 (SSE/TSS)

8
LINE Assumptions

• Linearity - check scatterplot of data or
  residuals vs. X
• Independence - check scatterplots of residuals
  vs. Yhat, X
• Normality histograms of residuals
• Equal Variance - scatterplots of residuals by
  Yhat, X

9
Inferences

• Tests of H0 b10 (r0) (1)
• Confidence interval b0, b1 (2,3)
• Prediction interval Ynext (4)
• Confidence interval mYx (5)

10
Caveats

• Outliers
• Extrapolations