Chapter 6: Multiple Regression I

1
Chapter 6 Multiple Regression I
Ayona Chatterjee Spring 2008 Math 4813/5813
2
Need for Several Predictor Variables

• Suppose you want to study operating cost of a
  branch office of a consumer finance chain.
  Predictor variables can be.
• Number of new loan applications
• Number of outstanding loans
• Study peak plasma growth hormone in children.
  Predictors can be
• Age
• Gender
• Height
• Weight

3
The Need for Multiple Regression

• At times single variable regression approach may
  be imprecise.
• More variables in the model help in better
  predictions.
• Multiple regression is very useful in an
  experimental set up where the experimenter can
  control a variety of predictor variables
  simultaneously.

4
First-Order Model with Two Predictor Variables

• When there are two predictor variables X1 and X2,
  the first-order regression model with two
  variables is given as
• As before we have E?i0.
• EY?0? 1X1? 2X2

5
Note

• For simple linear regression, the regression
  function is a line.
• For multiple regression, the regression function
  is a plane.
• The mean response EY corresponds to a
  combination of levels of X1and X2.
• The regression function is called the regression
  surface or the response surface.

6
Meaning of Regression Coefficients

• The value of ?0 represents the the mean response
  of EY when X10 and X20.
• In some models ?0 may not have any inherent
  meaning.
• The parameter ?1 indicates the change in the mean
  response EY per unit increase in X1 when X2 is
  held constant.
• Similarly we can define ?2.

7
Additive Effects

• When the effect of X1 on the mean response does
  not depend on the level of X2, and
  correspondingly the effect of X2 does not depend
  on the level of X1, the two predictor variables
  are said to have additive effects or not to
  interact.

8
Example

• A regression model relation test market sales (Y)
  to point-of-sale expenditure(X1) and TV
  expenditures (X2) is EY102X15X2

• Here ?12, if point-of-sale expenditure is
  increased by 1 unit while TV expenditures are
  held constant, then expected sales Y is increased
  by 2 units.

9
First-Order Model with more than Two Predictor
Variables

• Suppose there are p 1 predictor variables X1,
  , Xp-1. The first-order regression model can be
  written as below

10
Note

• This response function is a hyperplane, a plane
  in more than two-dimensions.
• The meaning of the parameters is analogous to the
  case of two predictor variables.
• The parameter ?k indicates the change in the mean
  response EY with a unit increase in Xk, when
  all other predictor variables are held constant.

11
General Linear Regression Model

• The variables X1, , Xp-1 do not have to
  represent different predictor variables.
• We define the general linear regression model,
  with normal error terms, simply in terms of X
  variables
• Here ?0, ?1, ., ?p-1 are parameters.
• Xi1, Xi2, , Xip-1 are known constants.
• ? i are independent N(0, ?2)
• i 1, , n

12
Qualitative Predictor Variable

• The general linear model allows for both
  quantitative and qualitative variables.
• Consider a regression analysis to predict the
  length of hospital stay (Y) based in the age (X1)
  and gender (X2) of the patient.
• We define X2 as follows

This is called an indicator function
13
Polynomial Regression

• Special cases of the general linear regression
  model.
• The following is a polynomial regression model
  with one predictor variable
• Polynomials with higher-degree polynomial
  response functions and transformed variables are
  also particular cases of the general linear
  regression model.

14
Interaction Effects

• When the effects of the predictor variables on
  the response variable are not additive, the
  non-additive regression model with two predictor
  variables is
• This can also be written as a GLRM.

15
GLRM in Matrix Terms

• Lets write it out by hand!
• Matrix notation hides the enormous computational
  complexities.
• We will perform matrix manipulations on computers.