Models With Two or More Quantitative Variables

1
Models With Two or More Quantitative Variables
2
Types of Regression Models
3
First-Order Model With 2 Independent Variables

• Relationship Between 1 Dependent 2 Independent
  Variables Is a Linear Function
• Assumes No Interaction Between X1 X2
• Effect of X1 on E(Y) Is the Same Regardless of X2
  Values

4
First-Order Model With 2 Independent Variables

• Relationship Between 1 Dependent 2 Independent
  Variables Is a Linear Function
• Assumes No Interaction Between X1 X2
• Effect of X1 on E(Y) Is the Same Regardless of X2
  Values
• Model

5
No Interaction
6
No Interaction
E(Y)
E(Y) 1 2X1 3X2
12
8
4
0
X1
0
1
0.5
1.5
7
No Interaction
E(Y)
E(Y) 1 2X1 3X2
12
8
4
E(Y) 1 2X1 3(0) 1 2X1
0
X1
0
1
0.5
1.5
8
No Interaction
E(Y)
E(Y) 1 2X1 3X2
12
8
E(Y) 1 2X1 3(1) 4 2X1
4
E(Y) 1 2X1 3(0) 1 2X1
0
X1
0
1
0.5
1.5
9
No Interaction
E(Y)
E(Y) 1 2X1 3X2
12
E(Y) 1 2X1 3(2) 7 2X1
8
E(Y) 1 2X1 3(1) 4 2X1
4
E(Y) 1 2X1 3(0) 1 2X1
0
X1
0
1
0.5
1.5
10
No Interaction
E(Y)
E(Y) 1 2X1 3X2
E(Y) 1 2X1 3(3) 10 2X1
12
E(Y) 1 2X1 3(2) 7 2X1
8
E(Y) 1 2X1 3(1) 4 2X1
4
E(Y) 1 2X1 3(0) 1 2X1
0
X1
0
1
0.5
1.5
11
No Interaction
E(Y)
E(Y) 1 2X1 3X2
E(Y) 1 2X1 3(3) 10 2X1
12
E(Y) 1 2X1 3(2) 7 2X1
8
E(Y) 1 2X1 3(1) 4 2X1
4
E(Y) 1 2X1 3(0) 1 2X1
0
X1
0
1
0.5
1.5
Effect (slope) of X1 on E(Y) does not depend on
X2 value
12
First-Order Model Worksheet
Run regression with Y, X1, X2
13
Types of Regression Models
14
Interaction Model With 2 Independent Variables

• Hypothesizes Interaction Between Pairs of X
  Variables
• Response to One X Variable Varies at Different
  Levels of Another X Variable

15
Interaction Model With 2 Independent Variables

• Hypothesizes Interaction Between Pairs of X
  Variables
• Response to One X Variable Varies at Different
  Levels of Another X Variable
• Contains Two-Way Cross Product Terms

16
Interaction Model With 2 Independent Variables

• 1. Hypothesizes Interaction Between Pairs of X
  Variables
• Response to One X Variable Varies at Different
  Levels of Another X Variable
• 2. Contains Two-Way Cross Product Terms
• 3. Can Be Combined With Other Models
• Example Dummy-Variable Model

17
Effect of Interaction
18
Effect of Interaction

• 1. Given

19
Effect of Interaction

• 1. Given
• 2. Without Interaction Term, Effect of X1 on Y
  Is Measured by ?1

20
Effect of Interaction

• 1. Given
• 2. Without Interaction Term, Effect of X1 on Y
  Is Measured by ?1
• 3. With Interaction Term, Effect of X1 on Y Is
  Measured by ?1 ?3X2
• Effect changes As X2 changes

21
Interaction Model Relationships
22
Interaction Model Relationships
E(Y) 1 2X1 3X2 4X1X2
E(Y)
12
8
4
0
X1
0
1
0.5
1.5
23
Interaction Model Relationships
E(Y) 1 2X1 3X2 4X1X2
E(Y)
12
8
E(Y) 1 2X1 3(0) 4X1(0) 1 2X1
4
0
X1
0
1
0.5
1.5
24
Interaction Model Relationships
E(Y) 1 2X1 3X2 4X1X2
E(Y) 1 2X1 3(1) 4X1(1) 4 6X1
E(Y) 1 2X1 3(0) 4X1(0) 1 2X1
25
Interaction Model Relationships
E(Y) 1 2X1 3X2 4X1X2
E(Y) 1 2X1 3(1) 4X1(1) 4 6X1
E(Y) 1 2X1 3(0) 4X1(0) 1 2X1
Effect (slope) of X1 on E(Y) does depend on X2
value
26
Interaction Model Worksheet
Multiply X1 by X2 to get X1X2. Run regression
with Y, X1, X2 , X1X2
27
Thinking challenge

• Assume Y Milk yield, X1 food intake and X2
  weight
• Assume the following model with interaction
• Interpret the interaction

Y 1 2X1 3X2 4X1X2
28
Types of Regression Models
29
Second-Order Model With 2 Independent Variables

• 1. Relationship Between 1 Dependent 2 or More
  Independent Variables Is a Quadratic Function
• 2. Useful 1St Model If Non-Linear Relationship
  Suspected

30
Second-Order Model With 2 Independent Variables

• 1. Relationship Between 1 Dependent 2 or More
  Independent Variables Is a Quadratic Function
• 2. Useful 1St Model If Non-Linear Relationship
  Suspected
• 3. Model

31
Second-Order Model Worksheet
Multiply X1 by X2 to get X1X2 then X12, X22.
Run regression with Y, X1, X2 , X1X2, X12, X22.
32
Models With One Qualitative Independent Variable
33
Types of Regression Models
34
Dummy-Variable Model

• 1. Involves Categorical X Variable With 2 Levels
• e.g., Male-Female College-No College
• 2. Variable Levels Coded 0 1
• 3. Number of Dummy Variables Is 1 Less Than
  Number of Levels of Variable
• May Be Combined With Quantitative Variable (1st
  Order or 2nd Order Model)

35
Dummy-Variable Model Worksheet
X2 levels 0 Group 1 1 Group 2. Run
regression with Y, X1, X2
36
Interpreting Dummy-Variable Model Equation
37
Interpreting Dummy-Variable Model Equation
?
?
?
?
Y
X
X
?
?
?
?
?
?
Given

i
i
i
0
1
1
2
2

Y
?
Starting s
alary of c
ollege gra
d'
s

X
?
GPA
1
0
i
f Male
X
?
2
1
if Female
38
Interpreting Dummy-Variable Model Equation
?
?
?
?
Y
X
X
?
?
?
?
?
?
Given

i
i
i
0
1
1
2
2

Y
?
Starting s
alary of c
ollege gra
d'
s

X
?
GPA
1
0
i
f Male
X
?
2
1
if Female
Males (
)
X
?
0
2
?
?
?
?
?
?
Y
X
X
?
?
?
?
?
?
?
?
?
?
(0)
i
i
i
0
1
1
2
0
1
1
39
Interpreting Dummy-Variable Model Equation
?
?
?
?
Y
X
X
?
?
?
?
?
?
Given

i
i
i
0
1
1
2
2

Y
?
Starting s
alary of c
ollege gra
d'
s

X
?
GPA
1
0
i
f Male
X
?
2
1
if Female
Same slopes
Males (
)
X
?
0
2
?
?
?
?
?
?
Y
X
X
?
?
?
?
?
?
?
?
?
?
(0)
i
i
i
0
1
1
2
0
1
1
Females (
)
X
?
1
2
?
?
?
?
?
?
?
Y
X
X
?
?
?
?
?
?
?
?
(?
?
?
? )
(1)
i
i
i
0
1
1
2
1
1
0
2
40
Dummy-Variable Model Relationships
Y

Same Slopes ?1
Females


?0 ?2

?0
Males
0
X1
0
41
Dummy-Variable Model Example
42
Dummy-Variable Model Example
?

Y
X
X
?
?
?
3
5
7
Computer O
utput
i
i
i
1
2
i
0
f Male
X
?
2
1
if Female
43
Dummy-Variable Model Example
?

Y
X
X
?
?
?
3
5
7
Computer O
utput
i
i
i
1
2
i
0
f Male
X
?
2
1
if Female
Males (
)
X
?
0
2
?
Y
X
X
?
?
?
?
?
3
5
7
3
5
(0)
i
i
i
1
1
44
Dummy-Variable Model Example
?

Y
X
X
?
?
?
3
5
7
Computer O
utput
i
i
i
1
2
i
0
f Male
X
?
2
1
if Female
Same slopes
Males (
)
X
?
0
2
?
Y
X
X
?
?
?
?
?
3
5
7
3
5
(0)
i
i
i
1
1
Females
)
(X
?
1
2
?
Y
X
?
?
?
?
3
5
7
(1)
X
?
(3 7)
5
i
i
1
i
1
45
Sample SAS codes for fitting linear regressions
with interactions and higher order terms

• PROC GLM datacomplex
• Class gender
• model salary gpa gender gpagender
• RUN

46
Conclusion

• Explained the Linear Multiple Regression Model
• Tested Overall Significance
• Described Various Types of Models
• Evaluated Portions of a Regression Model
• Interpreted Linear Multiple Regression Computer
  Output
• Described Stepwise Regression
• Explained Residual Analysis
• Described Regression Pitfalls