PSYC 3030 Review Session

1
PSYC 3030 Review Session

• Gigi Luk
• December 7, 2004

2
Overview

• Matrix
• Multiple Regression
• Indicator variables
• Polynomial Regression
• Regression Diagnostics
• Model Building

3
Matrix Basic Operation

• Addition
• Subtraction
• Multiplication
• Inverse
• A ? 0
• A is non-singular
• All rows (columns) are linearly independent

Possible only when dimensions are the same
Possible only when inside dimensions are the same
2x3 3x2
4
Matrix Inverse
Linearly Dependent
Linearly independent
5
Some notations

• n sample size
• p number of parameters
• c number of values in x (cf. LOF, p. 85)
• g number of family member in a Bonferroni test
  (cf. p. 92)
• J I H x(xx)-1x

6
Matrix estimates residuals

• LS estimates
• xy (xx)b
• xx
• xy
• (xx)-1

• Residuals
• e
• y xb
• I Hy

7
Matrix Application in Regression
df
MS

• SSE ee yy-bxy n-p SSE/n-p
• SSM 1
• SSR bxy SSM p-1
  SSR/p-1
• SST yy n
• SSTO y(1-J/n)y n-1
• yy SSM

8
Matrix Variance-Covariance
Var-cov (Y) s2(Y)
var-cov (b) est s2(b) s2(b)
MSE (xx)-1
9
Matrix Variance-Covariance
10
Multiple Regression

• Model with more than 2 independent variables y
  ß0 ß1X1 ß2X2 ei

11
MR R-square

• Coefficients of multiple determination
• R2 SSR/SSTO 0 R2 1
• alternative

• Coefficients of partial determination

12
SSTO
SSR(X1)
SSR(X2)
SSR(X1,X2)
SSR(X1X2)
SSR(X2X1)
SSE(X1)
SSE(X2)
SSE(X1,X2)
13
MR Hypothesis testing

• Test for regression relation (the overall test)
  Ho ß1 ß2 .. ßp-1 0 Ha not all ßs 0
• If F F(1-a p-1, n-p), conclude Ho.
• FMSR/MSE
• Test for ßk
• Ho ßk 0 Ha ßk ? 0
• If t t(1-a/2 n-p), conclude Ho.
• t bk/s(bk) F MSR(xkall others)/MSE

14
MR Hypothesis Testing (cont)

• Test for LOF
• Ho EY ßo ß1X1ß2X2. ßp-1Xp-1
• Ha EY ? ßo ß1X1ß2X2. ßp-1Xp-1
• If F F(1-a c-p, n-p), conclude Ho.
• F (SSLF/c-p)/(SSPE/n-c)
• Test whether some ßk0
• Ho ßh ßh1 .. ßp-1 0
• If F F(1-a p-1, n-p), conclude Ho.
• F MSR(xhxp-1x1xh-1)/MSE

15
MR Extra SS (p. 141, CK)

• Full y ßo ß1X1 ß2X2 ? SSR(x1,x2)
• Red y ßo ß1X1 ? SSR(x1)
• SSR (x2x1) SSR(x1,x2) - SSR(x1)
• Effect of X2 adjusted for X1
• SSE(x1) - SSE(x1,x2)
• General Linear Test
• Ho ß2 0 Ha ß2 ? 0
• F

16
Indicator variables
y-hat bo b1X1
y-hat bo b1X1 b2X2
girls
boys
bob2
slope b1
bo
17
y-hat bo b1X1 b2X2 b12X1X2
If b12 0, then there is an interaction ? boys
and girls have different slopes in the relation
of X and Y.
boys
girls
18
Polynomial Regression

• 2nd Order Y ßo ß1X1 ß2X2ei
• 3rd Order Y ßo ß1X1 ß2X2 ß3X3ei
• Interaction
• Y ßo ß1X1 ß2X2 ß11X2 1 ß22X2 2
• ß12X1X2 ei

linear
quadratic
interaction
19
PR Partial F-test (p.303, 5th ed.)

• Test whether a 1st order model would be
  sufficient
• Ho ß11 ß22 ß12 0 Ha not all ßs in Ho
  0
• F

In order to obtain this SSR, you need sequential
SS (see top of p. 304 in text). This test is a
modified test for extra SS.)
20
Regression Diagnostics

• Collinearity
• Effects (1) poor numerical accuracy
• (2) poor precision of estimates
• Danger sign several large s(bk)
• Determinant of xx 0
• Eigenvalues of c of linear dependencies
• Condition (?max/ ?i)1/2
• 15-30 watch out
• 30 trouble
• 100 disaster

21
Regression Diagnostics

• VIF (Variance Inflation Factor)
• 1/(1-R2i)
• When to worry? When VIF 10
• TOL (Tolerance)
• 1/VIFi

22
Model Building

• Goals
• Make R2 large or MSE small
• Keep cost of data collection, s(b) small
• Selection Criteria
• R2 ? look at ?R2
• MSE ? can ? or ? as variables are added

23
Model Building (cont)

• Cp p est. of 1/s2
• Svar(yhat) yhattrue yhatp
• SSEp/MSEall (n-2p)
• p(m1-p)(Fp-1)
• m available predictors
• Fp incremental F for predictors omitted

24
Model Building (cont)

• Variable Selection Procedure
• Choose min MSE Cp p
• SAS tools
• Forward
• Backward
• Stepwise
• Guided selection key vars, promising vars,
  haystack
• Substantive knowledge of the area
• Examination of each var expected sign
  magnitude coefficients