Multivariate statistical analysis

1
Multivariate statistical analysis

• Regression analysis

2
Regression vs. correlation

• ???????????????(??)????
• ?????????(association)??

3
Regression model

• (Y1, Y2, Yj)f(X1, X2,Xk)
• k?2, multiple regression(???)
• j?2, multivariate regression(????)
• The assumed model, ynß0ß1x1ß2x2ßnxnen,
• en is the random error term based on some
  prerequisite assumptions
• Normal i.i.d. N(0, s2)
• Normality
• Independence
• Variance equality

4
Modeling the regression line

• Ref.

5
ANOVA table for regression analysistotal model
testing
6
Sum of errors

• Sum of squares for error (SSE)
• Sum of squares for model (SSM)
• Sum of squares for total (SST)
• MSESSE/d.f. of errorSSE/K
• MSMSSM/d.f. of modelSSM/(N-K-1)
• d.f. of totalN-1
• FMSM/MSE

7
Determination

• Coefficient of determination
• R2SSM/SST1-SSE/SST, 0?R2?1
• Adjusted coefficient of determination
• Adjusted by means of dividing by degree of
  freedom
• Adj. R21-SSE/(N-K-1)/SST/(N-1)1-(1-R2)(N-1)
  /(N-K-1)
• NgtK1, ??????????????
• Determining the goodness of fit of a sampled
  regression line

8
t-test for the coefficients of explaining
variablesMarginal testing
9
Conflicts between total testing and marginal
testing

• Confidence interval vs. confidence region (a
  region composed with several more narrower
  interval confidence intervals respectively)

10
Determine the predictors

• Checking the contribution of additional variables
• Stepwise regression
• Forward regression
• Backward regression

11
Testing the assumptions

• Normality testing
• Wilk-shapiro statistics
• Q-Q/ P-P plotting (expected distribution vs. real
  distribution)
• Variance equality testing
• Scatter the error term along xn
• Verify the randomized pattern
• Durbin-Watson test for testing the first
  autocorrelation of residuals
• Mean2, if gt2, - relation, if lt2, relation
• Independence testing
• Assumed the random independent sampling process
  for the cross-sectional data
• Time-series analysis for the longitudinal data

12
Colinearity

• A pair of predictor variables that are strongly
  correlated
• Tolerance, 1-Rj2 ,
• if there exists strong correlation, the Tolerance
  will be smaller and near to zero
• VIF (variance inflation factor)
• The inverse of tolerance, if tolerance is small,
  VIF will inflate very large

13
Outliers

• Leverage hjj, (lt1)
• hjj1/nsquare(objj - obj mean)/S square(objj
  - obj mean)
• If hjj is comparatively too large, remove this
  observation.

14
Weighted regression

• The different impact of sample data
• Outliers ? set the influence weight near to 0

15
Data transformation

• Transformation for normality, variance equality
• Transformation by log, or inverse, square