Advanced Data Analysis: Multiple Regression

1
Advanced Data Analysis Multiple Regression
2
Advanced Data Analysis Multiple Regression

• What is regression analysis?
• Statistical technique that allows you to
  investigate the relationship between a dependent
  variable (Y) and one (X1) or several independent
  variables (X1, X2, etc.)
• Variables may be of interval or ratio scale

3
Advanced Data Analysis Multiple Regression

• Correlation versus regression
• Correlation -- closeness of the relationship
  between two variables (Y and X)
• Regression -- derivation of a linear equation
  that explains the relationship between two or
  more variables (Y, X1, X2, etc.)
• General equation Y a biXi e
• a is an intercept term b represents the change
  in Y that is explained (or predicted) by a one
  unit change in X e is an error term

4
Advanced Data Analysis Multiple Regression

• The regression equation (example)
• Y 32 .55X e
• When X 0, Y 32
• For each increase in X, Y increases by .55
• When X 1, Y 32.55
• When X 2, Y 33.10

5
Advanced Data Analysis Multiple Regression

• Coefficient of determination (r2 or R2)
• r2 1 - unexplained variation (in Y by X) /
  total variation in Y or
• r2 explained variation (in Y by X) / total
  variation in Y
• In a perfect world r2 1
• R2 for variation in Y explained by all Ys

6
Advanced Data Analysis Multiple Regression

• Data do not fit perfectly in a linear sense
• Must estimate the best linear equation
  (Ordinary Least Squares algorithm)

7
Advanced Data Analysis Multiple Regression

• Interpretation of Results
• Overall Model Evaluation -- Ho R2 0 Ha R2
  gt 0
• F-test
• Are individual b coefficients significant? -- Ho
  bi 0 Ha bi n.e. 0
• t-test
• EXAMPLE

8
Advanced Data Analysis Multiple Regression

• Multicollinearity -- two or more X variables are
  significantly correlated
• Reduces the overall predictive (or explanatory)
  power of each variable
• Non-Linear Relationship -- relationship between X
  and Y cannot be explained with a straight line
• Predictive ability of model for Y depends on
  predictions of X