Practical Model Selection and Multi-model Inference using R

1
Practical Model Selection and Multi-model
Inference using R

• Presented by
• Eric Stolen and Dan Hunt

2
Indigo snake home range

• radio telemetry study of eastern indigo snake
  home range sizes in central FL
• Response variable was ln(home range) (95
  fixed-kernel estimate)
• sex, landcover type, and length of time (weeks)
• General linear model
• Interested in effect sizes and predictions

3
Indigo snake home range

• Science questions
• Is there evidence for a difference in home range
  size between habitats?
• Is there evidence for a difference in home range
  size between sexes?
• Is there an effect of length of time tracked?
• Is the effect of habitats type different between
  the sexes (is there an interaction term)?
• Does the data support estimating an effect of
  habitats type with 3 levels or 2 levels?

4
Models Indigo Snake example
SEX land cover type 3 levels (lc1) land cover
type 2 levels(lc2) weeks SEX lc1 SEX lc2 SEX
weeks lc1 weeks lc2 weeks SEX lc1
weeks SEX lc2 weeks SEX lc2 SEX lc1
SEX lc2 SEX lc2 SEX lc2 weeks SEX
lc1 SEX lc2 weeks SEX lc2
5
Models Indigo Snake example
SEX land cover type 3 levels (lc1) land cover
type 2 levels(lc2) weeks SEX lc1 SEX lc2 SEX
weeks lc1 weeks lc2 weeks SEX lc1
weeks SEX lc2 weeks SEX lc1 SEX lc1
SEX lc2 SEX lc2 SEX lc1 weeks SEX
lc1 SEX lc2 weeks SEX lc2
6
Models Indigo Snake example
SEX land cover type 3 levels (lc1) land cover
type 2 levels(lc2) weeks SEX lc1 SEX lc2 SEX
weeks lc1 weeks lc2 weeks SEX lc1
weeks SEX lc2 weeks SEX lc1 SEX lc1
SEX lc2 SEX lc2 SEX lc1 weeks SEX
lc1 SEX lc2 weeks SEX lc2
7
Models Indigo Snake example
SEX land cover type 2 levels(lc2) weeks SEX
lc2 SEX weeks lc2 weeks SEX lc2 weeks SEX
lc2 SEX lc2 SEX lc2 weeks SEX lc2

8
Models Indigo Snake example
SEX land cover type 2 levels(lc2) weeks SEX
lc2 SEX weeks lc2 weeks SEX lc2 weeks SEX
lc2 SEX lc2 SEX lc2 weeks SEX lc2

9
Models Indigo Snake example
SEX land cover type 2 levels(lc2) weeks SEX
lc2 SEX weeks lc2 weeks SEX lc2 weeks SEX
lc2 SEX lc2 SEX lc2 weeks SEX lc2

10
I-T mechanics

• AICci -2loge (Likelihood of model i given the
  data) 2K (n/(n-K-1))
• or
• AIC 2K(K1)/(n-K-1)
• (where K the number of parameters estimated and
  n the sample size)

11
I-T mechanics

• AICcmin AICc for the model with the lowest AICc
  value
• Di AICci AICcmin

12
I-T mechanics

• Model Probability (also Bayesian posterior model
  probabilities)
• evidence ratio of model i to model j wi / wj

13
I-T mechanics

• Least Squares Regression
• AIC n loge (s2) 2K (n/(n-K-1))
• Where s2 RSS / n
• (explain offset for constant part)

14
I-T mechanics

• Counting Parameters
• K number of parameters estimated
• Least Square Regression
• K number of parameters 2 (for intercept s)

15
I-T mechanics

• Counting Parameters
• K number of parameters estimated
• Logistic Regression
• K number of parameters 1 (for intercept)

16
I-T mechanics

• Counting Parameters
• Non-identifiable parameters

17
Comparing Models
18
Comparing Models
Evidence Ratio 3.03
19
Comparing Models
Evidence Ratio 4.28 (.34.22.14.08) /
(.11.04.02.01)
20
Mathematical details

• What types of models can be compared within a
  single I-T analysis?
• Data must be fixed (including response)
• Must be able to calculate maximum likelihood
• (ways to deal with quasi-likelihood)
• Models do not need to be nested
• In some cases AIC is additive

21
Model Fitting Preliminaries

• Understanding the data/variables
• Avoid data dredging!
• safe data screening practices
• Detect outliers, scale issues, collinearity
• Tools in R

22
Tools in R

• Tools in R
• Generalized linear models
• lm
• glm
• Packages
• Design Package
• FE Harrell. 2001. Regression Modeling Strategies
  with Applications to Linear Models, Logistic
  Regression, and Survival Analysis. Springer.
• CAR package
• Fox, J. 2002. An R and S-plus Companion to
  Applied Regression. Sage Publications.

23
Tools in R

• Tools in R
• Model formula
• Ex)
• Output
• summary(model4)
• model4aic
• Model4coefficients

model4 lt- glm(helpage2 sex mom_dad suburb
brdeapp matepp density I(density2) ,
familybinomial,datachoices)
24
Tools in R

• Fitting the model set
• R program does the work
• Trouble-shooting
• Export results

25
Model Checking

• Model Checking
• Global model must fit
• Models used for inference must meet assumptions,
• Look for numerical problems
• Tools in R

26
Interpretation of models for inference

• Case 1 One or a few models best models
• Examining model parameters and predictions
• Effects
• Prediction
• graphing results
• nomograms
• Presenting Results
• Anderson, D. R., W. A. Link, D. H. Johnson, and
  K. P. Burnham. 2001. Suggestions for presenting
  the results of data analysis. Journal of Wildlife
  Management 65373-378.

27
Multi-model Inference

• Model selection uncertainty
• Model-average prediction
• Model-average parameter estimates

28
Model Averaging Predictions
29
Model Averaging Predictions
30
Model Averaging Predictions
31
Model Averaging Predictions
32
Model Averaging Parameters
33
Unconditional Variance Estimator
34
Unconditional Variance Estimator
35
Snake Example