9'Additive models, trees and related methods'

1
9.Additive models, trees and related methods.

• 9.1 Generalized additive models
• 9.2 Tree-based models
• 9.3 PRIM Bump hunting
• 9.4 MARS (Multivariate Adaptive Regression
  Splines)
• 9.5 Hierarchical Mixtures of Experts
• 9.6 Missing data
• 9.7 Computational considerations

2
9.1.Generalized Additive Models
3

• Idea
• - estimate for each predictor variable a
  unspecific (non-parametric) function
• - connect them to the dependent variable via a
  link function
• Usefull identification and caracterisation of
  no linear effects
• Advantages - non parametric form of fj ? more
  flexible models
• - maintain of
  additivity ? same interpretation

4

• Not all of the fj need to be nonlinear
• We can mix in linear and
• other parametric forms with the non
  linear terms
• Eg a semi parametric model
• X vector of predictors linearly
  modeled
• ak effect for the kth level of
  a qualitative input V
• f(Z) effect of predictor Z
  nonparametrically modeled

5
9.1.1.Fitting Additive Model

• Using an algorithm
• whose basic building block a
  scatterplot smoother
• ex the cubic smoothing spline (chap5)
• Additive model form
• Minimize penalized sum of squares for additive
  model
• ex section 5.4.

6

• Backfitting algorithm
• Initialize
• Cycle j 1,2,, p,,1,2,, p,, (m cycles)
• Until the functions change less than a
  prespecified threshold