Time and tide wait for no man

1
Time and tide wait for no man!
2

• It is a lifetimes business of study, practice,
  mistakes and successes.

3

• Trimmed and Winsorized Estimators
• Based on Scaled Deviation
• Mingxin Wu
• (under the guidance of Prof. Yijun Zuo)
• Department of Statistics and
  Probability
• Michigan State University

4
Outline

• Location
• Why trimmed mean
• What is scaled-deviation trimmed mean
  
• Why not the ordinary trimmed mean
  
• Scaled-deviation trimmed mean
  
  
• Scaled-deviation winsorized mean
• Scale
• Overview
• Why trimmed scale
  
• Scaled-deviation trimmed/winsorized scale
  
  
• Open Problems

5
Why trimming?

• Robustness
• Efficiency

6
(No Transcript)
7
(No Transcript)
8
Why not Ordinary Trimming?
Ordinary Trimming
Outliers
Scaled-deviation Trimming
?n
7?n
7?n
9
Why not Ordinary Trimming?
Ordinary Trimming
Outliers
Scale-deviation Trimming
?n
7?n
7?n
10
Why not Ordinary Trimming?
Ordinary Trimming
Scaled-deviation Trimming
?n
7?n
7?n
11
Robustness-Breakdown Point
Scale-deviation trimmed mean
Minimum fraction of ''bad points'' in a data set
that can render the estimator useless
highest possible
Depend on trimmed level
12
F(?, ?x)(1-?)F? ?x
13
(No Transcript)
14
IF(x, T(?)) at ?2
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
Scaled-deviation winsorized mean
Replace by Un?n??n
?n
??n
??n
Un
Ln?n-??n
Replace by Ln?n-??n
?n
??n
??n
Un
Ln
19
Scale-deviation Winsorized Mean
20
Why Winsorizing?

• Information
• Outliers may contain some useful
  information!!

21
2. Distribution

• Xnx1, x2, , xn Fn
• T(Xn)xi xi2 Ln, Un, 1 i n Ftn
• W(Xn)Ln(xiltLn)xi(Ln xi Un)Un(xigtUn), 1 i
  n
• 
  Fwn
• Fn F
• Ftn Ft
• Fwn Fw

22

• 2.Distribution (cont.)

F(x)Fw(x)? Ft(x) x2 L, U
Fw
Ft
b1
F
U
L
23
?3
Ft
F
Fw
U
L
24
Robustness

• Highest possible breakdown point (0lt?lt1).
• 2. Influence function

Cauchy ?2
25
Influence Function of winsorized mean
26
(No Transcript)
27
AREs of Trimmed and Winsorized Mean relative to
Mean
28
GES(M)supxIF(x, M(F))
29
Simulation
30
Scale setting

• Overview
• High breakdown scales
  
• Why trimmed/winsorized scales
  
• Scaled-deviation trimmed/winsorized scales
  
  

31
Overview on Measures of Scale

• standard deviation
• range
• average absolute deviation
• interquartile range
• trimmed standard deviations
• (Welsh and Morrison (1990))

32
Breakdown point for Scale
33
High Breakdown Scales

• Median Absolute Deviation (MAD)
• MADncm medi(xi-medj(xj))
• Rousseeuw and Croux (1993)
• 1. csmedi(medj(xi-xj))
• 2. Qncqxi-xj iltj(k) where k ,
  hn/21

34
Why scaled deviation trimmed/winsorized scale?

• Light-tailed distribution
• Situations when contaminated points presented
  around the center.

Normal distribution
35
Scaled-deviation Trimmed/Winsorized Scales
36
(No Transcript)
37
Influence Function
normal distribution ?3
IF(x, Sw(F))
IF(x, S(F))
38
Asymptotic representation and limiting
distribution
? 0.
39
(No Transcript)
40
Efficiency
AREs of S and Sw relative to the standard
deviation
compared with the inverse of fisher
information 2.
41
(No Transcript)
42
(No Transcript)
43
Simulation
44
(No Transcript)
45
Open problems

• Regression setting
• Confidence Interval based on T
• Hypothesis testing based on T

46

• Dec 20, 2005
• 889
• July 15, 2003

Lucky, Lucky forever!
47
Acknowledgements

• My supervisor Yijun Zuo.
• My guidance committee Dr. Page, Dr. Salehi, and
  Dr. Yang.
• Professors and Friends at stt dept.
• Statistics department.

48
Thank you!