The Good Shepherd

1
The Good Shepherd

• Sample Design and Control for
• Wealth Measurement in the
• Survey of Consumer Finances
• Arthur B. Kennickell
• Federal Reserve Board

2
Thanks!

• Martin Frankel, Stephen Heeringa, Roderick
  Little, Donald Rubin, Fritz Scheuren, Louise
  Woodburn
• FRB Brian Bucks, Gerhard Fries, Kevin Moore,
  Michael Neal, Brian Presolpsky
• SOI Barry Johnson
• NORC Leslie Athey, SCF field staff
• SCF respondents
• Opinions are those of the author alone and not
  necessarily those of the FRB.

3
Sampling

• Surveys are measurement engines
• Sample is the statistical foundation
• (Neyman, 1934 essential reading)
• Tool for guiding operations
• Key to post-survey adjustments
• (Ignoring interview content/administration)

4
Frequentist/Model-Based Sampling

• Replicability and randomness are key
• Bias means systematic error that persists across
  replications of a sampling process
• NOT that we get the right answer every time
• Efficiency characterizes how far from true value
  a given sample est. might tend to be

5
Sampling Error for Mean(y)
6
1 Y1000, 99 Y1 Error
7
1 Y1000, 99 Y1 Error
8
Limit potential variability

• Stratification
• Finding an adequate basis for stratification
• Particularly important with wealth data
• SOI data for the SCF area-probability sample

9
Frame Problems

• Income, not wealth (whatever that is!)
• Idiosyncratic rates of return
• Legal and illegal avoidance
• Taxpayers, not households
• Changes in household structure
• Use models to translate into an appropriate index

10
SCF Wealth Index Model
11
Why Index Might be Inefficient

• Inappropriate rates of return
• Bad model estimates
• Income fluctuations
• Changes in labor force participation
• Movements in financial markets
• Changes in tax laws
• Convenience

12
Variation in income over time
13
Net worth by WINDEXM2001
14
Sample Not what one gets

• Locating/contacting issues
• Refusals
• Differential application of effort

15
Sample Management
16
Effects of Systematic Management

• Break endogeneity between effort and nonresponse
• Ability to estimate nonreseponse models
  reflecting respondent behavior
• More ethical treatment of respondents equal
  opportunity

17
Post-Survey Adjustments

• Use sample/external information
• Helpful if sample identifies key dimensions of
  nonresponse

18
The Future

• Given data (easy!), straightforward to
  design/improve unbiased/efficient sample
• Nonresponse is likely to be a permanent problem
• Administrative (register) data may help to some
  degree, but frameworks differ
• Additional work needed on management of sample
  and nonresponse