Title: A Latent Class Callback Model for Survey Nonresponse
1A Latent Class Call-back Model for Survey
Nonresponse
- Paul P. Biemer
- RTI International and UNC-CH
- Michael W. Link
- Centers for Disease Control and Prevention
2Outline
- Motivation for the study
- Early cooperator effects (ECE) in the Behavior
Risk Factor Surveillance System (BRFSS) - Call-back models
- Manifest and latent
- Model extensions
- Application to the BRFSS
- Results
- Summary and conclusions
3Terms and definitions
- Cooperators units that will eventually respond
at some request or call-back - Non-cooperators (also called hardcore
nonrespondents) units that will not respond to
any call-back
4Terms and definitions (contd)
- Early cooperator Cooperators that respond at
early calls (say, 5 or less) - Later cooperators Cooperators that respond at
later calls (say, 6 or more) - Early cooperator effect (ECE) expected
difference in estimates based on early vs. early
later cooperators (say, )
5Response rates as a function of number of call
attempts
Number of call attempts
6Illustration 1- Have you ever been told by a
doctor, nurse or other health professional that
you had asthma?
Small ECE maximum of 5 calls is adequate
7Illustration 2- During the past 12 months, have
you had a flu shot?
Larger ECE max of 5 call attempts may be
biasing Could consider other definitions of
early cooperator.
8Why study ECE?
- Effort (and costs) could be saved if ECE is small
- If ECE is not small, adjustments may be applied
to reduce it - May need to adjust for HCNRs, not only later
cooperators
9What adjustments can be applied to reduce the ECE?
- Nonresponse adjustments
- Requires characteristics of nonrespondents
- Lack of information a limitation for some surveys
- Post-stratification adjustments
- Requires known target population totals within
adjustment cells - Variables limited to those available externally
- Call-back model adjustments
- Assumes response propensity is function of level
of effort required to obtain a response and
grouping variables - Related work of Drew and Fuller (1980), Politz
and Simmons (1949), others
10ECE in the BRFSS
- Survey details
- One of the largest RDD surveys in the world
- Estimates the prevalence of risk behaviors and
preventive health practices - Monthly, state-based, cross-sectional survey
- Target population is adults in telephone hhs
- Data source 2004 survey with 300,000
interviews
11ECE in the BRFSS (contd)
- Early cooperator defined as responding with 5
fewer call attempts - Examined differences in
- demographic characteristics
- 10 selected health characteristics overall and by
demographic domain - ECE estimated by
- Data weighted by base weights only
12Typical Values of ECE
13Typical Values of ECE (contd)
14Summary of the Results
- Early cooperators are different from later
cooperators on many dimensions - For most characteristics ECE is relatively small
- Less than 3 percentage points at aggregate level
- Rarely more than 3 points for domains
- For some characteristics, ECE may be important
- Other definitions of ECE also considered
15Hardcore Nonresponse Bias
- Hardcore Nonrespondents Units that will not
respond under the current survey protocol no
matter the number of call-backs - ECE does not include the bias due to hardcore
nonrespondents - Total nonresponse bias Bias due to cooperators
who did not respond bias due to hardcore
nonrespondents - Adjusting for ECE may not remove bias due to HCNR
16Call-back Models for Adjusting for ECE and HCNR
Bias
- General idea
- Estimate the response propensity for subgroups of
the population - Response propensity is modeled as a function
level of effort (LOE) to obtain a response - Two models are considered
- Manifest model (MM) Ignores HCNR
- Latent class model (LCM) Includes HCNR
- Includes a latent indicator variable to represent
the HCNRs in the population - Why latent?
17Illustration for 5 Call-backs
1 interview 2 noninterview 3 noncontact
18Illustration for 5 Call-backs
1 interview 2 noninterview 3 noncontact
19Potential Advantages over Post-Stratification
- Post-stratification adjustments (PSAs) depend
upon the availability of external benchmarks or
auxiliary data - Selection of control variables is quite limited
- Target populations also quite limited
- Adjust for ignorable nonresponse only
20Potential Advantages over Post-Stratification
- Call-back model can rely only on internal
variables - Weighting classes can be defined for any
variables collected in the survey - Can be applied for any target population
- Greater ability to selected variables that are
highly correlated with response propensity - Adjust for ignorable and nonignorable
nonresponse
21Modeling Framework
- Simple random sampling
- Survey eligibility is known for all sample
members - No right censoring
- (i.e., all noncontacts received maximum LOE)
- Extensions to relax these assumptions are
described in the paper
22Incorporating the Model-based Weights
- Unadjusted estimator of the mean
Based on the sample distribution
Adjusted estimator of the mean
Estimated from the model
23Two Models for Estimating
- MM (Manifest Model)
- Assumes all nonrespondents would eventually
respond at some LOE (i.e., all nonrespondents
have a positive probability of response) - LCM (Latent class model)
- Incorporates 0 probability of response for the
hardcore nonrespondents (HNCRs)
24Technical Details
25Notation
Levels of effort (LOE)
Outcome of LOE l where 1interview, 2
noninterview, 3noncontact
LOE associated with state S1 or 2
Grouping variable (weighting class variable)
26Notation
Probability person in group g is interviewed at
LOE l
Probability person in group g is noninterviewed
at LOE l
Probability person in group g is never contacted
Number of sample persons in group g interviewed
at LOE l
Number of sample persons noninterviewed at LOE l
Number of sample persons never contacted after L
(max LOE) attempts
27General Idea Outcome Patterns for 5 Call-backs
- Cooperator HCNR
- 11111 0
- 31111 0
- 33111 0
- 33311 0
- 33331 0
- 22222
- 32222
- 33222
- 33322
- 33332
- 33333
28Likelihood for the Manifest Model
- This model is appropriate when
- Every sample member has a positive probability of
responding at some LOE, or - Adjustment for ECE only is desired
29Likelihood for the Latent Class Model
Introduces a latent variable X where X 1, if
HCNR and X 2, if otherwise Appropriate when
some sample members have a 0 probability of
responding and adjustment for total nonresponse
(Later Cooperators HCNRs) is desired
30Model Restrictions
The key assumptions regard the form of the
probabilities
, and
where
and
31Model Restrictions (contd)
Manifest Model
(i)
, say, for l 3,...,L for Llt10
and (ii)
i.e., no HCNR,s and response probabilities are
equal for LOE 3,..., 9. The probabilities for
LOEs 1, 2 and 10 (if present in the model) were
not restricted
Latent Class Model
to represent the HCNR group in the model.
32Results
33Four Estimators were Considered
- Unadjusted estimator
- Estimator using MM estimates of
- Estimator using LCM estimates of
- Estimator using CPS estimates of
- i.e., usual PSA estimator
- treated as the gold standard
34Comparison of the ECE for a Maximum Five
Callbacks Strategy Before and After MM Adjustment
35Differences between PSA and Unadjusted and
Adjusted Estimates for a Maximum Five Callbacks
36Estimating the Potential Bias Reduction
- BRFSS data do not exhibit very large nonresponse
biases - Therefore, consider a variable, Y, that has
maximum nonresponse bias given the BRFSS
nonresponse rates - To do this, we form
- Yg BRFSS response rate for group g
- Compute the relative difference between
unadjusted and adjusted estimates and the PSA
estimate of the mean of Y
37A Measure of Potential Bias Reduction
- Define the relative difference operator for any
maximum LOE, L as
Gold standard estimator
38Absolute Relative Differences (RDL) for
Unadjusted and Adjusted Estimators as a Function
of Number of Call-backs
39Conclusions
- ECE for 5 call-backs is generally small, but can
be moderately high for some characteristics - The Manifest Model can be employed to reduce ECE
- The Latent Class Model can be employed to reduce
total nonresponse bias (Later Cooperators HCNR
bias) - Future research should focus on
- Variable selection
- Comparisons of MSEs of the estimators
- Small/medium size sample properties
- Integration with other post-survey weight
adjustments