Methodological Issues in Surveys of

1
Methodological Issues in Surveys of

• Consumer Satisfaction with
• Health Insurance Plans

2
by Donna J. Brogan

• Biostatistics Dept.
• Emory University
• Atlanta, GA
• September, 1999

3
Outline

• Introduction
• Current Survey Methodology
• Problems with Current Methods
• Suggestions for Improvement
• Discussion and Conclusions

4
Introduction

• U.S.--Many Health Insurance Plans
• Managed Care Plans--Dominant
• Access to Insurance -- Employer
• Subpopulations
• Medicare, Medicaid, Children

5
Introduction-2

• Evaluative data on plans
• HEDIS (Health Plan Employer Data
  Information Set)
• Accreditation, Marketing, Auditing
• Consumer/Client Satisfaction with Plan
• Annual Satisfaction Surveys

6
Current Survey Methodology

• Contract with Vendor
• Data Collection Instrument
• Get needed care
• Get care quickly
• Good doctor communication
• Courteous/helpful office staff
• Customer service
• Claims processing

7
Current Methodology-2

• Sampling Frame Construction
• Members vs. subscribers
• Deletions from list
• Sampling Technique--SRS
• Find Subject
• Home address/telephone
  

8
Current Methodology-3

• Field Work
• Mail QX
• Phone interview
• Combination of mail/phone
• Sample Size 300 completed QXs
• Response Rate Guideline gt 60
• RR ( completes) / ( eligible)

9
Current Methodology-4

• Data Analysis (as SRS)
• Manuals developed by research firms with
  government support, e.g. CAHPS
• Expansion of methodology
• Disenrollees--why they left plan
• Health/Functional Status--of members

10
Methodological Problems

• Deviation from Equal Probability Sampling AND No
  Weighting of Data
• random sample specified
• some frame members excluded to avoid selecting 2
  or more from same household
• adults covering only themselves children with
  no siblings-- higher selection probability

11
Methodological Problems-2

• Many in Sample Not Located/Contacted
• incorrect home phone number and/or address
• Some in Sample no Longer Plan Members
• Survey Response Rate Misleading
• denominator those who agree to provide info
• Subpopulations Not Represented Well

12
Suggestions for Improvement

• Remove Nonmembers from Frame
• Consider Various Sampling Plans
• Cluster (subscriber unit) sampling
• Equal probability or pps for cluster
  
• Choose 1 person per sample cluster
• Oversample Subpopulations of Interest

13
Suggestions-2

• Improve Locate/Contact Procedures
• Use Stage-Specific Response Rates
• Locate response rate
• Contact response rate
• Consent response rate
• Interview/QX response rate
• Survey Response Rate--Product of 4 Above

14
Suggestions-3

• Weighting Procedures--Unit Nonresponse
• Post-stratify to Population
• Perform Weighted Analyses
• Sample not likely to be equal probability
• Nonresponse/poststratification adjustments
• Analytical Analyses as Well as Descriptive

15
Discussion

• Suggestions--Standard Survey Procedures
• Surveys Not Using These Procedures
• Consultants--for Vendors and Plans
• Administrative Databases for Plans
• Accurate contact information
• Timely information on disenrollees
• Information for Oversampling

16
Conclusion

• Surveys Used for Important Decisions
• Accreditation
• Internal auditing or monitoring
• Marketing
• Information to General Public
• Need Better Quality Surveys on Satisfaction