Measuring the Quality of Health Care Providers

1
Measuring the Quality of Health Care Providers

• Alexander C. Tsai
• HSR Symposium Measuring and Improving the
  Quality of Health Care Services
• March 30, 2004

2
A Motivating Example

• Yij ßjXij ?ij
• Y 1 if patient i died in hospital j
• X hospital indicators

3
Importance of Risk Adjustment in Assessing Quality

• In 1986, HCFA released mortality figures on 5,750
  Medicare provider hospitals
• Average mortality rate 22.5
• 142 had death rates significantly higher than
  average
• One hospital had mortality rate 87.6

4
Importance of Risk Adjustment in Assessing Quality

• In 1986, HCFA released mortality figures on 5,750
  Medicare provider hospitals
• Average mortality rate 22.5
• 142 had death rates significantly higher than
  average
• One hospital had mortality rate 87.6
• Turned out to be a hospice

5
A Motivating Example

• Yij ?Rij ßjXij ?ij
• Y 1 if patient i died in hospital j
• X hospital indicators
• R case mix variables from administrative data
  (R1) and clinical data (R2)
• R R1 R2

6
Does X Cause Y?
ß
Hospital
Outcome
Y ?R ßX ?
X should not be correlated with ?
7
Omitted Variable Bias

• R Entire universe of case mix variables
• R R1 R2 R3
• R1 Fitting a risk adjustment model to
  administrative data yields c0.60
• R2 Clinical data is better (c0.85) but much
  more expensive
• You want R, but you only have R1

8
Does X Cause Y?
ß
Hospital
Outcome
Y ?R ßX ?
X should not be correlated with ?
9
Omitted Variable Bias
R2
ß
Hospital
Outcome
Y ?1R1 ßX ?
? ? ?R2
10
Solution 1 Better Data
ß
Hospital
Outcome
Y ?1R1 ?2R2 ßX ?
11
Solution 1 Better Data
ß
Hospital
Outcome
Y ?1R1 ?2R2 ßX ?
(but what about R3?)
12
Solution 2 Better MethodsRandomized, Controlled
Trial
Coin Toss
ß
Hospital
Outcome
Y ßX ?
13
Solution 3 Better MethodsInstrumental Variables
(IV)
Distance (IV)
ß
Hospital
Outcome
Y ?1R1 ßX ?
X dD ?
14
Solution 3 Better MethodsInstrumental Variables
(IV)
Distance (IV)
ß
Hospital
Outcome
Y ?1R1 ßX ?
X dD ?
15
Other Examples
16
Conclusions

• Measuring quality is a perilous undertaking
• Must have good risk adjustment
• Be sure that you have R and not just R1
• Regression analysis requires certain assumptions
• If you cant get better data, then you must
  invoke stronger assumptions
• Instrumental variables is one possibility

17
Bibliography

• McClellan, McNeil, Newhouse. Does more intensive
  treatment of AMI in the elderly reduce mortality?
  Journal of the American Medical Association
  272859-66.
• Zohoori, Savitz. Econometric approaches to
  epidemiologic data. Annals of Epidemiology
  7251-7.
• Gowrisankaran, Town. Estimating the quality of
  care in hospitals using instrumental variables.
  Journal of Health Economics 18747-67.
• McClellan, Newhouse. Overview of the special
  supplement issue. Health Services Research
  351061-9.
• Angrist, Kreuger. Instrumental variables and the
  search for identifcation. Journal of Economic
  Perspectives 1569-85.