Prehospital Outcomes Research

1
Prehospital Outcomes Research

• Establishing the Scope and Methodological Approach

2
PREHOSPITAL CARE

• Increasing scrutiny
• Questioning the value
• Interventions
• Transport
• Persistent concern--Lack of proof
• Effectiveness
• Cost-effectiveness
• Methodologically sound Outcomes Research long
  overdue

3
EMS Outcomes Research and Classical Health
Research

• Four classical health research disciplines
• Basic Research
• Clinical Research
• Epidemiological Research
• Health Systems/Services Research
• Recent development
• Outcomes and Effectiveness Research (OR/OER)
• Understanding the development of OR

4
THE DEVELOPMENT OF OUTCOMES RESEARCH

• Convergence of multiple forces
• 70s/80s Dramatic healthcare inflation
• Overwhelming cost ? political urgency
• Huge variation in cost
• NOT correlated with outcomes
• More/more expensive Little association with
  improvement

5
THE CONTINENTAL SHIFT

• Overwhelming cost
• Societal/corporate outcry
• Political urgency
• Growing awareness
• Non-association between cost/outcome

6
MID-80s A MASSIVE CHANGE

• Approach to Hospital Reimbursement
• Prospective Payment System
• Diagnosis Related Groups (DRGs)
• Not reimbursed based upon care
• Lump sum Diagnosis

7
COINCIDENTAL RESEARCH POLITICS

• Traditional approach to medical research
• Great strides in EFFICACY research
• Often did NOT translate ? Practice
• Real world outcomes not changed
• Outcry for a new discipline
• Able to identify EFFECTIVENESS

8
ANOTHER COINCIDENCE

• Societal/political concern
• DRGs ? Quicker but sicker
• D/C based upon days, not clinical status
• Congressional hearings
• Unintended consequences of incentives

9
WE WANT RESEARCH!!!

• Politicians wanted a new research discipline
• Methods of correlating
• Variations in practice
• Cost
• Outcome
• Why?
• To protect citizens against under-treatment
• To measure what they got for the federal dollar

10
AHEAD OF THEIR TIME

• YET ANOTHER COINCIDENCE
• 70s Pioneering work in Outcomes Research
• Concept
• Correlate CARE with outcomes
• Correlate COST with outcomes
• Approach
• Mining large databases
• Status
• Meaningful but obscure

11
A MATCH MADE IN HEAVEN

• 1986 HCFAHave we got data!!!
• Mortality
• Readmission rates
• Adverse outcomes
• Result
• Weve got the datayouve got the method
• YeeHaa!!!

12
THE REAL HISTORY OF OER

• The Outcomes MOVEMENT
• Societal/political/regulatory/financial urgency
• Outcomes RESEARCH
• A discipline conceived before the federal
  juggernaut
• Result
• Political tidal wave swept OR to the forefront
• Federal agenda to fund the new discipline

13
DHHS-1987

• Federally sponsored meetings
• Agenda Can OR use Medicare Databases?
• Variations in care ? Outcomes
• Quality monitoring
• Quality Improvement
• Help identify wasteful therapies/procedures
• Improve health with decreased cost

14
THE RESULT

• Major federal initiative to find what works
• 1989 Agency for Health Care Policy and Research
  (AHCPR)

15
CONCEPTUAL FOUNDATION FOR OR

• Dramatic assumption
• Guidance for optimal medical practice can be
  gleaned from analysis of data routinely gathered
  in the process of delivering and paying for
  patient care.
• (AHCPR Publication No. 99-R044)

16
EARLY YEARS OF OR

• Narrow definition for OR
• Mining large administrative databases
• Data from routine patient care
• Emphasis on
• Identifying cost
• Identifying effective care
• Meta-analysis
• Pooling studies to make a larger study

17
RESPONSE OF TRADITIONAL RESEARCHERS

• ABSOLUTELY
• BONKERS!!!

18
YEAHRIGHT!!!

• The assumption
• Identifying what impacts patient outcomes can be
  determined by retrospectively looking at large
  patient populations in administratively developed
  databases
• The response Heated debate
• NIH vs. AHCPR
• Philosophy
• Funding
• Politicians vs. politicians
• Researchers vs. researchers (L. Pauling)

19
THE MATURING OF OR

• The early years Mining and Meta-analysis
• Now
• No single definition accepted
• Quality analysis/Quality Improvement becoming
  prominent
• Emphasis on using multiple methodologies
• Emphasis on measuring MANY outcomes
• The Six Ds
• Agency for Healthcare Research and Quality
• Codifying terminology, methodology

20
LEVELS OF IMPACT OF RESEARCH RESULTS

• Level 1 impact Research findings that do not
  lead to a direct change in policy or practice.
• New research tools (severity indicators)
• New methodologies (Episode of Care Model)
• Instruments assisting clinical decision-making
• Identifying important outcomes questions
• Identifying changes needed in current practice

21
LEVELS OF IMPACT OF RESEARCH RESULTS

• Level 2 impact Research results that are
  translated directly into policy or program
  changes/development
• Legislation
• Bureaucracy
• Healthcare payment
• Healthcare planning
• Clinical guidelines (e.g. ACEP)

22
LEVELS OF IMPACT OF RESEARCH RESULTS

• Level 3 impact Research that leads to an actual
  alteration in clinical care provided.
• Findings leading to treatment changes
• Findings leading to alterations in patient
  behavior
• Level 4 impact Research that leads to actual
  improvement in patient outcomes
• Basic Research ? Efficacy ? Improved outcome
  in the real world

23
AHCPR/AHRC REPORT CARD

• The first decade
• Vast majority of OR Only level 1 impact

24
APPROACH TO EMS OR

• Still proponents Narrow definition
• EMSOPs opinion Wont work for EMS OR
• EMSOPs approach
• Integration among ALL appropriate research
  disciplines

25
EMSOPs DEFINITON OF OR

• OER evaluates the impact of healthcare (including
  discrete interventions such as particular drugs,
  medical devices, and procedures, as well as
  broader programmatic or system interventions) on
  the outcomes of patients and populations. OER
  may include evaluation of economic impact linked
  to outcomes. OER emphasizes evaluations of care
  delivered in general, real-world settings
  multi-disciplinary teams and a wide range of
  outcomes. OER may entail any in a range of
  primary data collection methods and secondary (or
  synthetic) methods that combine data from
  primary studies. Mendelson-1998

26
INTEGRATION IN FUTURE EMS RESEACH

• Utilize the strengths of classical OR
• Mining databases
• Meta-analysis
• Integrate with
• Traditional clinical research methods
• Epidemiology
• Systems research

27
A BALANCED APPROACH TO EMS OR

• The Outcomes Approach
• Inexpensive source of knowledge about
  effectiveness
• Utilizing data collected routinely in EMS systems
• Using pooled information from published studies
• However, there is a paucity of
• Rigorous systems research
• Clinical trials
• ESPECIALLY prospective, controlled trials

28
ENAMORED WITH RCTs

• Oversimplification to emphasize
• Strengths of RCTs
• Weaknesses of OR
• Why?
• Uncommon RCTs ? Clinical practice
• Very rare RCTs ? Clinical practice ? Improved
  outcome proven
• Very common
• Efficacious interventions never make it into
  practice
• Efficacious interventions ? Proven INEFFECTIVE
  (Defibrillation in NY City)

29
PREHOSPITAL SETTING

• Great risk of efficacious interventions being
  ineffective
• Even if funding available for RCTs, studies of
  effectiveness must occur in varied sizes and
  types of EMS systems.
• EMSOP urges a balanced view
• Recognize strengths and limitations of each
  methodology
• Must accelerate knowledge transfer to the field

30
USING OR METHODS IN EMS RESEARCH

• In one sensea perfect match
• EMS The epitome of potential efficacy/effectiven
  ess mismatch
• Cant assume ANYTHING
• Sooutcomes from real world practice MUST be
  studied
• Many large databases exist in many types and
  sizes of systems
• But

31
WARNING 1

• Current databases grossly inadequate
• Vestigial
• Locally developed
• No consistent terminology
• No consistent definitions
• No national data
• Incomplete
• Inaccurate

32
If were going MINING

• Challenge Robust, usable databases
• Long history ? Little progress
• Lacking
• Accuracy
• Precision
• Comprehensiveness
• Validated Risk Adjustment Measures
• Completeness
• BP ? Visual Analog Scale for SOB???
• Mandatory Innovative methods for obtaining RA
  data (automated?)

33
WARNING 2

• Poorly done database mining is a DISASTER
• Has led to MANY wrong conclusions
• MANY complaints by traditional disciplines are
  TRUE
• Remember the limitations
• Mining databases can identify associations
• It is very efficient for hypothesis generation
• It cannot PROVE cause and effect
• A TRAGIC example

34
LONG HAIR PROTECTS FROM FATAL INJURY

• Ron Maio, MD
• University of Michigan
• Herb Garrison, MD
• East Carolina University
• Journal of Irreproducible Results

35
LONG HAIR PROTECTS FROM FATAL INJURY

• Hypothesis A large database can be used to
  prove that increased hair length reduces injury
  severity and mortality.
• Participants Patients entered into the trauma
  registries of two university-based Level I Trauma
  Centers
• Interventions None.

36
METHODS

• Prospective evaluation of the impact of hair
  length on injury severity and survival
• Inclusion Mechanistic or physiologic criteria
  for Level I Trauma (ACS)
• Exclusion Age lt10 or bizarre hair cut
• IRB Dis-approval was obtained

37
METHODS

• Measurement Mean hair length
• Maio-Garrison Method
• Ten lengths of hair
• Randomly yanked from the scalp
• Hairs without an intact follicle excluded
• Bad samples Keep yanking
• Risk Adjuster Injury Severity Score
• Final Outcome measure Mortality

38
RESULTS

• Mean hair length 18.27 centimeters.
• Unexplained Distribution Bimodal
• Attempted to correlate hair length to
• Ethnicity
• Income
• Ethanol levels
• Hair color
• Pre-event Quality of Life
• No correlations

39
RESULTS

• 500 patients entered
• Short-hairs
• Long-hairs
• Mean ISS similar 9.7 vs 9.3 (p .73)
• Survival similar 94.3 vs 95.6 (p .89)

40
MAJOR TRAUMA RISK

• 350 (70) were short hairs
• Risk of Short-hairs and Long-hairs being involved
  in major trauma
• National Association of Hair Salon Owners
• Study areas Percentage of Short-hairs and
  Long-hairs in the general population nearly
  identical.
• Odds Ratio of Short-hairs suffering a major
  traumatic event 2.33

41
CONCLUSIONS

• When involved in injury events, long-hairs and
  short-hairs have similar severity of injury and
  mortality
• Being short-haired is an incredibly high risk
  factor for being involved in major trauma (OR
  2.33).

42
RECOMMENDATIONS

• Educational programs beginning in preschool to
  teach children the dangers of having short hair.
• Immediate post-injury education to discourage
  future hair cuts and thereby decrease recidivism
• Immediate legal sanctions against all hair
  cutting establishments and manufacturers of hair
  cutting instruments.

43
RECOMMENDATIONS

• A sin tax for getting a hair cut.
• Immediate establishment of an NIH study section
  to identify interventions that increase the rate
  of hair growth (e.g. minoxidil)
• NHTSA should promulgate regulations ensuring that
  these hair growth factors are placed in all
  public water supplies

44
RECOMMENDATIONS

• Passage of laws making it illegal to operate
  motor vehicles, motor cycles or bicycles with
  short hair
• Passage of laws making it illegal to wear bicycle
  or motorcycle helmets
• Law enforcement officers
• Short-hairs vs.
• Long-hairs (shovers)

45
ASSOCIATION DOES NOT IMPLY CAUSE

• What did they miss?
• The association between GENDER and HAIR LENGTH
• Embellishedbut not much
• There is EVER the tendency to use associations to
  prove causality
• This has happened in REAL publications

46
MINING DATABASES Beware of Selection Bias

• Analysis of Suicide rates in a county in the
  southern U.S.
• Study county High percentage of Catholics in
  population
• Control county High percentage of Protestants
• Suicide rate 3X higher in Catholic county
• Conclusion Being Catholic leads to higher
  likelihood of suicide

47
SUBSEQUENT REANALYSIS

• What was the problem with the conclusion?
• No one actually identified the religion of
  suicide victims
• 70 of all suicides Jumping from a high bridge
• The bridge was in the Catholic county
• Nearly half of the jumpers were residents of the
  Protestant county

48
WARNING 3

• Learn from the history of health research
• Large chasms between the disciplines
• Journals, conferences, associations sequestered
  and non-interactive
• Minimal crossover
• Terminology
• Expertise
• Information
• Findings

49
THE COST OF RESEARCH SILOS

• Few researchers able to identify the best
  methodology for answering many questions
• Methodologic Tunnel Vision
• Only use methods from the researchers own domain
• Problem
• Method from another domain may be far superior
  for a given question

50
INTEGRATED APPROACH TO EMS RESEARCH

• All EMS care is rendered within the framework of
  a complex, interactive system
• THEREFORE
• Outcomes Research, Systems Research, Basic
  Research, Clinical Research, and
  Epidemiologywill always be interdependenteach
  being informed by the other

51
(No Transcript)
52
EMSOPS APPROACH TO EMS RESEARCH

• Identifying effectiveness of an intervention
• Requirements
• Meaningful Conceptual Framework
• Approach that leads to broadly applicable
  conclusions
• Robust and pertinent specific methodology

53
EXAMPLE OF INTEGRATION

• A way to study effectiveness in EMS
• Conceptual Framework Episode of Care Model
• Origin Outcomes Research
• Assuring broad applicability Multiple System
  Approach ? Numerous system types and sizes
• Origin Systems Research
• Specific Methodology RCT
• Origin Classical Clinical Research

54
The Episode of Care Model
Identifying the impact from each "unit of
service"
EmergentSubspecialtyCare
PrehospitalCare
EDCare
InpatientCare
Follow-upCare
RA T OUT
RA T OUT
RA T OUT
RA T OUT
RA T OUT
Long-term outcomes
Precipitating Event
RA Risk Adjustment Measures TTherapeutic
Intervention(s) OUTOutcome Measure(s)
55
IMPLICATIONS OF THE INTEGRATED APPROACH

• 1) No discipline owns a methodology
• RCT Can identify efficacy AND effectiveness
• New asthma drug EFFICACIOUS in a pulmonary clinic
• Drug EFFECTIVE in numerous types of EMS systems
• Before-After Controlled Trial
• Traditional Clinical Research Each INDIVIDUAL
  is their own control for testing an intervention
• Systems Research A group of SYSTEMS being
  studied serve as their own control for testing an
  intervention

56
Prospective, Before-after System Trial
Phase I
Phase I(R)
Phase II
(Baseline)
(Run-in)
(ExperimentalPhase)
Prospectiverisk adjustmentand
outcomemeasurements
Begin trainingfor newintervention
Interventiongiven toall appropriatepatients
Analysis
57
EXAMPLE OPALS

• Prospective 20 EMS systems
• Population Patients presenting with SOB
• Phase I BLS only
• Phase II Systems add ALS
• Intubation
• Medications
• Results Decreased MORTALITY
• Implications--Second EMS condition to be PROVEN
  effective with
• Sound methodological approach
• Inclusion of MANY system types and sizes
• Broad applicability

58
IMPLICATIONS OF THE INTEGRATED APPROACH

• 2) Various methods can cross the Six Ds
• --Example
• --A drug found to be effective at decreasing
  morbidity in the prehospital environment
• --Economic analysis Not COST- effective in
  small EMS systems

59
CHALLENGES FOR THE FUTURE

• Get serious about GOOD EMS databases
• Methodology
• Terminology
• Definitions
• Completeness of data
• Quality of data
• Develop and validate RA measures
• Available from many types/sizes of system
• Implied
• Well-funded career researchers
• NATIONAL database

60
CHALLENGES FOR THE FUTURE

• Prevent the errors of the past
• Understand the limitations of OR
• Association/Cause
• Selection Bias
• GOOD Risk Adjustment
• No more research SILOS
• Researchers well versed across disciplines
• Sacrifice TURF for TRUTH

61
CHALLENGES FOR THE FUTURE

• Recognize the tendency to stagnate at level 1
  research
• Level 1 work is mandatory, but not sufficient
• Dont be impressed until OUTCOMES are affected

62
CHALLENGES FOR THE FUTURE

• Be TIRELESS in pursuing funding
• Science gets a freebie (PCN) about once aCENTURY
• Level 4 OR in EMS will NOT occur without
  continuous, major funding
• The history of MAJOR research fundingTwo
  prerequisites
• An important health/societal issue
• TIRELESS lobbying by stakeholders

63
CHALLENGES FOR THE FUTURE (THE KEY)

• Rejection of the Mount Everest approach to
  prehospital care
• Why do we do what we do????
• No more acceptance of interventions just because
  they work in rats
• Why is this so important?
• If EMS PROFESSIONALS dont REQUIRE good
  researchno one will PAY for good research

64
QUESTIONS

• ??????