EBM for Novice

1
EBM for Novice

• Maria Kwok, MD, MPH
• Assistant Clinical Professor
• Section of Pediatric Emergency Medicine
• Morgan Stanley Childrens Hospital of New York
  Presbyterian

2
Outline

• Why is EBM important?
• What is EBM
• Understand foreground vs. background questions
• Formulate PICO questions
• Find resources, do searches
• Validity of an article
• Appraise Therapy articles
• Appraise Diagnostic articles

3
Why is EBM Important?

• MEDLINE
• 400,000 new entries added each year
• To keep up-to-date
• Need to read 6,000 articles each day

4
Why is EBM Important?

• Lag time from time of knowing to time of
  implementation
• 13 years for thrombolytic therapy
• 10 years for corticosteroids for acceleration of
  fetal lung maturity
• (Antman EM, JAMA, 1992)

5
Traditional Approaches

• Self-reported reading time per week.
  (University setting)
• Medical students 60 min.
• Interns none
• Senior residents 10 min.
• Fellows 45 min.
• Attendings graduating
• Post 1975 60 min.
• Pre 1975 30 min.

6
There is simply no way we can keep up to date in
medicine using traditional approaches!
7
Outline

• Why is EBM important?
• What is EBM?
• Understand foreground vs. background questions
• Formulate PICO questions
• Find resources, do searches
• Validity of an article
• Appraise Therapy articles
• Appraise Diagnostic articles

8
Quiz Define EBM
A) A means of camouflaging biostatistics in
medical training
B) An annoying and overused catch phrase
C) The conscientious, explicit, and judicious use
of current best evidence in making decisions
about the care of individual patients
9
Research Evidence
Decision Making
Patient Preference
Clinical Expertise
10
Evidence Based Medicine A Process
Patients
Ask Questions
Searching
Appraise
11
Validity Results Applicability
12
Outline

• Why is EBM important?
• What is EBM
• Understand foreground vs. background questions
• Formulate PICO questions
• Find resources, do searches
• Validity of an article
• Appraise Therapy articles
• Appraise Diagnostic articles

13
Whats the difference between foreground and
background questions?
14
Asking the Precise Question

• Background questions
• Basic aspect of a disease
• Pathophysiology
• Etiology
• Basic treatment
• Who, what, when, how

• Foreground questions
• Specific knowledge
• Have 4 parts

• Patient/problem
• Intervention
• Comparison intervention
• Clinical outcomes

15
Background or Foreground?

• What is asthma?
• Is prednisone helpful in asthma?
• What are the newest medication for asthma?
• Does atrovent used acutely make you feel better?

16
Background or Foreground?

• What is asthma? (B)
• What are the newest medication for asthma? (B)
• Does atrovent used acutely make you feel better?
  (F)
• Is prednisone helpful in asthma? (F or B)
• Foreground if compare to other drugs
• Background if interested in how it works

17
Outline

• Why is EBM important?
• What is EBM
• Understand foreground vs. background questions
• Formulate PICO questions
• Find resources, do searches
• Validity of an article
• Appraise Therapy articles
• Appraise Diagnostic articles

18
Structure of a Well-built Question

• Patient or population
• Be specific to capture the group you want
• Ex Children w/ asthma
• Intervention
• Be specific
• Comparison group (if any)
• Compare to standard therapy or test
• Outcome
• Be precise
• What are the outcome of interest

19
Formulating the Clinical Question
PICO
20
Can You Identify PICO?

• In children under 6 months, how does sleeping on
  back compared to sleeping on the stomach in terms
  of risk of SIDS?

• In children under 6 months (P), how does sleeping
  on back (I) compared to sleeping on the stomach
  (C) in terms of risk of SIDS (O)?

21
Can You Form a PICO Question?
Clinical scenario 5 yo with moderate
persistent asthma now in severe acute asthma
exacerbation. Intern gave 2 albuterol and
orapred with minimal improvement. Intern asks
why how good is atrovent?
22
Searchable PICO Question
P Population I Intervention/diagnostic
test/risk factor C Comparison O Outcome
In children with acute asthma exacerbation (P),
will the addition of atrovent (I) to albuterol
(C) decrease the rate of hospitalization (O)?
23
Outline

• Why is EBM important?
• What is EBM
• Understand foreground vs. background questions
• Formulate PICO questions
• Find resources, do searches
• Validity of an article
• Appraise Therapy articles
• Appraise Diagnostic articles

24
Searching Superhero
Best Bets
25
The Evidence Pyramid

26
(No Transcript)
27
Pre-appraised Resources

• Cochrane
• http//www.cochrane.org/cochrane/revabstr/maininde
  x.htm
• Very high quality reviews
• Mostly questions of therapy
• National Guideline Clearinghouse
• http//www.guideline.gov/
• Guidelines of varying levels of quality
• Do broad searches
• PEM database
• http//researchinpem.homestead.com/homepage.html
• Not pre-appraised

28
Pre-appraised Resources

• Best Evidence Topics
• http//www.bestbets.org
• Developed in the ED of Manchester Royal Infirmary
  in UK
• Usually EM topics
• Free
• Clinical Evidence (CE Concise)
• http//www.clinicalevidence.com
• From UK
• Focus mostly on therapy
• Free

29
Primary Search Engines

• Pubmed
• Ovid

30
Primary Search Engines

• Both PubMed and Ovid
• Contains MEDLINE and other bibliographic
  databases (Cochrane, EMBASE, etc)
• Sells access to the databases to libraries

31
PubMed MEDLINE vs. Ovid MEDLINE

• Same DATABASE
• Different SEARCH ENGINES
• Searching differently may get you slightly
  different results
• If you search both databases EXACTLY the same,
  your results will be the same

32
What is MEDLINE?

• MEDLINE is a DATABASE
• MEDLINE is produced by the National Library of
  Medicine
• MEDLINE is available free via Pubmed
• MEDLINE is sold to many vendors, like Ovid, who
  search the DATA in MEDLINE through different
  search engines

33
Database Producer
Vendors
Medline EMBASE CINAHL
Ovid Pubmed Dialog LEXIS/NEXIS
Personal PC Library PC
34
PubMed Clinical Queries

• Pre-filtered searching
• Search on questions of
• Therapy
• Diagnosis
• Etiology
• Prognosis
• Or, search for Systematic Reviews

Limit any of the categories to sensitivity
(broad) or specificity (narrow)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
Sensitivity vs. Specificity

• SPECIFICITY
• Narrower search
• More relevant items
• Possibility of losing some useful citations
• LESS information to weed through

• SENSITIVITY
• Broader search
• More irrelevant items
• Less chance of losing useful citations
• MORE information to weed through

39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
OVID

• Most librarians favor this search engine
• Allow you to tailor your search

44
(No Transcript)
45
Design Execute Search Strategy

• MeSH terms
• How articles are indexed
• Predefined categories from the National Library
  of Medicine
• Explode
• MeSH all items underneath
• Focus
• MeSH only

46
Design Execute Search Strategy

• Limiters
• Limit by languages, human, age, etc
• Hedges
• Clinical trials for therapy
• Sensitivity Specificity for diagnostic
  tests
• Cohort studies for prognosis

47
More on Searching Tips

• Term
• Search engines will look for occurrences of the
  word with any combination of letters following.
  (Ex Hospital will include hospitalization,
  hospitals, hospitalized)
• Exp for explode
• Term.tw.
• Allow search engine to look in the titles and
  abstract

48
More on Searching Tips

• Term.af
• Search all fields
• ..pg term
• Perge/delete

49
For More Searching Tips
Ask your librarians!
50
Outline

• Why is EBM important?
• What is EBM
• Understand foreground vs. background questions
• Formulate PICO questions
• Find resources, do searches
• Validity of an article
• Appraise Therapy articles
• Appraise Diagnostic articles

51
Appraise (Validity)

• http//www.cebm.utoronto.ca/teach/materials/dx.htm
  
• http//www.cebm.utoronto.ca/teach/materials/therap
  y.htm

52
Validity of Therapy Articles

• Randomized?
• Blinded?
• Were the subjects similar at the start of the
  trial?
• Were all subjects treated the same except for the
  treatment of interest?
• Were all patients accounted for at the end of the
  trial?

53
Validity 1. Were the Subjects Randomized?
Methods section
54
Validity 2. Were They Blinded?
Methods section
55
Validity 3. Were the Subjects Similar at the
Start of the Trial?
Results section
56
Validity 4. Were All Subjects Treated the
Same except for the Treatment of Interest?

• Methods section

57
Validity 5. Were All Patients Accounted for
at the End of the Trial?
Results section
58
Validity of the Article

• Was the study randomized?
• Were the subjects blinded?
• Were the subjects similar at the start of the
  trial?
• Were all subjects treated the same except for the
  treatment of interest?
• Were all patients accounted for at the end of the
  trial?

59
Outline

• Why is EBM important?
• What is EBM
• Understand foreground vs. background questions
• Formulate PICO questions
• Find resources, do searches
• Validity of an article
• Appraise Therapy articles
• Appraise Diagnostic articles

60
Step 4 Appraise Therapy Study

• Therapy study
• Number Needed to Treat (NNT) to prevent an
  adverse outcome
• Mathematically
• NNT 1/ARR (absolute risk reduction)
• ARR Risk of adverse outcome in the control
  group (CER) those in the experimental group
  (EER)
• ARR CER EER
• ARR Absolute risk reduction
• CER control event rate
• EER experimental event rate

61
More on NNT

• Variables must be dichotomous
• Cannot calculate NNT from continuous variables
• For NNT to be meaningful, the difference between
  the 2 groups must be significant

62
Lets Calculate NNT
30
3
0.5
200
5
20
CER Control Event Rate EER Experimental Event
Rate ARR Absolute Risk Reduction NNT Number
Needed to Treat NNT1/ARR
63
Step 4 Appraise Therapy Articles

• ARR CER EER
• NNT 1/ARR
• ARR 52.6-37.5 15.1
• NNT 1/15.1 x 100 7

64
Step 5 Decision Making

• Atrovent
• Low cost medication
• Minimal to no side effects
• Every 7 children with severe asthma exacerbation,
  1 hospitalization will be prevented
• The benefits outweigh the risks. Therefore it
  would be worth treating the child with Atrovent
  in the emergency department.

65
Outline

• Why is EBM important?
• What is EBM
• Understand foreground vs. background questions
• Formulate PICO questions
• Find resources, do searches
• Validity of an article
• Appraise Therapy articles
• Appraise Diagnostic articles

66
Appraise (Validity)

• http//www.cebm.utoronto.ca/teach/materials/dx.htm
  
• http//www.cebm.utoronto.ca/teach/materials/therap
  y.htm

67
Validity of Diagnostic Articles

• Was there an independent, blind comparison with a
  reference (gold) standard of diagnosis?
• Was the diagnostic test evaluated in an
  appropriate spectrum of patients (like those in
  whom it would be used in practice)?

68
Validity of Diagnostic Articles

• Was the reference standard applied regardless of
  the diagnostic test result?
• Was the test (or cluster of tests) validated in a
  second, independent group of patients?

69
A B
C D
A C
B D
70
Sensitivity Specificity

• Sensitivity

The proportion of people with the target disorder
(AC) who have a positive test (A) A/(AC)

• SnNout

If a test has a high sensitivity, a negative
result rules out the disease
71
A B
C D
A C
B D
72
Sensitivity Specificity

• Specificity

The proportion of people without the disease
(BD) who have a negative test (D) D/(BD)

• SpPin

When a test has a high specificity, a positive
result rules IN the diagnosis
73
A B
C D
A C
B D
74
Predictive Values

• Positive Predictive Value

Proportion of people with a positive test (AB)
who have the target disorder A/(AB)

• Negative Predictive Value

Proportion of people with a negative test (CD)
who are free of the disease (D) D/(CD)
75
A B
C D
A C
B D
76
What is Likelihood Ratio?

• A likelihood that a given test result would be
  expected in a patient w/ the target disorder
  compared w/ the likelihood that the same result
  would be expected in a patient w/o the target
  disorder

77
Likelihood Ratio

• Likelihood ratio (LR) of 1 means
• The test result identifies patients w/ the
  disorder at the same rate as identifying patients
  w/o the disorders
• LR gt1
• The test result identifies patients w/ the
  disorder at a faster rate likelihood goes ?
• LR lt1
• The test result identifies patients w/ the
  disorder at a slower rate likelihood goes ?

78
LR of 3 means
The result is 3 times more likely to occur in a
patient w/ the target disease than in a patient
without.
In general, the LR for a test result is the
percentage of disease patients identified by
that result divided by the percentage of disease
patients so identified.
79
(a/X) (b/Y)
a/X
b/Y
(c/X) (d/Y)
c/X
d/Y
What rate are Dis patients identified by Result
A? What rate are Dis - patients identified by
Result A? What is the LR for Result A?
80
X A C
Y B D
At what rate are Dis patients identified by a
test ?
A/X
At what rate are Dis patients identified by a
test?
B/Y
What is the LR for a positive test result?
(A/X) (B/Y)
81
Positive Likelihood Ratio (LR )

• LR sensitivity/(1 specificity)
• True pos rate/False pos rate

82
X A C
Y B D
At what rate are Dis patients identified by a
- test ?
C/X
At what rate are Dis patients identified by a
- test?
D/Y
What is the LR for a negative test result?
(C/X) (D/Y)
83
Negative Likelihood Ratio (LR-)

• LR - (1 sensitivity)/specificity
• False neg rate/True neg rate

84
Guide to Likelihood ratios
85
Why Do We Care About LR?

• Allow us to think about pre-test and post-test
  probabilities so that we can personalize the test
  to our patients

86
Test Treatment Thresholds in the Diagnostic
Process
Test threshold
Treatment threshold
0
100
Testing completed treatment starts
No test needed
Further testing needed
87
Step 1 Pre-test Probability

• Prevalence

The number of events (instances of a given
disease or other condition) in a given population
at a designated time.
It is not a rate.
It is used as pre-test probability
It is the number of disease patients to the
total
88
Pre-test Probability

• 5 sources to determine this
• Clinical experience
• Regional or national prevalence statistics
• Practice databases
• Original report on accuracy of the test
• Studies that calculate pre-test probability

89
Step 3 Calculate LR

• Pretest probability is 30
• LR ?

Amir, 1994
Rapid Strep
LR TP rate/FP rate (65/75) ? (18/90) 4.35
LR - FN rate/TN rate (10/75) ? (72/90) 0.16
90
Fagan Nomogram Pre-test probability on the left
hand column LR in the middle Post-test
probability on the right hand column
91
Translation

• You see a 4 yo child who comes in c/o sorethroat,
  no other complaints
• Your pre-test probability is that she has a 30
  chance of having strep throat
• You wonder how good is rapid strep
• You look up an article (Amir, 1994) and find its
  sensitivity (86.5) and specificity (80)

92
Translation

• Your patient rapid strep is positive
• You calculated the LR to be 4.35
• You determined the post-test probability to be
  66
• Is that enough of a threshold for you to treat or
  would you rather do more test?

93
Test Treatment Thresholds in the Diagnostic
Process
Test threshold
Treatment threshold
0
100
Testing completed treatment starts
No test needed
Further testing needed
94
A Different Article Showed

• What if the LR is 12?
• What is the post-test probability?

95
(No Transcript)
96
Diagnostic Summary

• Use the 2 x 2 table to determine likelihood ratio
  (LR)
• LR True pos rate/False pos rate
• LR - False neg rate/True neg rate
• Use Fagan Nomogram

97
Outline

• Why is EBM important?
• What is EBM
• Understand foreground vs. background questions
• Formulate PICO questions
• Find resources, do searches
• Validity of an article
• Appraise Therapy articles
• Appraise Diagnostic articles

98
Knowledge is of two kinds we know a subject
ourselves or we know where we can find
information upon it. - Samuel Johnson