Folie 1

1
Evidence-based clinical practice Christian
Gluud The Cochrane Hepato-Biliary
Group Copenhagen Trial Unit Centre for
Clinical Intervention Research
Rigshospitalet Copenhagen University Hospital
2
Evidence-basedclinical practice

• The patients values
• (concerns, expectations, preferences)
• The best clinical experience (N 1 trials)
• The best clinical research evidence

3
The hierarchy of evidence

• Ia Systematic review of randomised clinical
• trials with low risk of bias taking
• random error risk into consideration
• Ib Single randomised clinical trial
• with low risk of bias and of random error
• II Cohort study
• III Case-control study
• IV Consensus reports, opinion of
• experts, narrative overviews,
• physiological studies

4
Important aspects of clinical research

• Systematic errors (bias)
• Random errors (play of chance)
• Design errors

5
Low risk of systematic error
High risk of systematic error
Low risk of random error
High risk of random error
6
Important aspects of clinical research

• INTERNAL VALIDITY
• Systematic errors (bias)
• Random errors (play of chance)
• EXTERNAL VALIDITY
• Design errors

7
The hierarchy of evidence

• Ia Systematic review of randomised clinical
• trials with low risk of bias taking
• random error risk into consideration
• Ib Single randomised clinical trial
• with low risk of bias and of random error
• II Cohort study
• III Case-control study
• IV Consensus reports, opinion of
• experts, narrative overviews,
• physiological studies

8

• Dangers in non-randomised studies
• Biological mechanisms
• - Limited time of diseases
• - Cyclical progression of diseases
• - When do we see patients?
• Psychological mechanism
• - The Rosenthal effect, we see what we want
• to see (BIAS)!
• - The Barnum effect, we believe what we want
• to believe (astrology)!
• Confounding by indication

9
Important aspects of randomised clinical trials

• Systematic errors (bias)
• Random errors (play of chance)
• Design errors

10
Low risk of systematic error
High risk of systematic error
Low risk of random error
High risk of random error
11
Meta-analysis of several trials
Low risk of bias High risk of bias Overall
12
Ratio of odds ratios Odds ratio of trials with
unclear or inadequate component divided by odds
ratio of trials with adequate component
13
Components associated with bias risk

• Generation of the allocation sequence
• Allocation concealment
• Double blinding
• Intention-to-treat analysis
• Outcome measure reporting bias
• Other components associated with bias (important
  baseline differences early stopping academic
  bias industry bias etc)

14
Control of selection bias

• Generation of the allocation sequence
• Adequate
• Computer system, table of random
• numbers, or similar
• Inadequate
• Not described or quasi-randomised
• (excluded)

15
Generation of the allocation sequence
16
Proportion of randomised trials in
GASTRENTEROLOGY with adequate generation of
allocation sequence (Kjærgard et al. 2002)
17
Control of selection bias

• Allocation concealment
• Adequate
• Central independent unit, sealed
• envelopes
• Inadequate
• Not described or open table of random
• numbers

18
Allocation concealment
19
Proportion of randomised trials in
GASTRENTEROLOGY with adequate allocation
concealment (Kjærgard et al. 2002)
20
Control of detection bias(reporting bias and
observer bias)

• Double blinding
• Adequate
• Identical placebo or comparator
• Inadequate
• Not described or not blinded

21
Double blinding and risk of bias
22
Proportion of randomised trials in
GASTRENTEROLOGY with adequate double blinding
(Kjærgard et al. 2002)
23
Important aspects of randomised clinical trials

• Systematic errors (bias)
• Random errors (play of chance)
• Design errors

24
Mean number of patients per intervention arm
(SEM) in 383 randomised trials published in
Gastroenterology from 1964-2000 (Kjærgard et al.
2002)
25
Low risk of systematic error
High risk of systematic error
Low risk of random error
High risk of random error
26
Random errors in small trials

• False positive results
• (type I error)
• False negative results
• (type II error)

27
(No Transcript)
28

• Group sequential analysis

29
Group sequential boundaries- similar group sizes
30
Group sequential boundaries- similar group sizes
31
Lan-DeMets group sequential boundary
- test when you want!
32
External validity

• The extent to which the results of a
• clinical trial can be applied to similar
• patients - the generalisability
• You get what you examine

33
(No Transcript)
34
CONSORT FLOW FIGURE
35
External validity

• External validity depends on factors
• that determine internal validity and
• external validity
• External validity first becomes of
• interest when we are sure that there
• are no problems with internal validity

36
External validity
Without internal validity, there is no interest
in external validity!
37
External validity
So, start with internal validity!
38
Important aspects of meta-analysis of randomised
clinical trials

• Systematic errors (bias)
• Random errors (play of chance)
• Design errors

39
Important aspects of of meta-analysis of
randomised clinical trials

• INTERNAL VALIDITY
• Systematic errors (bias)
• Random errors (play of chance)
• EXTERNAL VALIDITY
• Design errors

40
Meta-analysis - increases power and precision
META Means after in Greek Metascience
Science about a science Pratt JG, Rhine JB,
Smith BM, Stuart CE, Greenwood JA Extra-sensory
perception after sixty years a critical
appraisal of the research in extra-sensory
perception New York Henry Holt 1940
41
Meta-analysis - increases power and precision
Glass 1976 A meta-analysis is a statistical
analysis of a collection of results in order to
integrate the results. Meta-analysis The
scientific method which critically evaluates and
statistically combines scientific results.
42
Low risk of random error and low risk of
systematic error - as close to the truth as one
can get
Precision (Sample size or inverse variance or
standard error )
1.0
Relative risk
43
Low risk of random error, but high risk of
systematic error
Precision (Sample size or inverse variance or
standard error )
1.0
0.8
Relative risk
44
High risk of random error and low risk of
systematic error
Precision (Sample size or inverse variance or
standard error )
1.0
Relative risk
45
High risk of random error and high risk of
systematic error
Precision (Sample size or inverse variance or
standard error )
1.0
0.8
Relative risk
46
(No Transcript)
47
The pernicious yin-yang interplay between random
errors and systematic error
PUBLICATION BIAS
48
Unfair test - randomised clinical trial
Control intervention
Experimental intervention - random errors (PUB
bias) - systematic errors (bias)
why most research findings are
false! JP Ioannidis
49
THANK YOU !
50
Funnel plot asymmetry
51
(No Transcript)
52
Jules Gavaret 1840

53
Jules Gavaret 1840
The rules of logic are inadequate for
judging the influence of a given medication in
an equally given disease

54
Jules Gavaret 1840
The principles of the law of large numbers are
strictly applicable to therapeutic research and
they alone can furnish the solution of the
important problems

55
Jules Gavaret 1840
To be able to decide in favour of one treatment
method over another, it is not enough for the
method to yield better results the difference
found must also exceed a certain limit, the
extent of which is a function of the number of
observations

56
Povl Heiberg 1897

Bibliotek for Læger 189771- 40
57
Povl Heiberg 1868 -1963

58
Povl Heiberg 1897

• Heiberg reviews the history of evidense-based
  medicine
• Heiberg knows the risk of random errors (play
  of chance)
• Heiberg knows the risk of systematic errors
  (bias)

59
Povl Heiberg 1897

• Heiberg knows the solutions
• - large numbers !
• - randomisation !
• - blinding !
• - independent research !

60
(No Transcript)
61
External validity

• This is what we normally do!
• More easy to understand

62
Design errors affecting external validity of
randomised trials include wrong

• Centres
• Participants
• Experimental intervention
• Control intervention
• Goal - explanatory or pragmatic
• Trial structure - parallel group, crossover,
  etc
• Objective - superiority, equivalence,
  non-inferiority
• Outcome
• Unit of analysis

63
Design errors - wrong centres (1)

• Wrong investigators
• - want the money
• - do not want to do the job
• Wrong patients
• - they disappear
• - they do not fulfill in- and exclusions
• criteria

64
Design errors - wrong centres (2)

• Single-centre trial
• higher internal validity
• lower external validity
• Multi-centre trial
• lower internal validity
• higher external validity

65
Design errors - wrong centres (3)

• Use tertiary centre for intervention aimed at
• primary care
• Use centre in high-income country for use
• in low-income country, or vice versa

66
Exclusion of participants (1)
Van Spall, JAMA 2007 283 randomised trials
published in high impact journals between 1994
and 2006
67
(No Transcript)
68
Exclusion of participants (2)

• Exclusion of common medical conditions, 81
• Exclusion due to age, 72
• Exclusion of common medications, 54
• Drug trials more likely to exclude
• Industry-sponsored trials more likely to
  exclude

69
Exclusion of participants (3)

• Mean number of exclusion criteria 10
• Justification weak, 53
• Unjustified in 84 of trials

70
Narrow inclusion participants (4)

• Increase precision
• Increase adherence
• Increase compliance
• Reduce external validity
• Delay progress for most patients

71
Wrong experimental intervention (1)

• Too high a dose
• Too low a dose
• Too few dose finding studies
• Interaction with common medications

72
Wrong control intervention (1)

• Too high a dose
• Too low a dose
• Wrong route of administration
• Not the best comparator
• Too often use of placebo

73
Wrong goal (1)

• Explanatory trial (early) - efficacy trial
• Pragmatic trial (late) - effectiveness trial

74
Wrong trial structure (1)

• N 1 participant
• Parallel group
• Cross-over
• Factorial trial
• Cluster

75
Wrong trial structure (1)

• N 1 participant
• Parallel group
• Cross-over
• Factorial trial
• Cluster

76
Wrong trial structure (1)

• N 1 participant
• Parallel group
• Cross-over
• Factorial trial
• Cluster