CLINICAL TRIALS POTPOURRI

1
CLINICAL TRIALS POTPOURRI

• WHY TRIALS--WHY NOT DATA BASES??
• TRIAL DESIGN
• RANDOMIZATION
• PROPER CONTROLS
• INTENTION TO TREAT-EXCLUSIONS
• ADHERENCE, WITHDRAWALS, LTF
• METANALYSES-BIG AND SIMPLE TRIALS

2
LIMITATIONS OF INFORMATION FROM DATABASES - No. 1

• Prospective approach not used.
• Randomization not used.
• Selection Bias should be assumed
• regardless of direction of treatment effect.
• -Regimen depends on
• patient presentation
• severity of disease
• -Hence, treated and controls systematically differ

3
LIMITATIONS OF INFORMATION FROM DATABASES - No. 2

• Statistical adjustment not adequate.
• Covariate adjustment -- a shaky solution
• 1. Unknown covariates effect outcome
• 2. Relevant covariates not available
• 3. Incomplete collection covariate data
• 4. Covariate errors compromise adjustment
• 5. Mismodeling -- never really known
• Time Zero not identified.
• All patients or random sample.
• Inadequate test of new therapy.

4
PREREQUISITES SUCCESSFUL CLINICAL TRIAL

• Trial Needs to be Done
• Question Posed Appropriate/Unambiguous
• Trial Architecture is Valid
• Inclusion/Exclusion balance Efficiency/Generaliza
  bility
• Trial Protocol is Feasible
• Trail Administration is Effective
• Sackett 1983.

5
KEY POINT OF ALL EXPERIMENTATIONPROPER CONTROL

• Group against which intervention group is
  compared
• (placebo or standard of care)
• How to identify an appropriate control.
• How to identify and maintain control group.
• Can be an extremely difficult problem.
• Serious consequences if not done.

6

• Daniel resolved not to defile himself with royal
  food and wine and proposed the following
• FIRST CLINICAL TRIAL

Please test your servants for ten days give us
nothing but vegetables to eat and water to drink.
Then compare our appearance with that of the
young men who eat the royal food and treat your
servants in accordance with what you see. So he
agreed to this and tested them for ten days. At
the end of the 10 days they looked healthier and
better nourished than any of the young men who
ate the royal food. So the guard took away their
choice of food and wine the wine they were to
drink and gave them vegetables instead. Daniel
112-16
7
FUNDAMENTAL POINT-FFD

• The Randomization Process
• Randomization tends to produce study groups
  comparable with respect to known and unknown risk
  factors, removes investigator bias in the
  allocation of participants and guarantees that
  statistical tests will have valid significance
  levels
• Trialists most powerful weapon against bias

8

• RANDOMNESS IN EXPERIMENTATION-
• FIRST DONE IN 1920s
• Sir Ronald Alymer Fisher - COIN FLIP

• 3 Benefits
• 1) Treatment groups should be comparable for both
• known and unknown variables.
• 2)p-values may be computed without assumptions.
• -Intrinsic to randomization process-biologically
  plausible.
• -Not dependent on statistical model - not a
  mathematical trick.
• 3) Result - randomized clinical trial most
  cogent and
• convincing scientific proof of therapeutic
  efficacy.

9
EMPIRICAL EVIDENCE OF BIAS

• Objective to determine if inadequate approaches
  to randomized control design and execution are
  associate with evidence of bias in estimating
  treatment effects.
• Design 250 controlled trials from 33
  meta-analyses
• (Cochrane Collaboration)
• Outcomes inadequate allocation, inadequate
  sequence generation, exclusions after
  randomizations, lack of double blind design
• KF Schulz and colleagues JAMA 1995273408-412

10
ODDS RATIOS UNCLEARLY AND INADEQUATELY CONCEALED
TRIALS VS ADEQATELY CONCEALED TRIALS

• Concealment Odds Ratio
• allocation (95 CI) X2(df)p
• Adequate 1.00 (referent)
• Unclear 0.67 (0.60-0.75)
• 57.9 (2) lt.001
• Inadequate 0.59 (0.48-0.73)

11
ASSOCIATION METHODOLOGICALQUALITY AND TREATMENT
EFFECTS

• Measure of Odds Ratio
• Methods Quality (95 CI) X2(df) p
• Allocation Concealment
• Adequate 1.00 (ref)
• Inadequate 0.70 (0.62-0.70) 32.9 (1)
  lt0.001
• Sequence Generation
• Adequate 1.00 (ref)
• Inadequate 0.95 (0.81-1.12) 0.31 (1) 0.58
• Exclusions
• No 1.00 (ref)
• Yes 1.07 (0.94-1.21) 0.99 (1) 0.32
• Double Blinded
• No 1.00 (ref)
• Yes 0.83 (ref) 6.16 (1) 0.01

12
APPROPRIATE CONTROLS (unbiased sample)-HARD TO
GET

• What is the appropriate control group?
• Comparison groups similar as possible
• Example Select a number 0 - 9.

13
RESULTS OF SELECTION NUMBERS 0-9

• 0 and 9 usually under-represented.
• The extremes are not as often selected.
• Seeming preference for odd numbers.
• 3, 5, 7 will be more often selected.

14
RANDOMIZATIONTHE ONLY WAY?

• PHYSICIAN AND PARTICIPANT PREFERENCE
• Comprehensive cohort design -participant given
  preference of treatment
• Wennberg Design-Randomized to groups, first give
  choice, second randomized.
• Rucker Design-Given preference the first
  assignment, no strong preference randomized
• ONLY SMALL EVIDENCE OF BIAS-PREFERENCE
  
• LIMITATIONS-7/32 2 STAGE PREFERENCE DESIGN,
  CONFOUNDING BIG ISSUE
• FALSE ASSUMPTION-Large numbers refuse
  randomization, limits generalizability

JAMA 20052931089-1099
15
INTENTION TO TREAT - ONCE IN ALWAYS COUNTED -
No. 1

• Issue Avoid Bias
• Fundamental Point FFD
• Excluding randomize subjects from analysis and
  sub-grouping on the basis of outcome or response
  variables can lead to biased results. This bias
  can be of unknown magnitude or direction.

16
INTENTION TO TREAT - ONCE IN ALWAYS COUNTED -
No. 2

• Preserves the benefits of randomization by
  including all randomized patients based on their
  original allocation.
• Safeguards Against
• Erroneous claims of efficacy by exclusion of
  those who do not adhere to protocol.

17
EXCLUSION FROM TRIALS

• Before Randomization
• Rationale Eligibility criteria not met
• No bias in comparison of group results
• Affects Generalizability of results
• Follow-up of those excluded?

18
EXCLUSION FROM TRIALS-2

• Burden of Proof Comparability on investigators
• Management versus Explanatory Trials
• Optimize management vs mimic practice
• Seven day rule vs all events
• Hopelessly Tangled Violates fundamental point.
• 
  Sackett and colleagues, NEJM 1979 3011410-1412

19
EXCLUSION OF SOME EVENT S

• Prevention Trials - Immediate Events
• - Understand intervention?
• - Adverse effects may occur earlier
• Present both analyses
• Really a withdrawal trial. Do Not Do This!

20
WITHDRAWALS

• INELIGIBILITY
• - Clerical error - Canadian Cooperative Trial
• - Definition- Surgical vs Medical Therapy in
  Bilateral Carotid Stenosis
• - Laboratory error - BHAT
• - Misinterpretation - MILIS
• - Mis-classification - ANTURANE

21
HIPPA

• HEALTH INSURANCE PRIVACY AND PORTABILITY ACT

Cl Trials 2008
22
PARTICIPANT WITHDRAWAL

• POSSIBLE MAJOR THREAT TO ITT
• MUST ANTICIPATE
• IN CONSENT-MANDITORY LAST CONTACT
• IRB POSITION-PRIVACY PROTECTION
• CLINICAL CENTER STAFF-ACTIVE ROLE
• ESPECIALLY PI

Cl Trials 2008
23
TRIAL RESULTS AND EXTERNAL VALIDITY

• TRIAL RESULTS VS RESULTS IN COMMUNITY
• OBSERVATIONAL RESULTS MATCH TRIALS
• AMOUNT OF BIAS UNKNOWN IN OBS STUDIES
• ISSUES AFFECT EXTERNAL VALIDITY
• SETTING OF TRIAL
• SELECTION OF PARTICIPANTS
• CHARACTERISTICS OF PARTICIPANTS
• TRIAL PROTOCOL VS COMMUNITY PRACTICE
• OUTCOME MEASURES AND FOLLOWUP
• ADVERSE EFFECTS OF TREATMENT.

Lancet 200536582-93
24
TREATMENT EFFECTS MODEST

• Limits our ability to identify true benefit.
• Crucial to maintain trial integrity
• Benefits not limited to sickest patients
• Broad enrollment crucial
• Consistent across subpopulations
• Under enrollment of some still common

Circulation 20021061015-1021
25
MORTALITY STUDY GROUP AND ELIGIBILITY
STATUS ANTURANE (Sulfinpyrazone) RE-INFARCTION
TRIAL

• a-priori definition-interpreted differently
• Randomized Ineligible Eligibles
• Anturane 813 (9.1) 38 (26.3) 775 (8.3)
• Placebo 816 (10.9) 33 (12.1) 783 (10.9)
• p in all 0.20 p in eligibles0.07
• Reanalysis 19 not 14 deaths found in ineligibles
• Original 10 versus 4 Reanalysis 12 versus 7

26
SURGICAL VERSUS MEDICAL THERAPY BILATERAL
CAROTID STENOSIS - (1)

• Outcomes Among Available Participants-
• Therapy Recurrent TIA Total
• Stroke or Death Participants
• Yes No
• Surgery 43 38 81
• Medical 53 19 72
• RR 0.73, p0.02

27
SURGICAL VERSUS MEDICAL THERAPY BILATERAL
CAROTID STENOSIS - (2)

• Among All Participants Randomized
• Therapy Recurrent TIA Total
• Stroke or Death Participants
• Yes No
• Surgery 58 36 94
• Medical 54 19 73
• RR 0.84, p0.09
• def. alive and stroke free after
  hospitalization

28
STUDY POLICY ON WITHDRAWALS IF DONE ---

• Methods stated a-priori
• Blinded
• Early in the study
• Minimize Number
• May still be challenged.

29
3 DESIGN POLICIES THAT RELATE TO WITHDRAWALS

• Enroll only those with secure diagnosis.
• Enroll confirmed and unconfirmed and later
  withdraw those with diagnostic error.
• Enroll confirmed and unconfirmed but allow no
  withdrawals.

30
NON-ADHERENCE

• Drop Outs and Drop Ins
• Many Reasons
• Both Groups Contaminated
• Reduces Power
• Intervention - Adherence Interaction
• CDP
• AMIS
• Adjustment Does Not Compensate
• Compensate by Increasing Sample Size

31
POOR QUALITY OR MISSING DATA - No. 1

• Lost to follow-up - endpoint ascertainment
• Complete or near complete ascertainment
• is possible on clinical events
• CDP - 4
• LRC - 0
• PHS - 0
• SHEP - 6

32
SYST - EURA WORST CASE ANALYSIS?
33
POOR QUALITY OR MISSING DATA - No. 2

• Other response variables
• Imputation- 20 increase in efficacy
• Survival analysis - time to event
• Removal of outliers - similar results?
• Worst case analysis

34
COMPETING EVENTS

• Choose endpoint wisely.
• Cause-specific versus total mortality.
• Other deaths treated as lost to follow-up.
• When reporting include total mortality,
  cause-specific mortality and morbid events.

35
EFFICACY ANALYSIS (ON TREATMENT)

• Expected in pharmaceutical industry
• Precisely describe - denominator not same
• True estimate benefit between two analyses
• Should be reported without p-value
• Crucial in non-inferiority trials

36
WITHDRAWAL AND BIAS
37
HOW DO WE USE META-ANALYSIS, SMALL TO MODERATE
SIZED TRIALS AND LARGE SIMPLE TRIALS

• Sequence of Investigations
• Meta-analysis - 1991Moderate Size Trial -
  1992Mega Trial - 1995
• Large Trial-2002
• Subject - Effect of magnesium on mortality in
  immediate post infarction.

38
INITIAL META-ANALYSIS - MAGNESIUM AND DEATH RATE
POST MI

• 7 trials 1301 participants
• 657 (25) magnesium
• 644 (53) controls
• 55 reduction in mortality
• 95 CI 0.28-0.71, plt0.001
• Biologically plausible result
• Ventricular arrhythmia reduced 7 versus 109
• Adverse effects rare

39

• 7 TRIALSS CHARACTERISTICS
• All randomized.
• 6 blinded.
• Baseline characteristics balanced.
• Treatment usually started within 12 hours.
• 99.4 follow-up for mortality (8 patients).
• Dosage varied 30-90 mmols.
• Infusion over 24-48 hours.
• - Some bolus injections.
• Baseline and follow-up Mg levels similar.
• 1 year mortality 20 vs 32 from 2 studies.

40
MAGNESIUM TRIAL META-ANALYSIS
41
MAGNESIUM TRIAL META-ANALYSIS
42

• Leicester Intravenous Magnesium Intervention
  Trial LIMIT-2
• 2316 patients with suspected acute MI
• Blinded placebo controlled
• 8 mmol over 5 minutes 65 mmol over 24 hrs
• Primary outcome - total mortality _at_ 28 days
• 99.3 ascertainment
• 24 reduction (95 CI 0.57 - 0.99, 2p 0.04)
• 25 reduction in left ventricular failure (95
  CI 0.61 - 0.91, 2p 0.009)
• 65 confirmation of MI in both groups

43
Randomization of Patients Admitted to
CCU(September, 1987 to February, 1992)
44
LIMIT- 2
45
MAGNESIUM TRIAL META-ANALYSIS
46

• ISIS 4 INTERNATIONAL STUDY OF INFARCT SURVIVAL
• 58,050 Participants
• Entry - up to 24 hours after onset of chest pain.
• No contraindications to other therapies
• 2x2x2 factorial design
• Treatments vs. Placebo
• - 1 month, up to 100 mg/d captopril
• - 1 month, controlled-release nitrate 60 mg/d
• - 24 hours magnesium.
• 8 mmols bolus
• 72 mmols infusion

47

• ISIS 4 INTERNATIONAL STUDY OF INFARCT SURVIVAL
• Primary endpoint all arms total mortality
• Results
• 7 reduction captopril
• No change mononitrate
• No change magnesium
• No significant interactions between treatments

48
ISIS 4 CLINICAL EVENTS IN HOSPITALS(up to day
35 or earlier discharge)
2plt0.01 2plt0.001 2plt0.0001
49
MORTALITY IN DAYS 0-35 SUBDIVIDED BY OTHER
RANDOMLY ALLOCATED STUDY TREATMENTS
50
SYSTEMATIC OVERVIEW OF EFFECTS ON SHORT-TERM
MORTALITY OF STARTING INTRAVENOUS MAGNESIUM EARLY
IN ACUTE MYOCARDIAL INFARCTION
51

• WHAT WENT WRONG?
• WHAT SHOULD WE BELIEVE?
• Increase in deaths ISIS-4
• (p 0.07, 95 CI 12-0)
• No convergence or divergence - 1 year
• 23,000 given bolus within 6 hours7.9 magnesium,
  7.6 control
• 17,000 no fibrinolytic therapy - no change
• ISIS an open trial - no apparent problems
• Previous meta-analysis and clinical trial - small
  numbers
• 99 CI in LIMIT-2 - no benefit

52

• LIMITATION OF META-ANALYSIS
• Numbers small.
• Number of outcomes (deaths) small.
• Selection of arrhythmias - problematic.
• Potentially biased patient selection.
• Duration of follow
• - Generally covered hospital stay.
• - Limited long-term.

53
MAGIC(MAGNESIUM IN CORONARIES)

• Design and Context
• Randomised/ Double Blind
• 6213 participants-Acute STEMI
• Magnesium Sulfate/Placebo
• 2g IV bolus (15min) 17g infusion (24hr)
• 30 day all cause mortality
• Strata 65 and older (rep)/any age (no rep)
• Lancet 20023601189-1196

54
MAGIC(MAGNESIUM IN CORONARIES)

• Mg P
• ACM (prim) 475 472
• Stratum 1 161 175
• Stratum 2 311 300
• MI (Yes/No) 131/344 136/336
• Reper (Y/N) 131/344 125/347
• lt65/gt65 74/237 67/233
• New onset HF 597 562
• Lancet 20023601189-1196

55

• META-ANALYSIS
• A systematic way of combining data to get a more
  precise estimate of the effect of a therapy.
• Positives
• - Combine all available data.
• - Larger numbers of events available.
• - Estimate of therapeutic benefits possible.
• Negatives
• - Loss of equipoise.
• - Outcome numbers may be small.
• - Uncritical examination.
• Importance
• - Sample size estimates.
• - FDA submissions.
• - Medical policy formulation.

56

• CUMULATIVE META-ANALYSIS
• Lau et al NEM 1992327 248-254
• The Performance Of Updating A Meta-Analysis
  Every Time A New Trial Appears
• Goal Evaluating the results as a continuum.
• Outcome Supply practitioners with up-to-date
  information
• Methods Fixed effects model
• ( Mantel-Haenszel Statistic)
• _ Random effects model
• (DerSimonian-Laird Statistic)
• Methods Evaluation Little difference in results
• Recommendation Use both methods

57
THROMBOLYTIC TRIALS
58
(No Transcript)
59
(No Transcript)
60
(No Transcript)
61
Attitudes About Intravenous Fibrinolytic Therapy
Among Those Consultants Who Rarely or Never
Used It (59 in 1987 and 31 in 1989)
62
LARGE, SIMPLE TRIALS

• YUSUF, COLLINS AND PETO
• Ask an important question, answer it reliably
• Statistics in Medicine 19843409-420

63
LARGE, SIMPLE TRIALS

• Identification of effective treatments
• Disease common
• Treatment widely practicable
• Concentrate on major vs minor outcomes
• Stratification does not improve
• No need to subcategorize participants
• If no reliable answer yet-effect moderate
• Statistics in Medicine 19843409-420

64
EXPECTED EFFECTS OF TRIAL SIZE ON TRIAL RESULTS
65
Effects of trial size on trial results.
Relationship between the total number of deaths
in the two treatment groups and the result
actually attained