When is Small Beautiful?

1
When is Small Beautiful?

• Richard Simon, D.Sc.
• Chief, Biometric Research Branch
• National Cancer Institute
• http//brb.nci.nih.gov

2
For Demonstrating a Large Treatment Effect
3
When the Size of the Treatment Effect is Large
Relative to Inter-patient Variability
4
(No Transcript)
5
(No Transcript)
6

• ?.05, z1-?1.96
• ?.10, z1-?1.28
• HR0.67, ?log(.67).40, Events263
• HR0.5, ?log(.5).69, Events88

7
Clinical Trials Show Small Treatment Effects
Because(choose one)

1. Treatments are minimally effective uniformly
   across patients
2. Ineffectiveness of treatments for most patients
   dilutes average effects

8
Develop Predictor of Response to New Drug
Using phase II data, develop predictor of
response to new drug
Patient Predicted Responsive
Patient Predicted Non-Responsive
Off Study
New Drug
Control
9
Evaluating the Efficiency of Targeting Clinical
Trials to Best Candidates

• Simon R and Maitnourim A. Evaluating the
  efficiency of targeted designs for randomized
  clinical trials. Clinical Cancer Research
  106759-63, 2004 Correction and supplement
  123229, 2006
• Maitnourim A and Simon R. On the efficiency of
  targeted clinical trials. Statistics in Medicine
  24329-339, 2005.
• reprints and interactive sample size calculations
  at http//linus.nci.nih.gov

10

• Relative efficiency of targeted design depends on
  
• proportion of patients test positive
• effectiveness of new drug (compared to control)
  for test negative patients
• When less than half of patients are test positive
  and the drug has little or no benefit for test
  negative patients, the targeted design requires
  dramatically fewer randomized patients
• The targeted design may require fewer or more
  screened patients than the standard design

11
Comparison of Targeted to Untargeted DesignSimon
R, Development and Validation of Biomarker
Classifiers for Treatment Selection, JSPI
Treatment Hazard Ratio for Marker Positive Patients Number of Events for Targeted Design Number of Events for Traditional Design Number of Events for Traditional Design Number of Events for Traditional Design
Percent of Patients Marker Positive Percent of Patients Marker Positive Percent of Patients Marker Positive
20 33 50
0.5 74 2040 720 316

12
TrastuzumabHerceptin

• Metastatic breast cancer
• 234 randomized patients per arm
• 90 power for 13.5 improvement in 1-year
  survival over 67 baseline at 2-sided .05 level
• If benefit were limited to the 25 assay
  patients, overall improvement in survival would
  have been 3.375
• 4025 patients/arm would have been required

13
Small is Beautiful

• When treatment effect can be measured with
  precision on individual patients
• Little placebo effect
• Comparative treatment effect not of interest

14
Small is Beautiful

• When there is substantial prior information about
  the effect of the treatment compared to control

15
Frequentist Meta-Analysis of Two Trials of the
Same Treatment
16

• Random effects meta-analysis tests whether
  hypothetical distribution F from which ?1 and ?1
  are drawn has mean zero.
• With only two-trials, random effects
  meta-analysis does not have any information on
  variance of F and so no meaningful combined
  inference is possible

17
Principles of Bayesian Analysis

• Evidence from data for a hypothesis should be
  based upon the likelihood of the actual data
  given the hypothesis, not upon the probability of
  data as extreme
• Evidence from data for a hypothesis should be
  modulated by the prior probability of the
  hypothesis

18
Bayes Theorem
19
Specifying Prior Distributions

• Non-informative
• Elicit opinion
• Skeptical/optimistic
• Past data
• Community concensus

20
Frequentist Methods are in many cases equivalent
to Bayesian Methods Based on Non-informative
Prior Distributions
21
Non-informative Prior Distributions are
Sometimes Extreme and Unrealistic
22
Fallacies about Bayesian Methods

• Require smaller sample sizes
• Require less planning
• Are preferable for most problems in clinical
  trials
• Have been limited in application primarily by
  computing problems

23
Facts About Bayesian Methods

• Require careful selection of prior distributions
• Are valuable for some problems in clinical trials

24
Simple Bayesian Model
25
(No Transcript)
26
Bayesian Analysis May Be More Conservative Than
Frequentist Analysis

• Two hypotheses ?0 and ? ?1
• Trial data

27
(No Transcript)
28

• If trial is designed for power ? and results are
  just significant at level ? then
• (Simon, Statistical Science 15103-105,
  2000)

29
(No Transcript)
30
(No Transcript)
31
Bayesian Posterior Probability of Null Hypothesis
When Trial Results are Just Significant
Prior Probability ?0 Posterior Probability ?0
0.75 0.5
0.5 0.25
0.25 0.1
32
(No Transcript)
33
(No Transcript)
34
Bayesian Posterior Probability of Null Hypothesis
When Trial Results are Just Significant(flat
prior under alternative)
Prior Probability ?0 Posterior Probability ?0
0.75 0.158
0.5 0.059
0.25 0.020
35
Small May Be Beautiful For

• Randomized phase II study comparing a new regimen
  to control
• Objective to obtain unbiased estimate better
  than using historical control
• Phase II endpoint may provide more events than
  phase III endpoint and therefore a smaller trial
• Phase II endpoint may permit more sensitive
  estimate of treatment effect but not be a
  suitable phase III endpoint
• Partial surrogate endpoint
• Phase II study can be sized based on inflated ??

36
Randomized Phase II Design Comparing Vaccine
Regimen to Control

• ? 0.10 type 1 error rate
• Endpoint PFS
• Detect large treatment effect
• E.g. Power 0.8 for detecting 40 reduction in 12
  month median time to recurrence with ?0.10
  requires 44 patients per arm with all patients
  followed to progression
• Two vaccine regimens can share one control group
  in a 3 arm randomized trial

37
Small May Be Beautiful

• When the objective is to select the most
  promising regimen from a set of candidates
• May or may not contain control arm
• Null hypothesis is never tested
• All candidate regimens should be equal with
  regard to endpoints other than the one used as
  the basis for selection

38
Randomized Phase II Multiple-Arm Designs Using
Immunological Response

• Randomized selection design to select most
  promising regimen for further evaluation. 90
  probability of selecting best regimen if its
  mean response is at least ? standard deviations
  above the next best regimen

39
Number of Patients Per Arm for Randomized
Selection DesignPCS 90
Number of treatment arms ? 0.5 ? 0.75 ? 1.0
2 13 6 4
3 21 9 6
4 24 11 6
5 27 13 7
6 30 14 8
7 31 14 8
8 35 15 9
40
Patients per Arm for 2-arm Randomized Selection
Design Assures Correct Selection When True
Response Probabilites Differ by 10
Response Probability of Inferior Rx 85 Probability of Correct Selection 90 Probability of Correct Selection
5 20 29
10 28 42
20 41 62
40 54 82
41
Randomized Selection Design With Binary Endpoint

• K treatment arms
• n patients per arm
• Select arm with highest observed response rate
• pi true response probability for ith arm
• pi pgood with probability ?, otherwise pbad
• With N total patients, determine K and n to
  maximize probability of finding a good rx

42
Probability of Selecting a Good Treatment When
pbad0.1, pgood0.5 and ?0.1
n K Probability
5 20 0.626
10 10 0.590
15 7 0.511
20 5 0.414
25 4 0.344
43
Probability of Selecting a Good Treatment When
pbad0.1, pgood0.3 and ?0.1
n K Probability
5 20 0.319
10 10 0.375
15 7 0.383
20 5 0.341
25 4 0.309
44
Probability of Selecting a Good Treatment When
pbad0.1, pgood0.3 and ?0.25
n K Probability
5 20 0.615
10 10 0.708
15 7 0.717
20 5 0.673
25 4 0.642
45
Probability of Selecting a Good Treatment When
pbad0.02, pgood0.15 and ?0.15
n K Probability
5 20 0.45
10 10 0.52
15 7 0.52
20 5 0.47
25 4 0.44
46
Probability of Selecting a Good Treatment When
pbad0.02, pgood0.10 and ?0.15
n K Probability
5 20 0.29
10 10 0.37
15 7 0.39
20 5 0.37
25 4 0.36
47
Small May Be Beautiful

• When the objective is to effectively treat the
  largest number of patients when the population of
  patients is small and several good candidate
  treatments are available

48

• N patients in horizon
• 2 treatments
• Perform RCT with n pts per rx
• Select treatment with best observed response rate
  and use that treatment for the remaining N-2n
  patients
• Binary endpoint with unknown response
  probabilities p1 and p2

49
Approximate Total Number of Responses
50
N1000, p10.6, p2 0.4
51
N200, p10.6, p2 0.4
52
Conclusions

• In clinical trial sizing, small is often not
  beautiful. It is often uninformative, duplicative
  and results in misleading results.
• In some cases however, small is appropriate and
  valuable.
• Having clear objectives is essential to properly
  sizing a clinical trial.