Statistical Issues in Specification of D

1
Statistical Issues in Specification of D

• Daphne Lin, Ph.D.
• Erica Brittain, Ph.D.
• Division of Biometrics III

2
Outline

• Intro Non-inferiority trials
• What is D?
• History of D Selection
• Principles for determining D
• Difficulties in practice
• Alternate designs
• Summary

3
The Problem

• How can we show the new drug has identical
  efficacy to the standard drug (active control
  drug)?
• Answer WE CANT!
• SOLUTION We must allow some potential
  difference in efficacy D

4
What is D?

• The largest clinically acceptable difference
• Should be smaller than differences observed in
  superiority trials of active comparator

5
Non-inferiority Trials

• Non-inferiority trial to show the new drug is no
  worse than the standard drug by some margin D.

6
Estimation of treatment effect Confidence
Interval

• Define treatment effect as the absolute
  difference of percent cure rates (e.g. 85-75
  10)
• A confidence interval around this estimate of
  treatment effect is used as the primary analysis
  for non-inferiority trials
• Interpretation of the 95 confidence interval
  95 confident that the true difference in
  efficacy between these two drugs is contained in
  the confidence interval.

)
7
Example 1 Non-inferiority demonstrated
200 pts/arm, test drug 88, control drug 90,
CI (-8.6 4.6)
D 10
Observed success rate between test and control
drug e.g. - 2

95 confidence interval

- 8.6
4.6
8
Example 2 Non-inferiority not demonstrated
200 pts/arm, test drug 84, control drug 90,
CI (-13, 1.1)
D 10
Observed success rate between test and control
drug e.g. - 6

95 confidence interval
1.1
- 13
9
Goals of non-inferiority trials

• Indirectly determine if the test drug is better
  than placebo
• Directly determine if the test drug is similar to
  the active control drug

10
History of D Selection

• in FDAs Division of
• Anti-infective Drug Products

11
1992 PTC Step Function Approach
12
Concerns about 1992 PTC approach

• Seriousness of disease and consequence of
  treatment failure were not taken into account
• whether D was small enough that a drug with no
  efficacy could meet the standard was not
  considered
• Has undesirable statistical properties ( e.g. D
  jumps from .15 to .2 when success rate changes
  from .8 to .79)

13
Concern of Potential Bio-Creep

• Trials over time used progressively less
  effective control arms
• Example if D of 20 is used

Drug 1
Drug 2
100
60
70
0
14
July 1998 AC Meeting

• The FDA stated D should reflect
• historical cure rate (with and without therapy)
• risk associated with treatment failure
• advantages and disadvantages of study drug

15
July 1998 AC Meeting Proposal

• Clinically relevant delta
• Indication specific
• Special situations
• Discuss with Medical Division during protocol
  development
• Sponsor should provide rationale for selection of
  control arm

16
Committee for Proprietary Medicinal Products
(CPMP)

• Published Guidance on evaluation of new
  anti-bacterial medicinal products in 1997
• D10 for common non-serious infections
• Smaller for very high cure rates
• Based on minimum clinically relevant difference
• Justified in protocol

17
Transition

• For past 1-2 years, have worked with sponsors on
  case-by-case basis to specify D
• In February 2001, a disclaimer added to PTC
  document stating that PTC approach phased out
• There is a need to establish standards (ICH-E10
  principles)

18
Now what?
19
Road Map

• Principles for determining D
• Based on ICH-E10
• Difficulties in practice
• This is the hard part we need advice
• Alternate designs
• Summary
• GOAL Choice of D is not a technical matter

20
To demonstrate efficacy

• Experimental drug should be better than placebo
• In some settings, experimental drug should also
  have similar efficacy to existing therapy
• Choose D to assure that both of these goals are
  met

21
ICH-E10 says

• Non-inferiority design is appropriate and
  reliable only when the historical estimate of
  drug effect size can be well supported by
  reference to results of previous studies of the
  control drug

22
Translation

• We must KNOW with good precision the magnitude of
  advantage of the active control drug over placebo
  in setting of clinical trial
• In practice if advantage is large, precision of
  estimate probably wont matter
• If potentially modest, precision is critical
• If only a single trial of AC w/ borderline
  efficacy -- poor info about magnitude
  (Statistical significance not enough)

23
Principles from E10

• D based on both statistical reasoning and
  clinical judgment
• D cannot be larger than the advantage the
  active drug would be reliably expected to have
  compared with placebo in the setting of the
  planned trial
• Usually choose D to be even smaller to ensure
  some clinically acceptable treatment benefit
  maintained

24
Possible Choices for D
True Success Rate of Placebo
True Success rate of Active Control
D 3
D 7
D 15
70 72 74 76 78 80
82 84 85 True Success Rate
25
General approach to D

• Determine two values
• Conservative estimate of advantage of AC over
  placebo (D1)
• DATA-BASED
• Largest clinically acceptable difference between
  AC and experimental drug (D2).
• JUDGMENT-BASED
• D smaller of these two values Value used in
  upcoming NI trial

26
Estimation of benefit of AC over placebo (D1)

• The true success rate of active control minus the
  true success rate of placebo in setting of
  clinical trial
• By how much is AC better than placebo in the NI
  trial, if placebo were present?
• Based on historical data -- NOT A CHOICE
• Not important when benefit is large

27
Estimating benefit of AC (D1) Be Conservative

• E10 D should reflect uncertainties in evidence
  on which choice is based and should be suitably
  conservative
• If D1 is over estimated, chance of concluding
  efficacy when new drug is no better than placebo
  is too high
• Err on side of underestimating benefit. IF poor
  historical info dont use best guess, use
  smallest of reasonable values

28
Estimating benefit of AC(D1 )What is best
information?
Best information for determining D1
Multiple PCTs w/ same design as NI Single PCT
with same design as NI Multiple PCTs with
different design Single PCTs with different
design Observational studies Anecdotal / Case
Series
No information for determining D1
29
Estimating benefit of AC(D1)Placebo Control
Trial Data

• What if placebo controlled data exist, but
• Trials are old antibiotic resistance, changes in
  clinical care management
• Proposed active control not studied
• Few in number / consistency?
• Differences in entry criteria, assessment
  criteria, timing, populations
• Take data with big grain of salt --Then what?

30
Estimating D1 Bottom Line

• Use historic data -- preferably from placebo
  controlled trials with same designs as upcoming
  NI trial
• Bad news historic data is often poor because of
  ethical constraints
• conservative estimate not straightforward
• Good news precision is probably irrelevant if
  benefit is KNOWN to be very large

31
D Step 2

• Recall two step process
• Step 1 Determine estimate of advantage of AC
  over placebo (D1)
• Step 2 Select acceptable loss from AC (D2)
• D smaller of D1 and D2
• Selection of D2 is primary concern for most
  Anti-infective indications

32
Selection of clinically acceptable loss (D2)

• NOT STATISTICAL DECISION
• CLINICAL judgement of largest acceptable
  difference between AC and the new drug
• a difference D2 is so important clinically that
  it must be ruled out -- no tolerance for any
  difference D2
• D2 is borderline value between just barely
  acceptable and NOT acceptable

33
Selection of clinically acceptable loss (D2)

• Consequence of treatment failure is important
  factor in determining D2
• What proportion of study failures are deaths or
  other very serious morbidity?
• Can treatment failure be easily
  reversed/addressed?

34
Clinically Acceptable Loss (D2)Consider
consequence to patients
35
Clinically Acceptable Loss (D2)Consider
Clinical Trial Realities

• Some patients do not have disease
• Trt diff Pts with bacterial infection 12
• Trt diff Pts without bacterial infection 0
• Measured trt diff If 50/50 mix 6
• If D10, may conclude New is efficacious
• BUT key population difference10
• BE CAREFUL Other factors dilute observed
  treatment difference too

36
Selection of clinically acceptable loss (D2)

• SUMMARY
• Consequence of treatment failure
• Potentially large impact on patient care
• Be careful about clinical trial realities
• CLINICAL judgement

37
General Approach D for each indication

• Provides regulatory consistency
• But, requires vigilance
• New AC may be less efficacious than original AC
  (is D smaller than advantage of AC over placebo?)
• Emerging resistance and other temporal changes
  may have diminished the efficacy of any AC

38
Consequence to Sample Size

• D cut in half, sample size quadruples
• But if new drug is slightly better than AC,
  sample size can be sharply reduced
• 80 Power, D10
• Assume cure rates both 80, SS/grp252
• Assume new drug rate is 82, SS/grp168
• Sample Size cut by 1/3

39
BIGGEST CHALLENGES

• Biggest challenges in selection of D
• Indications where treatment effect is potentially
  modest but not precisely known
• Serious indications with low incidence
• Superiority designs may offer an important
  alternative to NI design stronger evidence and
  potentially smaller sample size
• Can they be done ethically?

40
Early Escape Ethical?

• Applicable to few indications (less serious)
• Two arms Experimental vs. placebo
• KEY Pts seen several days after baseline
• If blinded assessment no improvement
• Failure in analysis, therapy switched
• Ethically consistent with wait and see
• Variant Early Escape with 3 arms

41
Other Superiority Designs

• Placebo Add-on
  Existing New vs.
  Existing Placebo
• Does New have benefit in presence of Existing
  therapy labeling implications
• Dose-response (High vs Low dose)
• Superiority to some comparator
• Combination

42
Selection of D BIG PICURE

• Take Home Message

43
Choice of D impacts patients

• If D is incorrectly chosen so that greater than
  advantage of AC over placebo
• Patients may get drugs with no benefit, while
  exposed to toxicity and potential for development
  of resistance
• Potential benefits of using smaller D even where
  no concern about placebo rate
• More pts cured overall, higher survival
• Subtle but important differences detected
• Smaller D larger and longer studies

44
Take Home Message

• Absolute D MUST be smaller than conservative
  estimate of the advantage of Comparator over
  placebo
• Challenge insufficient historic data
• Clinical judgement may decrease D further
• To rule out important loss in efficacy
• Superiority designs can play important role in
  some settings