Progress and Regression in clinical trials

1
Progress and Regression in clinical trials

• 1950-1990 False POSITIVES increasingly well
  controlled by randomisation
• 1990-2000 False NEGATIVES increasingly well
  controlled by mega-trials and meta-analyses
• 2000 beyond Increasing regulation,complexity
  and costs may prevent many important public
  health questions from being answered reliably
  (REGRESSION)
• URGENT NEED TO SIMPLIFY TRIALS TO ENHANCE THE
  CONDUCT OF IMPORTANT TRIALS ESP IN VULNERABLE AND
  UNDERSERVED POPULATIONS BOTH IN DEVELOPED AND
  DEVELOPING COUNTRIES

2
Effect Sizes Diminishing Effects

• In the current era, where multiple effective
  therapies for a condition already exist, the
  incremental effects of a new treatment may be
  harder to detect
• Benefits may be more moderate when added to other
  treatments
• Benefits may be more moderate when tested against
  established treatments (e.g. 10 RRR, not 20
  RRR)
• Adverse effects may be more than established
  treatments
• THEREFORE, THE FUTURE GENERATION OF TRIALS
  COMPARING TWO ACTIVE AGENTS MAY HAVE TO BE
  SEVERAL TIMES LARGER THAN INITIALTRIALS OF ACTIVE
  VS CONTROL OR FOR NON INF TRIALS.

3
Examples of Smaller Treatment Effects in the
Modern Era

• Antiplatelet agents Chronic CADASA vs Control
  25 RRR in vascular events, Thienopyridine vs ASA
  10 RRR, Oral GP IIb/IIIa inhibitors vs ASA
  no diff. AMI 20 RRR of ASA v plac but Clop v
  plac on top of ASA10 RRR.
• Thrombolytic agents SK vs Control 25 RRR in
  mortality tPA vs SK 10 RRR (in
  mortality/disabling strokes)
• Bolus new agents vs infusion No diff in
  mortality, but increase in intracranial bleeds by
  30
• Thrombin inhibitors UFH/LMWH vs Control in UA
  45 RRR, Fonda v Enox 0(20) RRR but 50 RR in
  major bleeds.

4
Potential Cumulative Impact of 4 Simple Secondary
Prevention Treatments
CUMULATIVE BENEFITS ARE LIKELY TO BE IN EXCESS
OF 75 RRR, WHICH IS SUBSTANTIAL
5
Questions that Require Substantial Efficiency in
Costs for Large Trials to be Conducted

• Non-pharmaceutical Lifestyle modification (e.g.
  wt reduction), surgical procedures, eval
  diagnostic procedures, nutrition supplements or
  mod (e.g. vitamins,breast v formula), health care
  delivery (e.g. parameds to provide antenatal
  care/deliveries, handwashing)
• Generic drugs New uses of old drugs (e.g. HOPE)
• Combination therapies (Polypill)/Extending
  duration of therapies (Duration of tamoxifen in
  breast cancer)
• Developing country questions e.g. Chagas
  disease,HIV, TB ,antithrombotics in resource poor
  settings.

6
Key Elements of a Good TrialAnswering an
important question reliably.
Randomization Large No. Events Good adherence and
complete followup Unbiased Evaluation
7
Complexities of a Trial
Voluminous data/patient
Adjudication
Regulators
Randomization Large No. Events Unbiased Evaluation
Detailed Eligibility
Proliferating Committees
Audits
Multiple approvals
Complex Monitoring
Complex informed consent procedures
8
Proliferation of laws and guidelinesmay make
trial results LESS reliable(and so harm, not
help, patients)

• Clinical trial conductICH Guideline for GCPEU
  Clinical Trials Directive
• NHS Research Governance
• Data access/confidentiality1998 Data Protection
  ActGMC guidance on confidentialityHealth
  Social Care Act/PIAG
• Ethics consentHelsinki Declaration

9
Publics Attitudes vs Legal/Regulatory
Restrictions

• Over 98 of the general public do not have
  concerns on data misuse or violation of
  confidentiality in research studies where ethics
  committees have reviewed the protocol.
• In CRASH, only 1/10,008 enrolled withdrew the
  consent initially provided by the relative. In
  PAC-MAN, only 3.3 refused consent when they
  regained capacity.

10
Privacy and Confidentiality Laws on Clinical
Trials

• Allow use of medical records to screen patients
  for trials Facilitated by informing all patients
  that this is the case, but they can opt out.
• -IRBs/ethics committees should be encouraged to
  agree to this
• Use patient identifiers for follow-up (within and
  beyond the trial) through central mechanisms
  (coordinating center, national registries, etc).
• Access records to confirm events.
• 2 and 3 can be facilitated by obtaining the
  consent of the participant

11
Good Clinical Practice?

• Chiefly a bureaucratic document that is related
  to documentation and mechanics of research and is
  neither good, clinical nor practical for clinical
  trials.
• Reaction to perceived/documented(rare) sloppy
  data collection, suspicions that investigators
  and sponsors may be dishonest
• While simple guidelines on ensuring unbiasedness
  and accuracy of data are reasonable, current
  guidelines are far too defensive and make many
  trials of good questions (especially non
  industry) almost impossible to conduct.
• Suggestion Need new set of sensible guidelines
  by an independent Professional Body (eg.Society
  for Clinical Trials)
• NBMost trials that have changed practice have
  NOT used GCP.

12
Reg requirements that can be substantially
simplified/eliminated

• Multiple IRB/REB approvals ( central per country
  /reciprocity)
• Approved informed consent forms(simplify)
• All REB amendment approvals(simplify and only
  major changes thru a central website)
• All future REB annual reviews(?eliminate/post
  progress on a website)
• Contracts(simplify/standardize)
• Lab certif and ref normal ranges(only for
  special tests)
• Import licenses and HQP inspections(?elimin/simpl
  ify)

13
MRC review Potential for EU Clinical Trials
Directive (2001) to be a major obstacle to
important trials

• Increased bureaucracy due to requirement for
  single sponsor (possibly the funding source)
• Burdensome drug authorisation and supply (GMP
  labelling) processes
• Threat to trials of emergency treatments for
  patients unable to give consent
• Rigid approach to pharmacovigilance and site
  monitoring (through over-interpretation both by
  regulators, pharma beaureacrats and recently
  IRBs)
• Substantial cost increases may result in fewer
  important trials being conducted

14
Impact of EU Clinical Trials Directive(2001)on
non-commercial cancer trials in UK(Eur J Cancer
2006)

• Doubling in costs of running non-commercial
  cancer trials and 6-12 month delays to starting
• Major concerns about correct interpretation due
  to lack of central guidance, lack of clarity
  regarding interpretation of guidance notes, and
  increased documentation
• Clinical trial units unable or unwilling to start
  in non-UK centres due to different
  interpretations in different European countries

15
New EU Directive 2005/28/EC (Recital
11)simplified procedures for non-commercial
trials

• Non-commercial clinical trials conducted by
  researchers without the participation of the
  pharmaceutical industry may be of great benefit
  to the patients concerned. The conditions
  under which the non-commercial research is
  conducted by public researchers, and the places
  where this research takes place, make the
  application of certain of the details of good
  clinical practice unnecessary or guaranteed by
  other means.

16
EU definition of non-commercial trials

• Sponsor is university, hospital, public
  scientific organisation, non-profit institution,
  patient organisation or researcher
• Data from trial belongs to this non-commercial
  sponsor
• Design, conduct, recording and reporting under
  their control
• 
  Impractical
• No agreement in place between sponsor and
  third parties that allows use of trial data for
  regulatory or marketing purposes and
• Trial should not be part of the development
  programme for a marketing authorisation of a
  medicinal product.

17
ICH GCP Guidance on monitoring

• extent and nature of monitoring should be
  based on considerations such as the objectives,
  purpose, design, complexity, blinding, size and
  endpoints of the trial. In general there is a
  need for on-site monitoring before, during and
  after the trial however central monitoring
  can assure appropriate conduct of the trial in
  accordance with GCP
• ICH GCP 5.18.3

18
On-site monitoring

• (...) the trial management procedures ensuring
  validity and reliability of the results are
  vastly more important than absence of clerical
  errors. Yet, it is clerical inconsistencies
  referred to as errors that are chased by the
  growing GCP-departments.
• Refs Lörstad, ISCB-27, Geneva, August 28-31,
  2006
• Grimes et al, Lancet 2005366172

19
Examples of Cost Escalations that Damage the
Feasibility of Trials

• CREATE-ECLA(20,000 AMI eval GIK/LMWH) Costs for
  a CRO in India to obtain regulatory approvals,
  import and distribute drugs exceeded the entire
  study budget
• UNNAMED TRIAL A trial of 20,000(statin/combo BP
  lowering) followed for 5 years proposed at a
  total cost of 80 million. Funding approved by
  XYZ company.
• -Added complexities related to monitoring,and
  perceived regulatory requirements , escalated
  costs to 140 million. Funding withdrawn.
• -Revised simple trial with 10,000 (higher risk)
  individuals ongoing at 30 million

20
Quality Assurance why ?

• The purpose of quality assurance is not to ensure
  that individual data items are 100 error-free.
• Its purpose is to ensure that the clinical trial
  results are reliable, i.e.
• observed treatment effects are real
• their estimated magnitude is unbiased

21
A taxonomy of errors

• Random errors (Random with respect to treatment
  assignment and unlikely to materially influence
  study results , unless large).
• Measurement errors (eg due to assay precision or
  frequency of visits)
• Errors due to transcription errors,variations in
  entry criteria.
• Some types of fraud (fabrication of non key
  data items)
• Systematic errors(Differential with respect to
  treatment assignment and could substantially bias
  results).
• Design flaws (eg post rand exclusion,biases in
  event ascertainment,failure to use intent to
  treat)
• Some types of fraud (usually with knowledge of
  treatment assignment as in a single center study)

22
VeGF trial for macular degenerationMedian
simulated P-values (and IQ range)
23
A randomized study of the impact of on-site
monitoring
Stratify by - Type (Academic vs Private) -
Location (Paris vs Province)
Centers accruing patients in trial AERO B2000
Group A (site visits)
Group B (no visits)
Ref Liénard et al, Clinical Trials 200631-7
24
RCT of onsite monitoingImpact of initiation
visits on patient accrual
No difference
25
Impact of initiation visits on volume of data
submitted
No difference
26
Impact of initiation visits on quality of data
submitted
No difference
27
Prevalence of fraud?

• Industry (Hoechst, 1990-1994)1 case of fraud in
  243 (0.43) randomly selected centers
• FDA (1985-1988)1 of 570 routine audits led to a
  for-cause investigation
• CALGB (1982-1992)2 cases of fraud in 691 (0.29)
  on-site audits
• SWOG (1983-1990)no case (0) of fraud in 1,751
  patients
• McMaster(1992-2007) 2 (0.1) in 46 trials
  involving gt250,000 patients from over 5000
  centers.
• ?fraud is probably rare (but possible
  underestimation esp in era of paying more than
  the costs of trials?)
• Ref Buyse et al, Statist in Med 1999183435

28
Impact of fraud

• Most frauds have little impact on the trial
  results(unless widespread or at central) because
• they introduce random but not systematic errors
  (i.e. noise but no bias) in the analyses
• may affect secondary analysis (e.g. subgroup
  analyses if baseline data are incorrect)
• their magnitude is too small to have an influence
  (one site and/or few patients)
• Refs Altman, Practical Statistics for Medical
  Research 1991
• Peto et al, Controlled Clin Trials 1997181

29
Example of fraud detection thru central checks

• Invitation to collaborate in being a coauthor(by
  Dr XY thru well respected invest and friend) on a
  paper demonstrating the marked benefits of a Vit
  in individuals with specific genotype in
  preventing CVD.(Result would have major
  implications, attract great attention, and
  targetted for a leading journal).
• Requested the data base randomization did not
  match various explanations of process and
  sequence,diff in events marked and early(both
  implausible), event rate patterns and rates not
  clinically consistent ,indication of an
  independent DSMB who stopped the trial but
  named chair unaware of even being on the
  DSMB(who was named as the last author and thru
  whom my involvement as a coauthor was requested).
• WE PROVIDED A REPORT OF OUR FINDINGS AND DECLINED
  PARTICIPATION

30
COMMIT Example of central checks indicating
problem at one of 1250 participating hospitals
31
Statistical approaches to data checking

• Humans are poor random number generators ? test
  randomness (e.g. Benfords law)
• Plausible multivariate data are hard to fabricate
  ? test correlation structure
• Clinical trial data are highly structured?
  compare expected vs observed
• Clinical trial data are rich in meaning? test
  plausibility (e.g. dates)
• Fraud or gross errors usually occur at one
  center? compare centers

32
Approaches to Study Monitoring (1)

• Trial oversight Operations or Trial Management
  Committees, Steering Committee, Independent Data
  Monitoring Committee
• Central Monitoring
• -fax consent forms centrally
• -central faxing of key documents (e.g. ECGs,
  laboratory reports, discharge summaries)

Statistical Monitoring
33
Approaches to Study Monitoring (2)

• 3. On Site Monitoring
• a) Random and infrequent
• b) Guided by Central Monitoring
• c) Onsite mentoring instead of monitoring
• A combined approach of central with directed
  onsite monitoring(random and for cause) is likely
  to be both efficient and effective.

34
Safety Monitoring, Reporting and Reviewing AEs

• Current
• Any event including those that are the outcomes
  of interest,and events that are common in the
  condition of that age are considered to be AE
• -They(sometimes even the primary outcome) are
  often recorded, reported (expedited)
• -Reviewed individually,so difficult to discern
  patterns
• -Unblinded, and SAEs in the active group only are
  reported to investigators, their IRBs and to
  regulatory authorities

35
Safety Monitoring, Reporting and Review

• Problems
• -Enormous amount of effort (upto about 25 hrs/SAE
  reported) BUT is it useful and worthwhile?
• -Misleading as to the safety situation of the
  trial,as no between group comparisons are
  possible and impossible to reliably attribute
  causality on a case by case basis (except perhaps
  for very unusual events eg thrombocytopenia or
  liver failure).
• -Lack of a balance between safety and efficacy,
  e.g.in OASIS 5, catheter thrombosis (excess of
  0.2), bleeding (reduced 2.5), mortality
  (reduced 1.0) with fonda in OASIS-5

36
Alternative Approach to Adverse Event Reporting
and Review

• Report all relevant data to a coordinating
  center, which regularly shares SAE and efficacy
  to the DSMB.
• Report to regulatory bodies and centers, only if
  the DSMB judges that harm exceeds benefit

37
VALIANT
Central Events Committe versus Site Investigator?

• 11,751 events reviewed in VALIANT
• Events identified by Site Investigator

Events Agee Cause of Death 2897 66 CHF 384
1 73 MI 2159 63 Resucitated Death 636 27 Strok
e 541 91
Who is correct? Investigator with more clinical
info or the comm thousands of miles away?
38
Effect of Adjudication on Estimated Treatment
Effect McMaster Experience(108,000 pts)
Adjud. Investigator OR OASIS-1 0.70 0.71 0.99 OA
SIS-2 0.90 0.85 1.06 HOPE 0.78 0.80 0.98 HOPE-2 0.
95 0.93 1.02 CURE 0.82 0.80 1.03 OASIS-5 1.01 1.01
1.00 OASIS-6 0.86 0.86 1.00 CREATE 0.87 0.87 1.00
WAVE 0.82 0.94 0.87 ACTIVE-W 1.43 1.42 1.01
39
Primary Endpoint Results According to Adjudication
Treated Placebo P-value EPIC Adjudicated
8.3 12.8 0.009 Investigator 9.0 12.4 0.12
0 IMPACT II Adjudicated 9.2 11.4 0.063 Inves
tigator 5.5 7.8 0.018 GUSTO IIB Adjudicated
8.9 9.8 0.058 Investigator
8.4 9.6 0.016 PURSUIT Adjudicated
14.2 15.7 0.04 Investigator
8.0 10.0 0.0007 CHARM-Preserved
Adjudicated 22.0 24.3 0.12 Investigator
21.4 24.7 0.028
40
Central Adjudication of Events When and to What
Degree?

• Not needed Hard endpoints, especially blinded
  studies
• Needed for open trials, especially when the
  outcome may be subject to interpretation
• Occasionally Central screening for missed
  events
• Fundamental need is to avoid BIASES and LARGE
  MISCLASSIFICATIONS
• Accuracy may be enhanced by collecting
  information on CRFs on outcomes in a structure
  than matches protocol definitions

41
Factorial Designs

• Increased efficiency as two for the price of
  one
• Only way to answer generic questions, by
  piggybacking it to a more fundable question
• Avoid overfactorializing (e.g. 2x2x2x2)
• Our approach ALWAYS try to factor a generic
  question(Vit, fish oils,GIK,intervention type or
  procedures, lifestyles, vaccines,etc), unless
  impossible.
• FACTORIAL DESIGNS ARE UNDERUTILIZED DUE TO BOTH
  ACADEMIC, INDUSTRY AND REGULATORY FEARS THAT
  QUALITATIVE INTERACTIONS WILL OCCUR THAT ARE
  LARGELY MISPLACED.

42
ISIS-2 Study
ISIS 2 Lancet 1988
43
Increasing Clinical Trials in Disadvantaged/Vulner
able Populations (1)

• 90 of the global burden of disease occurs in LIC
  MIC yet only 10 of the 70 billion of the
  research expenditures occur in these countries,
  e.g. Chagas disease affects 18 mill people in L.
  America yet only 4 RCTs involving 800
  individuals followed for 3 to 6 months are
  available .BENEFIT 3000 patients x 3 yrs,
  evaluating benznidazole T.B. pericarditis (50 6
  mo mortality), no major trials.IMPI steroids and
  a vaccine in 800( 3000) people.
• Neglected populations e.g. children, vulnerable
  groups, etc.
• Neglected conditions, e.g. road traffic accidents
  (CRASH steroids in head injuries), cardiac
  arrest, other critical illness

44
Intensive care management of severe head injury

• Percent therapy for intracranial
  hypertension
• 1995 USA 1996 UK No.pts(trials) OR(CI)
• Barbiturates 33 56 208(3)
  1.09(.91,1.47)
• Corticosteroids 64 49 2119(16)
  0.96(.85,1.08)
• CSF drainage 44 - 0(0) --
• Hyperventilation 83 100 77(1)
  0.73(.36,1.49)
• Mannitol 83 100 44(1) 1.75(.48,6.35)

45
CRASH Trial Death within 14 days
46
Increasing Clinical Trials in Disadvantaged/Vulner
able Populations (2)

• Consider trials with consent of relatives or
  surrogates and when time is of the essence
  perhaps without any consent but IRB approvals
• CRASH

47
Need for Randomized Trials in Developing Countries

• Global disease burden occurs largely in
  developing countries, but few RCTs (e.g. only 3
  trials in Chagas disease, no large trials in TB
  pericarditis or Rh Heart disease)
• Simple, widely practicable therapies are likely
  to have greater impact than complex, expensive
  treatments
• The same disease, its manifestations, its
  severity and complications may vary or progress
  very different ways in developing countries
  compared to developed countries
• TRIALS HAVE SCIENTIFIC, POLITICAL AND
  DEMONSTRATION VALUES THAT OFTEN HAVE TO BE
  REESTABLISHED IN MANY DIFFERENT SETTINGS (E.G.
  BCG VACCINE TRIALS OR VITAMIN A SUPPLEMENTATION)

48
The Globalization of Cardiovascular Trials
49
Globalization of Trials in CVD
50
ACTIVE Recruitment, Compliance Data Quality by
High Middle Income Countries
51
Globalization of Trials in CVD

• 1. Most CVD trials include LIC/MIC to reduce
  costs and speed enrollment, and are primarily
  aimed at answering questions relevant to the
  West.
• BUT, these trials may also have applicability in
  LIC/MIC if the condition is common and the
  treatments are affordable.
• 2. Need trials in LIC MIC of locally relevant
  conditions and treatments, and locally conducted.

52
Barriers to Funding Global Trials

• No (e.g. in LIC/MIC) or modest investment in
  clinical trials by governmental/charitable bodies
  even in Western countries (usually 2 to 8, in
  Canada about 4)
• Balkanization in funding (e.g. HSFS in Canada or
  need special justification to include foreign
  centers) and restrictions in their use by
  territory (e.g. by province or country)
• Misperceptions about the importance of large
  clinical trials (cookbook and mundane) vs basic
  science (more fundamental and exciting
• Perception that clinical trials funding should be
  left to industry

53
Improving global health thru reliable trials of
important questions

• Randomize a large number of (high risk)
  individuals
• Minimize data collection per subject drastically
• Minimize complexities (e.g. adjudicatIion,
  monitoring, standardization, AE reporting,
  approvals, etc.)
• MANY MORE trials with factorial designs.
• Need more trials in vulnerable populations and
  developing countries, where the disease burdens
  are the largest
• PARADOXICALLY LESS MONITORING, ADJUDICATION,
  AUDITING etc,AND DELIBERATE SIMPLIFICATION LEAD
  TO MORE RELIABLE RESULTS