Introduction to Regulatory Statistical Principles

1
Introduction to Regulatory Statistical Principles

• Peter A. Lachenbruch
• Oregon State University
• College of Public Health and Human Sciences (Ret.)

2
Conclusion

• There can be few areas where the discipline of
  statistics is conducted with greater discipline.
  Pharmaceutical statisticians may be engaged in
  work that may sometimes involve routine
  calculations, but the regular application of
  statistical principles to produce high quality
  experiments, data and analysis promotes a
  professionalism that can itself be a source of
  satisfaction.
• Stephen Senn (2000 The Statistician)

3
Research vs. Development

• Planned development vs. Exploratory analyses
• Implications regarding significance tests
• Journal Publication or Licensing
• Understanding a biological process or
  demonstrating a therapeutic benefit
• Assess efficacy and Safety
• Submit data for review by regulators

4
The Cast of Characters

• Sponsor the entity that will market the
  product, be responsible for writing the marketing
  application (NDA, BLA etc.)
• Usually have many skills - Biology,
  Biochemistry, Medicine, Regulatory Affairs,
  Biostatistics, etc.
• These people have roles that change over time
  biologists in pre-clinical times, biostatistics
  in clinical phases and pre-clinical, etc.
• May be drug company, university, government
  agency, etc.

5
Characters (2)

• Investigators the scientists who conduct the
  trials. Must demonstrate their qualifications.
  May be physicians, biochemists, etc. Employees
  of sponsor or contracted to sponsor (e.g., a
  University scientist), CRO (contract research
  organization)
• May also be independent scientists e.g., many
  statisticians serve as consultants to drug
  companies

6
Characters (3)

• Regulatory agency in US the FDA plays this
  role. Has 6 centers CBER, CDER, CDRH, CVM,
  CFSAN, NCTR. Most of what we will talk of here
  is related to the first three (Biologics, Drugs
  and Devices)
• Requirements are governed by the authorizing laws
  and regulations that have evolved over many
  years. See the Code of Federal Regulations (CFR)

7
Two trials needed

• For many years, the FDA required two adequate
  and well-controlled trials that showed clinical
  benefit to license a drug or biologic. The sole
  exception was vaccines which usually were studied
  in large (ngt5000) trials.
• Recently, the agency has adopted a more flexible
  approach, especially in rare diseases and
  conditions in which obtaining enough patients for
  two trials would be difficult.
• If you wish to license a product with one trial,
  you should contact the agency and discuss the
  case .

8
ICH

• The International Conference on Harmonization
  (ICH) is an effort to require the same standard
  of evidence for licensure in various regions of
  the world.
• Joint by US, Europe, and Japan regulatory
  agencies and by the pharmaceutical industry
  organizations in those regions.
• You can find links to these at www.ich.org

9
Phases of Drug Development

• Pre-clinical
• Prior to the first human study, the sponsor must
  show evidence that the product does not have
  apparent unacceptable risks for humans
• Done in animals rats, mice
• Carcinogenicity, teratogenicity
• Stability, shelf-life, potency, contaminant
  detection
• Studies of Maximum Tolerated Dose, route of
  administration, frequency of administration (also
  done in phase 1 studies
• ICH S series (safety) and Q series (quality)

10
Phase 1 Studies

• Must have a valid Investigational New Drug
  authorization allows sponsor to conduct studies
  in human populations
• Find MTD
• Show a therapeutic response in a wide enough dose
  range that the product can be used safely
• Primarily safety studies. May be in non-diseased
  population or a diseased population. Depends on
  the drug.
• Often not randomized

11
Phase 2 Studies

• Learn more about proper dose, schedule of
  administration, route of administration (oral,
  subcutaneous, intravenous)
• Vaccines are frequently single dose, sometimes a
  booster dose is given
• Therapeutic drugs may be given multiple times
• Some products are given by multiple routes (at
  physician discretion)
• Pain medications usually given until relief is
  obtained, so concern about cumulative dose
• Chronic medication e.g., cholesterol lowering
  medication, cardiovascular meds, diabetes
  medication may be given for long periods
  lifetime. Must monitor for problems.

12
Phase 2, continued

• Must pre-specify response/outcome
• Cant collect many outcomes and select one that
  seems to work well. This is exploratory analysis
  and is never accepted by regulators.
• Analysis plan must be specified
• Report to regulatory agency of results, including
  safety, efficacy, other data.
• It is wise to study several doses, routes of
  administration, and schedules of administration
• Sometimes there is information from products of a
  similar class that will assist in these studies

13
Phase 3 Studies

• Must show superiority of the product
• Two trials usually needed
• Prefer there to be two different populations
  e.g. tertiary care and primary care centers
• Need reports for each trial often detailed
  information on patients is given.
• The statistical analysis plan (SAP) must be
  specified and shouldnt be changed unless there
  is new evidence of changes in the endpoints.
  Changes after the data have been locked are
  highly suspicious.

14
Working with Regulators (with reference to FDA)

• Goal of both sponsor and FDA is to bring safe,
  effective products to market
• Sponsors want to present their product in the
  most favorable light (sometimes downplaying the
  bad things)
• Regulators (FDA especially) are cautious and
  dont want to approve a product that is
  ineffective or has a poor balance of benefit and
  risk.

15
Regulatory work (2)

• Note that regulatory agencies are not monoliths
• Advice may differ based on which center,
  division, or branch you deal with sometimes
  reviewers will affect the advice
• Advice you get may depend on what the favored
  research paradigm is, approach, custom of
  division/branch e.g., how missing values are
  treated
• Note that you can appeal the decision if you
  differ from the reviewer

16
Regulatory work (3)

• Evidence standard
• Two convincing trials p-value below 0.05
• What if the two convincing trials happen after 10
  attempts?
• Can use a negative binomial to suggest this isnt
  persuasive. I find the probability of 2
  successes by the 10th trial is 0.0186 under the
  null hypothesis
• If p(success ) is 0.8, the probability of 2
  successful trials by 4 trials is 0.97
• Usually need to show superiority, rather than
  non-inferiority although safer products can be
  persuasive

17
Regulatory work (4)

• Single trial can be accepted if
• A very large trial such as a vaccine trial (tens
  of thousands of observations)
• A very serious and rare disease is being studied
• Speak to the regulators! Many sponsors are
  reluctant to discuss their concerns with the FDA
  since they seem to fear that if they tell some
  problem, the FDA will focus on it.
• This is a recipe for disasters. Its better to
  avoid problems in interpretation early than have
  a fight later on.

18
Some suggestions

• Know your reviewers
• Perspective
• Listen to any advice they offer and make
  counter-proposals if appropriate reviewers will
  listen. They may not agree
• Reviewers may change during your study its
  important to have a record of any agreements
  dont rely on memory.
• Dont rely on biological arguments clinical
  trials may contradict the biological argument

19
Types of Paperwork

• IND or IDE
• The application to conduct a clinical trial. No
  trial can proceed without one. Can be amended
• Outline studies to be conducted, modifications to
  ongoing studies, where the study will be
  conducted, when it will be, etc.
• What drugs, dosage, schedules, route of
  administration will be examined, etc.
• Specify statistical analysis plan at an early
  amendment
• Non-standard analyses can be proposed should be
  justified
• Any IND can be amended studies are not cast in
  stone.

20
Paperwork (2)

• One-arm studies (no control) are usually
  unacceptable
• FDA has seen too many trials that have the single
  arm later shown to be inferior to a control arm
• Non-randomized studies are notorious for having
  bias the investigator knows what drug is given,
  rates the patients more highly than in a
  controlled trial.

21
Paperwork (3)

• A famous study (Moertel) extracted results from
  many oncology studies and found the
  non-randomized ones always showed high response
  rates, while the randomized ones were much
  poorer. Some studies of this type have been
  accepted when the population is small and the
  natural history of the disease is well known.

22
Paperwork (4)

• Make use of ICH guidelines and FDA guidelines.
• These are not mandatory, but represent best
  current thinking in their areas. Alternatives
  can be proposed.
• Most important for clinical trials are E3, E9,
  E10
• Generalizing to a population
• Clinical trials are conducted on a small subset
  of a population. In order for a product to be
  approved for marketing, FDA and other agencies
  want to be convinced that the results apply to
  the broader population. For this reason,
  sponsors either use broad entry criteria, or
  have multiple studies on different populations
• This also includes patients or subjects of
  different ages, different physical conditions, or
  different hospitals.

23
Paperwork (5) Marketing applications

• NDA and BLA are applications to market a new
  product. NDAs are used by CDER (center for
  drugs) and BLAs are used by CBER (center for
  biologics)
• The BLA requires close scrutiny of the
  manufacturing process and facilities, because of
  the greater variability of biological products.
• The application will include complete reports on
  all studies including patient listings, analysis
  according to the original statistical analysis
  protocol (SAP) as well as any exploratory
  analyses.

24
Paperwork (6)Marketing Applications

• Full safety analyses
• Listing of all adverse events
• By organ system
• By frequency
• Since neither FDA nor sponsors can define what
  events will occur a priori, such analyses tend to
  be exploratory. Some direction may be found in
  other drugs of the same class (e.g., a new statin
  or bet blocker, a new vaccine, etc.)

25
Regulators needs/wants

• Well-written good grammar, concise,
  well-organized
• Correct relevant tables, proper statistics
  (e.g. no women in prostate cancer denominators,
  etc.) no adults in tables referring to pediatric
  applications
• Proper number of digits (no spurious precision)
• Data set noted as source for each table/graph or
  analysis

26
Regulators needs (2)

• Justify any statements data should demonstrate
  this.
• Be open and honest about the data
• If there are problems, discuss them show the
  warts as well as the beauty.
• Do not lie about the data if an analysis is
  determined after the data has been locked, it can
  be treated as exploratory. FDA will usually not
  accept discovered hypotheses.

27
Research paper vs. Marketing Application

• In research, the goal is to have a good paper
  published in a good journal. The author(s) may
  contact the editor to ascertain interest in the
  topic. The paper itself will rarely contain the
  data and the analysis will not be reproduced by
  referees.
• For a marketing application, the sponsor will
  have been working with the regulatory agency for
  years and will submit all data. The analysis
  will be reproduced by the reviewers.

28
Research vs. Marketing (2)

• For a marketing application, FDA and sponsor may
  have agreed on some points. These generally
  arent binding on either party. It is similar
  to the journal in either case, the journal may
  decide it isnt appropriate for the journal the
  regulatory agency may decide it isnt adequate
  for licensure if the data dont show convincing
  evidence of efficacy
• The ultimate goal in regulation is to write a
  clear, good label for the product that the
  prescriber and public can understand

29
Reasons a marketing application may fail to be
approved

• The application is not reviewable data arent
  adequate, application is incomplete (sections
  missing)
• Data do not show evidence of efficacy
• There are safety concerns even if efficacy is
  demonstrated in one case I know of, there were
  serious safety concerns but a blinded review of
  the data indicated that these were not the
  problem the investigators thought they were.

30
Options if Marketing Application is not Approved

• Respond to criticisms
• Appeal to Office Director or Center Director
• Appeal to FDA Ombudsman
• Have advocacy group place pressure on FDA
• Have congressman put pressure
• The last two are rarely effective since science
  usually has left the debate
• Conduct a new study

31
Understanding FDA language

• Must is directive no options to do otherwise
• No means no
• Recommend is not directive you should discuss
  your plans in detail
• Should is strong, but less directive than must.
• Generally, FDA will not comment on something
  without reviewing the data
• Show clinical benefit usually means they are not
  interested in means.

32
FDA language(2)

• Time issues
• What happens at k weeks is different from what
  happens by k weeks or within k weeks
• If confused, ask!
• In reporting, define what you mean.
• clinically meaningful improvement must be
  carefully defined and not rely on a physicians
  impression
• Be careful about interpreting FDA words wanting
  a sensitivity analysis means FDA wants to be sure
  small changes in assumptions dont affect the
  conclusions.

33
FDA language (3)

• FDA will often look at subgroups to ensure the
  findings are robust. This is not doing lots of
  subgroup analyses to find something it is more
  looking at various groups to ensure a finding is
  real.
• In one study, a sponsor had found an effect, but
  FDA did some subgroup analysis and found 3 of 4
  sites had no effect and one had a major effect.
  Subsequent investigation found that the study
  coordinator for the site in question had
  unblinded the randomization and gave the
  treatment to patients she thought would have a
  better chance of benefit.

34
Interacting with Regulators

• Dont ignore advice the regulators have likely
  seen similar products and know pitfalls
• Make your submissions clear ensure your
  analysis answers the questions fully. If they
  differ from advice, explain why
• Ensure your data are complete and consistent
• Have a plan for dealing with missing values
• Always present the pre-specified analyses and any
  others you have done.
• Analyses not pre-specified will be considered
  exploratory

35
Interacting with Regulators (2)

• In one example, I had been working on a 2 part
  model theory and a sponsor had a study that I
  thought was appropriate for the application.
• I suggested it and the regulatory affairs person
  said well do it!
• I responded that the sponsor should check it out
  to see if it was appropriate.
• A week or two later, the statisticians had run a
  small simulation study and decided that a
  Mantel-Haenszel test would work better.

36
Interactions with Regulators (3)

• Communicate with regulators
• Some sponsors ban such communications unless a
  regulatory affairs representative is present or
  on the phone can result in delays
• Often the regulatory affairs person does not
  understand the statistical details being
  discussed
• This is a message for the regulatory affairs
  people dont delay work to maintain your
  control trust your statisticians and other
  staff.

37
Interactions with Regulators (4)

• Working within a system
• Concurrent documentation of procedures and
  studies
• Include data cleaning and editing procedures
• Data analysis methods and formulas
• SOPs for lab tests and all measurements of
  patients
• Normal ranges for key variables
• How various labs were standardized

38
Interactions with Regulators (5)

• Definitions
• Treatments including dose, schedule, route
• Population being studied
• Null and alternative hypotheses
• Tests to be conducted
• Size of tests
• Power of tests at alternatives
• Guidances and Regulations, ICH

39
Contents of IND

• For a phase 3 trial
• Design of study often involves two group
  comparisons superiority vs. non-inferiority
• Inclusion and exclusion criteria
• Outcomes (pre-specified!), how measured, how
  standardized across investigators
• Covariables (not too many remember label must
  be fairly general) phase 3 is NOT the time for
  variable selection
• The covariates should be predictive of outcome,
  not of imbalance.

40
IND (2)

• Sample size computation often its good to give
  several alternatives for different size, power
  and effect sizes. Give details
• Distinguish between non-inferiority and
  superiority trials/analysis

41
IND (3)

• Assumptions involved in analysis
• How will missing values be handled (maybe have
  several methods)
• I prefer not using LOCF
• Multiple imputation
• Model based methods likelihood, mixed models,
  GEE
• Details of randomization within site, balancing
  (minimization)
• Proposed analysis equations, references,
  justification of effect size based on pilot
  studies, the literature, etc.

42
IND (5)

• Modifications of a study
• Its almost always done as details change e.g.,
  recruiting is slow, so inclusion criteria may be
  broadened (age, disease inclusion, extending
  duration of recruitment)
• SAP can be changed as long as data are still
  blinded to sponsor.
• Changing endpoints is possible early on, but
  after data are complete, changing becomes suspect
  even if sponsor swears they havent peeked.

43
IND (5)

• Documenting work- YES
• Data cleaning and editing steps if programs are
  used, give them regulator may not examine, but
  can be useful
• Acceptable ranges for covariables and endpoints
• Handling if out of range (go to site and check?)
• Handling inconsistencies in data
• Software used some differences due to different
  algorithms e.g. SAS offers several definitions
  of percentiles, Stata does not
• Distinguish between exploratory and confirmatory
  generally not helpful to have p-values with
  exploratory analyses

44
Meeting the Regulators

• Prepare a briefing book describing issues to be
  discussed
• Dont bring up new results at the meeting as the
  FDA will say to submit the information and they
  will respond. Extreme example of a consultant
  presenting new, un-reviewed data at an advisory
  committee meeting.
• FDA will answer questions, often close to the
  meeting date making it difficult to reply to
  their concerns

45
Meetings (2)

• Useless question
• Does the FDA agree that these results support
  licensure?
• Better to ask what additional analyses are
  needed to support licensure?
• More helpful discussions related to schedule of
  submissions, etc.

46
Meetings (3)

• Type A meetings when drug development is
  stalled and sponsor cant proceed without FDA
  input e.g., clinical hold (need to know what
  is needed to remove the hold), or to resolve a
  dispute between sponsor and FDA, or for a Special
  Protocol Assessment.
• FDA will schedule a type A meeting within 30 days
  of receipt of a written request

47
Meetings (4)

• Type B meetings occur at natural junctures during
  the course of drug development e.g. pre-IND,
  end of phase 1, end of phase 2-pre phase 3,
  pre-NDA/BLA
• Pose clear questions based on interaction with
  FDA, problems observed during study
• Type C meetings are all other meetings regarding
  the development and review of a product.

48
Meetings (5)

• Regulators want to approve products but only on
  the basis of convincing data.
• Sponsors will discuss issues before the meeting
  (dress rehearsals)
• The FDA review team will also discus issues in
  each of the areas safety, efficacy, quality
• Note that FDA reviewers are privy to information
  from other studies in the same class and comments
  may be related to problems observed in other
  products. The FDA cant disclose specifics, but
  sponsors should be aware that issues mentioned
  may be relevant to them.

49
A Few Statistical Issues

• Recent developments and methodological research
  of importance to FDA
• Incorporating Bayesian methods into drug approval
  informativeness of priors. CDRH has accepted
  these for years.
• Genomics large number of variables, relatively
  few patients. What standards should be used for
  approval? License a process or a product? Seems
  to suggest using theory rather than clinical data

50
A Few Statistical Issues (2)

• Selecting the margin for non-inferiority studies
  (Im not sure where this is at present)
• Robust methods for complex statistical models
  (e.g., hierarchical models for non-normal data)
• Follow on Biologics
• Biologics equivalent to generics since
  Biologics are much more variable than drugs, many
  tough issues
• Adaptive Methods

51
Conclusion

• There can be few areas where the discipline of
  statistics is conducted with greater discipline.
  Pharmaceutical statisticians may be engaged in
  work that may sometimes involve routine
  calculations, but the regular application of
  statistical principles to produce high quality
  experiments, data and analysis promotes a
  professionalism that can itself be a source of
  satisfaction.
• Stephen Senn (2000 The Statistician)