Lessons about risk-benefit assessment from OMERACT

1
Lessons about risk-benefit assessment from OMERACT

• Jasvinder Singh, MBBS, MPH
• Associate Professor of Medicine, UAB School of
  Medicine VA Medical Center, Birmingham, Alabama
• Mayo Clinic School of Medicine, Rochester, MN

IMMPACT-XIV, Arlington, VA June 16-17, 2011
2
Acknowledgement

• Entire OMERACT Executive Developed OMERACT 3 X 3
• Slides Dr. Maarten Boers
• Comments Drs. Maarten Boers Lee Simon

3
Overview

• Definitions
• Summary and Discussions at 2008 OMERACT Drug
  Safety Summit
• OMERACT approach
• Benefit Risk Action Team (BRAT) approach
• Pharmaceutical Research and Manufacturers of
  America
• Questions and potential solutions
• The Future

4
Definitions Risk and Benefit

• Risk
• Oxford a situation involving exposure to
  danger
• Merriam-Webster possibility of loss or injury
• Medical context an effect that is harmful to the
  patients or publics health and which can relate
  to safety, efficacy or quality of a product.
• Benefit
• Oxford an advantage or profit gained from
  something
• Merriam-Webster something that promotes
  well-being

5
Overview

• Definitions
• Summary and Discussions at 2008 OMERACT Drug
  Safety Summit
• OMERACT approach
• Benefit Risk Action Team (BRAT) approach
• Pharmaceutical Research and Manufacturers of
  America
• Questions and potential solutions
• The Future

6
OMERACT Drug Safety Summit (2008)

• Assigning causality
• Bradford-Hill criteria biological plausibility
  Bayesian methods
• No agreement on magnitude of frequency, i.e.,
  when does something become a signal
• Utility of large trials to define risk
• Large, simple trials such as prospective
  randomized open blinded endpoint (PROBE)
• Randomization to two effective treatments
• Alternative observational studies with
  confounding by indication and channeling bias

7
OMERACT Drug Safety Summit (2008)

• Utility of metanalyses of RCTs
• Multiple small RCTs ?statistically equivalent to
  a large single trials with same denominator
• 200-300 events needed for credible estimates
• Role for indirect comparisons- unclear
• Observational studies may be included
• Selection and confounding bias issues
• Outcome definition and measurement and follow-up

8
OMERACT Drug Safety Summit (2008)

• Postmarketing Surveillance
• Essential for drug safety evaluation
• All drugs in a class should be analyzed
  identically and concurrently
• Desired components
• More than one defined population
• Full protocol with outcomes assessed at regular
  intervals
• Concurrent control subjects
• Outcomes- patient-recoded documented in
  electronic records
• Pre-defined hypothesis
• Oversight by FDA and data management council

9
OMERACT Drug Safety Summit (2008)

• Utilities of Drug registries
• Advantages real world experience, long-term FU
• Disadvantages lack of comparator group, data
  quality, patients not obligated to follow
  protocol, loss to follow-up
• Pharmacoepidemiologic studies
• Cohort studies generally better than case-control
  studies in providing risk estimates
• Issues need to addressed
• Misclassification bias
• Validation of outcomes
• Guidelines for confounding bias and methods for
  adjustment

10
OMERACT Drug Safety Summit (2008)

• Simple versus complex metric
• Simple OMERACT 3 x 3
• Benefit and risk categorized into 3 levels each-
  none, substantial, (near) remission or death
• Complex multicriteria decision analysis
• Complex Frameworks Quantitative methods
• Decision analysis method
• Conjoint analysis
• Incremental net-benefit
• BRAT approach

11
OMERACT Drug Safety Summit (2008)

• GRADE approach
• Classifies evidence into 4 levels high,
  moderate, low and very low based on
• Study design, weaknesses, special strengths
  (large effect, dose response)
• 2 recommendations strong and weak
• RCTs always starting at high and non-RCTs
  starting at low?
• Other models of Risk Nontreatment
• Risk and value of available treatment versus
  nontreatment options
• Type, intensity and severity of adverse event
• Acute, subacute or chronic manageable, treatable
  or not

12
Overview

• Definitions
• Summary and Discussions at 2008 OMERACT Drug
  Safety Summit
• OMERACT approach
• Benefit Risk Action Team (BRAT) approach
• Pharmaceutical Research and Manufacturers of
  America
• Questions and potential solutions
• The Future

13
OMERACT 3 x3

• Create three categories of harm and benefit
• Benefit none/minimal, major, (near) remission
• Harm none/minimal, major, (near) death
• Key components of this approach
• Harms versus benefits

14
Problems in benefit-risk assessment

• measurement of benefit is specific and evolved,
  but measurement of risk (or harm) is generic and
  more primitive
• Benefit and harm not placed on a single scale

15
Conceptual problems

• placing value judgments on scientific facts
• values will vary depending on the perspective of
  the assessor
• comparing benefit and risk involves
• comparing apples and pears
• trading off (and discounting) long-term against
  short-term effects
• assessing multiple benefits and risks
• assessment mostly driven by the need to make
  decisions, whereas most research is
  truth-driven

16
Practical problems

• ...placing value judgments on scientific facts
• Requires qualitative research
• ...multiple comparisons and trade-offs
• Difficult science
• Requires decision analysis
• ...mostly driven by decision-makers
• Low speed of developments
• Literature difficult to access

17
OMERACT 3 x 3 Table
18
Examples

• COBRA trial
• VIGOR trial

19
COBRA trial

• RCT comparing combination therapy (COBRA)
  including step-down high-dose prednisolone with
  single drug therapy (sulfasalazine, SSZ) in
  early rheumatoid arthritis
• Main result showed COBRA to be more effective and
  result in less side effects than SSZ

20
3 x 3 COBRA
Major benefit Disease activity score
lt3.7 (near) Remission Disease activity score
lt2.4 Severe Harm treatment discontinuation due
to serious harm, loss of efficacy or both (near)
death death or severe inacapacitation
21
2 x 2 COBRA
ARR Unqualified success 80-54 26 NNT
4 ARR Unmitigated failure 5-14 9 NNH ?
22
Success/failure
Unqualified success
Unmitigated failure
23
VIGOR trial

• large industry-sponsored trial to compare
  high-dose rofecoxib with naproxen in RA
• object was GI safety, not efficacy
• rofecoxib proved safer for GI, but less safe for
  cardiovascular events
• Merck withdrew the drug voluntarily amidst much
  controversy
• no access to individual patient data
• safety and efficacy assumed independent

24
2 x 2 VIGOR
Data from original report and from Strand V.
Lancet 20073702138-51. Major harm is defined as
cardiovascular event gastrointestinal event
death.
25
Success/failure
Unqualified success
Unmitigated failure
26
Overview

• Definitions
• Summary and Discussions at 2008 OMERACT Drug
  Safety Summit
• OMERACT approach
• Benefit Risk Action Team (BRAT) approach
• Pharmaceutical Research and Manufacturers of
  America
• Questions and potential solutions
• The Future

27
STEP 1 Defining the decision context

• Specifying the therapeutic context
• Explicit definition of product, indication,
  target population, formulation, dosage and
  contraindications
• Specifying the comparator
• Standard of care, best in class, watchful
  waiting, placebo and nonpharmacological
  intervention
• Defining the time horizon
• Duration of exposure and time period over which
  benefit-risk events are measured
• Specifying the stakeholder perspective
• Sponsor, regulators, patients, physicians

1Coplan PM Clin Pharma Therap 2011 892 312-15
28
Step 2 Identifying benefit and risk outcomes
Building the value tree

• Identifying pool of candidate outcomes for the
  assessment
• Deciding which outcomes to include or exclude
  from the framework
• Documenting all critical assumptions for these
  inclusion and exclusion decisions
• Value tree is a visual hierarchical presentation
  of key ideas, values or concepts

1Coplan PM Clin Pharma Therap 2011 892 312-15
29
Step 3 Identifying data sources for the framework

• Information may come from multiple data sources
• A repository of all data, called data source
  table is kept

1Coplan PM Clin Pharma Therap 2011 892 312-15
30
Step 4 Customizing the framework

• Customization done based on data quality and
  limitations
• Study end-points are organized in value tree at 2
  levels
• Body system category of the benefit or risk
• The end-point measured in studies
• Approaches to capturing level of severity of
  outcomes identified and the value tree is enhanced

1Coplan PM Clin Pharma Therap 2011 892 312-15
31
Step 5 Assess relative importance of different
outcomes

• Assess relative weights
• Stated choice methods
• QALYs
• Utilities
• Value functions
• Application of weighted approach in analysis
• Net clinical benefit
• Number needed to harm (NNH)
• Multicriteria decision analysis

1Coplan PM Clin Pharma Therap 2011 892 312-15
32
Step 6 Display and interpret key benefit-risk
metrics

• Key risk benefit table
• Flexible table summarizing the key information
  needed to quantify outcomes in the value trees
• Additional items to enhance key table
• Heat map color coding
• Embedded graphs

1Coplan PM Clin Pharma Therap 2011 892 312-15
33
BRAT framework Example Acute treatment of
headache phase of migraine
1Levitan Clin Pharma Therap 2011 892 217-234
34
Example 2 BRAT framework
1Levitan Clin Pharma Therap 2011 892 217-234
35
Example 2 Key Benefit-risk table
1Levitan Clin Pharma Therap 2011 892 217-234
36
Overview

• Definitions
• Summary and Discussions at 2008 OMERACT Drug
  Safety Summit
• OMERACT approach
• Benefit Risk Action Team (BRAT) approach
• Pharmaceutical Research and Manufacturers of
  America
• Questions and potential solutions
• The Future

37
Questions and potential solutions

• How to come up with method (s) for risk-benefit
  assessment that is/are
• Easily understood by regulators, patients and
  physicians
• Take into account non-treatment or alternative
  treatments
• Allow sensitivity analyses?
• How to mandate and conduct efficient
  postmarketing surveillance studies?
• Requirement by FDA and EMEA etc.
• Funding
• Preventing bias
• Design and follow-up methodology

38
Questions and potential solutions

• Can efficacy trials be better designed to allow
  better short-term risk-benefit analysis?
• What steps must be taken to use large databases
  to conduct postmarketing studies?
• Standardization of methods for reduction of bias
• Data integrity and privacy issues
• How to achieve an agreement on methods by major
  regulators (FDA, EMEA)?

39
Overview

• Definitions
• Summary and Discussions at 2008 OMERACT Drug
  Safety Summit
• OMERACT approach
• Benefit Risk Action Team (BRAT) approach
• Pharmaceutical Research and Manufacturers of
  America
• Questions and potential solutions
• The Future

40
Future

• Collaborative networks of academia, regulatory
  agencies, patients and pharma
• Use of preferences, values, cost and delay in
  drug approval (risks of alternatives) in
  risk-benefit analysis
• Universally applicable methods
• Ability to vary values, weights and perform
  sensitivity analyses
• Utilize non-randomized studies
• Design and conduct useful large simple
  comparative effectiveness trials

41
Thank You