Details of FDA

1
Details of FDAs Pilot Risk-Ranking Model for GMP
Inspections

• Nga Tran, Dr.P.H., Exponent
• Brian J. Hasselbalch, FDA
• July 2004

2
Application of Risk Ranking

• Regulatory programs
• EPA, Cal EPA
• USDA
• Risk management
• DoD
• Industry

3
Examples of Risk Ranking Models
4
EPA Waste Minimization Prioritization Tool (WMPT)

• A regulatory decision tool
• Foundation for the EPAs Resource Conservation
  and Recovery Act Persistent, Bioaccumulative, and
  Toxic (PBT) List of chemicals.
• Framework of expert judgment
• Identify chemicals or emissions of potential
  concern
• Using key physical-chemical properties and
  associated cutoff criteria.

5
EPA Waste Minimization Prioritization Tool (WMPT)
-- Scoring system
6
The TRI Relative Risk-Based Environmental
Indicator (Bowves et al, 1997)
7
A Hierarchical Framework of Scoring and Ranking
Methodology, Pennington and Yue (2001)
8
DoD

• Comparative risk pre-deployment
• Rank locations based on potential chemical risks
• Risk score is a function of volume and inherent
  chemical toxicity
• Comparative risk during deployment
• Ranking and comparing disparate risks
• Quick/simple tools

9
Risk Assessment Matrix U.S. Army Field Manual
3-100.12.
  E Extremely high risk H High risk M
Moderate risk L Low risk
10
Risk Level Definitions (US CHPPM, Technical
Guidance-248)  
11
Hazard Severity Ranking Chart for Military
Deployments (US CHPPM, Technical Guidance-248)

12
Chemical Hazard Probability Ranking Chart for
Military Deployments (US CHPPM, Technical
Guidance 230)
13
Consumer Product Example

• Ranking to target products for which additional
  toxicity testing of High Production Volume
  chemical is needed.
• Laundry detergent, consumer care products, etc..
• Many products, limited resources
• Ranking based on exposure to consumer products
• High end product use frequency (e.g., number of
  times use per day) as surrogate for exposure

14
Example Consumer Product Exposure Model for
Ranking Dermal Direct Contact
15
Ranking of Consumer Product by Exposures to High
Production Volume Chemical A
 
16
Food Microbiological Hazard Example Van Gerwen
et al (2000)

• SIEFE Stepwise and Interactive Evaluation of
  Food Safety by an Expert System.
• A decision support system for microbiological
  risk assessment for food products and their
  production processes
• Risks are first assessed broadly, using order of
  magnitude estimates, i.e. level 1 risk
  assessment.
• Characteristic numbers are used to quantitatively
  characterize microbial behavior during the
  production process to highlight the major
  risk-determining phenomena 

17
Categorization of the probability of having
problems, growth and inactivation by
characteristic numbers
PC (probability characterization) SC (step
characterization) OC is the characteristic
number for occurrence characteristic HC for
health problem characterization.
18
Food risk ranking model (Ross and Sumner, 2002)

• Tool to compare food-borne risks for ranking and
  prioritizing risks from diverse sources.
• 11 questions relating to 3 main factors
• severity of hazard
• likelihood of disease causing dose of hazard
  being present in a meal
• probability of exposure to the hazard

19
(No Transcript)
20
Comments
21
Comments
Weights
22
USDA- Food Safety and Inspection Service

• "Inspector Optimization System" (IOS)
• Basis of assigning the processing inspector
  workforce to provide for greater flexibility in
  targeting inspection resources to areas of
  greater public health risk and concern
• Report to Congress on Risk-Based Inspection to
  the United States House and Senate Appropriations
  Committees, March 2, 2001
• Each plant receives a Food Safety Hazard
  Coefficient (HC)
• Inherent hazards
• Expert Elicitation to rank inherent hazards

23
Food Safety Hazard Coefficient
Species variable designed to reflect the inherent
biological, chemical, and physical hazards in
meat and poultry arriving at inspected
establishments Process variable captures the
inherent hazards (biological, chemical, and
physical) of the establishments operations Take
into account how a process normally works, the
likelihood of fluctuations or deviations from the
norm, the effect these fluctuations or deviations
may have on hazards in the product, and the
potential resulting implications for public
health as the product leaves the establishment.
Volume variable facility size correlated with
the number of persons who consume food from the
establishment (exposure).
24
Expert elicitation

• To determine the species and process hazards
• Two groups of experts (species and processing)
• Government, academia, and industry.

Species Categories Used in Hazard
Elicitation Steers/Heifers     
Lambs/Sheep/GoatsVeal/Calves         Young
ChickensCows/Bulls          Young TurkeysMarket
Hogs       Older PoultrySows/Boars        
Ducks/Geese
25
Process Categories Used in Hazard
Elicitation Raw product, ground             
Heat-treated, shelf-stable product Raw product,
not ground        Heat-treated, but not fully
cooked, not shelf-stable Slaughter
                               Fully cooked, not
shelf-stable product Thermally processed,
             Not heat-treated, shelf-stable
product commercially-sterile Secondary
inhibitors, not shelf-stable product
26
(No Transcript)
27
The CDER-OC process

• Began one year ago
• Internal experts from CDER, CVM, CBER, and ORA
  (Risk Mgmt Workplan WG)
• Generated a list of risk factors that are
  descriptors of site risks for consideration in
  development of site selection model
• Qualitatively valued High, Medium, Low

28
Examples of risk factors
Factor Risk
Administered through membranes Medium
Aseptically produced finished dosage forms High
Dermatological OTC liquids/creams/ointments/powders with no daily dose limit Low
Excipients Low
Aseptic fill using isolator (self-contained isolators and form-fill-seal), non-terminal sterilized Medium
Aseptic sterile filtration and filling (non-isolator, non-terminal sterilization) High
Assembly/fabrication processing (as opposed to batch processing) High
Cartoning / packaging Low
Large number of approvals/time Low
Complaints received suggesting Class I or Class II health hazard situations High
Contract micronizers Medium
First-time applicants/filers Medium
29
Develop Risk Ranking Framework

• High level risk management begins with
  organization
• Categorized list of risk factors into 3
  components
• Product
• Process
• Facility

30
Categorization of risk factors, e.g.
Component Factor
Product Administered through membranes
Aseptically produced finished dosage forms
Dermatological OTC liquids/creams/ointments/powders with no daily dose limit
Excipients
Process Aseptic fill using isolator (self-contained isolators and form-fill-seal), non-terminal sterilized
Aseptic sterile filtration and filling (non-isolator, non-terminal sterilization)
Assembly/fabrication processing (as opposed to batch processing)
Cartoning / packaging
Facility Large number of approvals/time
Complaints received suggesting Class I or Class II health hazard situations
Contract micronizers
First-time applicants/filers
31
A RISK-BASED FRAMEWORK FOR PRIORITIZING SITES FOR
cGMP INSPECTION
SITE RISK POTENTIAL
PRODUCT
PROCESS
FACILITY
Site risk potential (SRP) A function of the risk
potentials for the PRODUCT, PROCESS and FACILITY
component risk factors. The SRP is a weighted
combination of the PRODUCT, PROCESS and FACILITY
risk potentials Semi-quantitative risk potentials
represented as high, medium, or low, and
are numerically discrete variables in
calculations.
32
Drilling Down to Component Factors

• Selected factors for product, process and
  facility components
• Assigned weights
• Empirically derived
• Expert judgment
• Developed logical algorithm to combine factors to
  calculate a site score The Model

33

• Drilling Down and Populating the Product,
  Facility, and Process Components

34
Product Component Factors
SRP (Site Risk Potential)
Site-Product Score
Site-Process Score
Site-Facility Score
Product Scores
1997 2003 Recall
Intrinsic Factors
STERILITY
Frequency
MEDICAL GAS
Severity HHE Class III, II, I
RX/OTC
35
Product Component Data

• Data source FACTS (Field Accomplishments and
  Compliance Tracking System)
• Site information/identifier
• Product codes
• CDER Recall Database
• Product codes
• Rx/OTC and Classification of Hazard
• Years
• Recall 1997-2004
• Product 2000-2004

36
Recall Weight Matrix
37
Product Categorization

• Need to link recall data to sites
• Indirect linkage -- correlate product codes in
  recall data with product codes in site database
• Recall weights assigned to Recall-FACTs
  harmonized product codes

38
Examples of Product Code Correlation
FACTS Process Indicator Code letter Recall System Product Codes Recall Code Description General Category
Q OAER Aerosol, Inhalation/Systemic inhaled
S BULK Bulk Pharmaceutical API/Excipient
R OPWRR Oral Solid For Reconstitution oral, solid, immediate release capsules/powders
A OTABE Tablet, Effervescent oral, solid, immediate release tablets
A OTABS Tablet, Sublingual/Buccal oral, solid, immediate release tablets
C TEMR Tablet, Extended/Modified Rel oral, extended-release
B OTABX Tablet, Extended Release oral, extended-release
E OCAP Oral Capsule oral, solid, immediate release capsules/powders
E OCAPH Capsule, Hard Gelatin oral, solid, immediate release capsules/powders
F OCAPX Oral Capsule Extended Release oral, extended-release
L TSPY Topical Spray topical, dermal
L TSUS Topical Suspension, Mixture topical, dermal
L TTCT Topical Tincture topical, dermal
39
Facility Component Factors
SRP (Site Risk Potential
Site-Product Score
Site-Process Score
Site-Facility Score
History of Compliance and Violation
Volume (est.)
History of inspection
40
Facility Component Data

• Data source FACTS (Field Accomplishments and
  Compliance Tracking System)
• Years 2000-2004
• All sites, foreign and domestic

41
Process Component Factors
SRP (Site Risk Potential)
Site-Process Score
Site-Facility Score
Site-Product Score
Relevant process controls and risk mitigating
factors
Relevant inherent process risk factors
Producti, facilityj
42
Process Component Data

• No data, but do have staff with experience and
  knowledge
• ? expert elicitation survey
• Elicitation survey drafting began Nov. 2003
• Inter-Center/ORA workgroup effort
• ORA CVM CBER CDER CDER OC

43
Expert Elicitation Survey Challenges

• What are the relevant process-related risk
  factors? (i.e., sources of variability and poor
  quality)
• What, if any, unit operations are more liable to
  a loss of control or at risk to contamination?
• What products? Answers are product dependent
• Large number of products
• Identify mutually exclusive category of
  products -- tradeoffs in number of product
  categories and responder burden

44
Starting point methodology... ISPE Baseline
Pharmaceutical Engineering Guide, vol. 2, Oral
Solid Dosage Forms
45
Defining product types and unit operations

• Product types
• Several CDER/agency coding schemes too much
• Types selected are representative of most product
  types in the market
• Codes grouped into product types with similar
  unit ops
• Distinguish high/low actives
• Unit operations
• WG member knowledge
• Remington Science and Practice of Pharmacy, 20th
  edition
• Modern Pharmaceutics, 3rd edition
• Pharmaceutical Process Validation, 3rd edition
• ISPE Baseline Pharmaceutical Engineering Guide,
  vol. 2, Oral Solid Dosage Forms (1st edition,
  1998)

46
Product group DRLS / EES / FACTS profile class codes Profile class description
Solid oral immediate release CHG Capsules, Prompt Release
Solid oral immediate release CSG Capsules, Soft Gelatin
Solid oral immediate release TCM Tablets, Prompt Release
Solid oral modified release TTR Tablets, Extended Release
Solid oral modified release CTR Capsules, Modified Release
Solid oral modified release TCT Tablets, Delayed Response
Powders POW Powders (Includes Oral and Topical)
Semisolids OIN Ointment, Non-sterile (Includes Cream, Jelly, Paste)
Semisolids SUP Suppositories
Liquids, nonsterile (solution / suspension) LIQ Liquids (Solutions, Suspensions, Elixers and Tinctures)
Liquids, sterile SVL Small Volume Parenterals (Lypholized)
Liquids, sterile SVS Sterile-Filled Small Volume Parenterals (Drugs)
Liquids, sterile SVT Terminally Sterilized Small Parenterals
Liquids, sterile LVP Large Volume Parenterals
Liquids, sterile SNI Sterile Non-Injectable
Biotech CBI Biotechnology Crude Drug
Biotech TRP Therapeutic Recombinant Products
Transdermal TDP Transdermal Patches
MDI ADM Aerosol Dispensed Medication
API synthesis CEX Plant/Animal Extraction Crude Drug
API synthesis CRU Crude Bulk Drugs - Non-Synthesized
API synthesis CSN Non-Sterile Bulk by Chemical Synthesis
API synthesis CSS Sterile Bulk by Chemical Synthesis
API fermentation CFN Non-Sterile Bulk by Fermentation Crude Drug
API fermentation CFS Sterile Bulk by Fermentation Crude Drug
47
(No Transcript)
48
PROCESS RISK FACTORS

• A) Process control
• To what degree does this unit of operation
  contribute to variability in quality of the final
  product?
• 2) How difficult is it to maintain this unit
  of operation in a state of control?
• 3) If a problem does occur, how reliable are
  the current detection methods?

• B) Contamination
• 4) Is this unit of operation more or less
  vulnerable to contamination from previous
  product?
• 5) Is this unit of operation more or less
  vulnerable to contamination from the environment?

49
Solid oral drugs, immediate release (i.e., tablets, capsules), "High Active" Solid oral drugs, immediate release (i.e., tablets, capsules), "High Active" Solid oral drugs, immediate release (i.e., tablets, capsules), "High Active" Solid oral drugs, immediate release (i.e., tablets, capsules), "High Active" Solid oral drugs, immediate release (i.e., tablets, capsules), "High Active" Solid oral drugs, immediate release (i.e., tablets, capsules), "High Active" Solid oral drugs, immediate release (i.e., tablets, capsules), "High Active"
Variables Variables Process control Process control Process control Contamination Contamination
Questions Questions 1) To what degree does this unit of operation contribute to variability in quality of the final product? 2) How difficult is it to maintain this unit of operation in a state of control? 3) If a problem does occur, how reliable are the current detection methods? 4) Is this unit of operation more or less vulnerable to contamination from previous product? 5) Is this unit of operation more or less vulnerable to contamination from the environment?
Answering Choices Answering Choices 1 minimal 2 minimal to moderate3 moderate to high4 high to very high5 very high 1 slightly difficult2 slightly to moderately difficult3 moderately difficult4 moderately to very difficult5 very difficult 1 very reliable2 very to moderately reliable3 moderately reliable4 moderately to slightly reliable5 slightly reliable 1 slightly vulnerable2 slightly to moderately vulnerable3 moderately vulnerable4 moderately to very vulnerable5 very vulnerable 1 slightly vulnerable2 slightly to moderately vulnerable3 moderately vulnerable4 moderately to very vulnerable5 very vulnerable
Units of Operation Measuring          
Units of Operation Mixing/Blending          
Units of Operation Wet Granulation          
Units of Operation Dry Granulation          
Units of Operation Milling          
Units of Operation Drying          
Units of Operation Compression (Tablet)          
Units of Operation Encapsulation (Hard Gel)          
Units of Operation Coating          
Units of Operation Pelleting          
Units of Operation Primary packaging          
50
The survey also asks experts to rank products
alone (not by unit of operation) by the general
areas of concern
51
Survey Status

• Delivery By e-mail
• 50 FDA experts participated
• reviewers from CDER
• senior CDER compliance staff
• senior ORA field staff
• 90 response rate
• Results being analyzed

52
Data analysis

• Exploratory Data Analysis (EDA)
• Product rank consistent with unit of operation
  drill down data correlation
• Developing process weights based on unit of
  operation drill down survey data
• K-mean cluster analysis
• Average rank per product category per question
• a) Process controls questions 1, 2 and 3
• b) Contamination potential questions 4 and 5
• Coefficient of Variance (CV) Weighted average
  rank per product category per question
• Principal component and fuzzy arithmetic for
  expert categorical data analysis

53
Process Component Factors
SRP (Site Risk Potential)
Site-Process Score
Site-Facility Score
Site-Product Score
Contamination
Process Controls
Variability in quality Survey question 1
Previous product Survey question 4
Maintain state of control Survey question 2
Environmental Survey question 5
Reliable detection Survey question 3
54
Plain language summary

• A site will tend to be less frequently selected
  for inspection if it
• has been inspected recently and/or relatively few
  previous violations of GMPs and/or a smaller
  volume of product (facility weight)
• makes non-sterile, OTC drugs, and/or other
  product types that are not associated with a high
  frequency of serious recalls (product weight)
• makes products estimated to be relatively
  straightforward to manufacture and not vulnerable
  to contamination (process weight)

55
Current and Future Risk Factors in Site Score
Modules Factors in Baseline SPR Score Factors in Baseline SPR Score Factors to Update SPR Score (including mitigating factors) Factors to Update SPR Score (including mitigating factors) Field Inputs to Update SPR Score Field Inputs to Update SPR Score
Product Rx x Recalls x Never covered? future
Product Sterile x Other Defects future    
Product Not medical gas x Complaint future    
Product Must work right future        
Product Therapeutic range future        
Product Higher intrinsic toxicity, e.g., enhanced risk mgt controls, require periodic patient monitoring future        
Process Expert Elicitation x Enhanced controls (e.g., PAT, process capability metrics) future New construc.? future
Facility Compliance history x     Corp. changes? future
Facility History of Inspection x
Facility Volume (est.) x      
Facility Successful practices study future
56
Questions to the Subcommittee

• Can you identify alternative approaches that
  would systematically prioritize manufacturing
  sites for GMP inspections?
• In what areas would additional data provide the
  most value added in prioritizing manufacturing
  sites for GMP inspections?
• Are there other metrics that should be
  incorporated, e.g., measuring process control?