Lessons Learned in Integrating Risk Management and Process Validation

1
Lessons Learned in Integrating Risk Management
and Process Validation
Medical Device Congress Harvard March 2007 Jim
Handzo Senior Manager QA Innovative Spinal
Technologies Fran Akelewicz Principal Practical
Solutions
2
Agenda

• Optimizing the validation activities sequence
  using risk management
• Determining the statistical approach and
  acceptance criteria based on risk
• Maximizing test plans and resources

3
Approach

• For illustration purposes we will use an example
  of
• Safety product
• (functional approach)
• New feature -a collar with a hinged rectangular
  plastic guard which will clip over the needle
  after use to prevent the user from getting stuck
  with the needle

4

• Optimizing the validation activities sequence

5
Key questions for process validation

• What are the important design characteristics?
• Where and how are they impacted in the process?
• What resources do I need?
• What do I do first?
• How should I conduct the validation?
• What do I do if I dont like the answer?

6
What are the important design characteristics?

• Product inputs to process validation
• Key design features
• Residual risk profile (dFMEA)
• Controls for the risk-prioritized design features

7
What are the important design characteristics?

• Product Inputs (User and Patient)
• Key Design Features
• Tools
• Customer Requirements Matrix
• Design Requirements Matrix

8
What are the important design characteristics?

• Product Inputs (User and Patient)
• Customer Requirements Matrix
• Focused on user needs
• Focused on business needs

9
What are the important design characteristics?

• Product Inputs (User and Patient)
• Design Requirements Matrix
• Focused on risk analysis

10
What are the important design characteristics?

• Product residual risk profile
• Without safety feature

X
11
What are the important design characteristics?

• Product residual risk profile
• With safety feature

X
12
Where and how are key design features impacted in
the process?

• Process Inputs
• Determine process control points that potentially
  impact the design features
• Are process control points
• One to one?
• One to many?
• Many to one?
• Determine residual risk impact (pFMEA, FTA, etc.)

13
Where and how are key design features impacted in
the process?

• Determine various process points that potentially
  impact the design feature

14
Where and how are key design features impacted in
the process?

• Determine if the control points are
• One process point to one design feature
• One process point to many design features
• Many process points to one design feature

15
Where and how are key design features impacted in
the process?

• Use tools to characterize process outputs vs.
  design features
• Design of Experiments (DOE)
• Edge of Failure (EOF) Studies
• Ranging Studies
• Mean Time between Failures (MTBF)
• Cpk vs. Ppk
• Determine process residual risk
• Process Map pFMEA
• FTA
• Waterfall with Feedback Loop
• EN 60601-1-4 PEMS V Diagram

16
Where and how are key design features impacted in
the process?

• Process residual risk profile
• Changes probability, not severity

X
17
What resources do I need?

• Dependent on type of product and/or process being
  evaluated
• Electrical Engineer
• Software Engineer
• Reliability Engineer
• Quality Engineer
• Commensurate with the degree of residual risk

18
What do I do first?

• Develop Process Validation Priority Risk Chart
• Similar methodology to Product Risk Chart
• Process effects on product are rated based on
  their impact
• Safety
• Performance
• Acceptance
• Compliance
• Verification
• Can also be used to develop a company Validation
  Master Plan

19
Process Map

• Formation and attachment of needle guard with
  collar attachment

20
What do I do first?

• Process Validation Priority Risk Chart

21
What do I do first

• Apply to
• Entire processes
• Sub-processes (then add)

22
Where should validation testing occur?
23
What do I do if I dont like the answer?

• Have a Plan B
• Identify assignable causes
• Justify discounting results with great care
• Revisit assumptions made during validation setup
• Revisit probabilities assigned to key design
  features
• But you cannot change severity unless you change
  the design
• Define contingencies in acceptance criteria

24

• Determining the statistical approach and
  acceptance criteria based on risk

25
Determining acceptance criteria

• Document in company policy
• Similar to product risk chart
• Key elements derived from product risk chart
• Severity drives confidence levels
• Confidence levels and probability drive sampling
  plans
• Qualitative probabilities (Frequent, Probable,
  etc.) must be expressed quantitatively
• Sampling plans should be discriminating enough to
  detect defects that relate to high
  severity/frequently occurring hazards

26
Determining acceptance criteria

• Acceptance Criteria Risk Chart

27
Sampling plan determination

• Determine the operating characteristic (OC) curve
  of selected sampling plan
• Incorporate quantitative probabilities
• Determine acceptable errors
• Type I (Producers Risk) we say its bad when
  its not
• Type II (Consumer Risk) we say its good when
  its not
• Does it match residual risk and probability?

28
Sampling plan determination

• Determine how much risk (error) you can tolerate
  in your estimates
• Attributes
• Percent defective error tolerance
• Variables
• Error tolerance of the average
• Must constantly weigh sample size vs. estimate
  risk (error)
• Economics vs. certainty

29
Sampling plan determination
30
Sampling plan determination

• Use appropriate probability distributions when
  determining sampling plans
• Binomial
• Large populations
• Sampling with replacement
• Hypergeometric
• Small populations
• Sampling without replacement

31
Sampling plan determination
32
Sampling plan determination
33
Sampling plan determination

• Sampling Calculators available for download at
• www.devicecongress.com/agenda/day2.html
• (Once on the website, go to Track A, 410pm
  session, for links to presentation materials.
  Calculators are in the excel document.)

34

• Maximizing test plans and resources

35
Maximizing Test Plans and Resources

• Identify resources based on risk
• Prioritize the process validation activities
  based on risk
• Choose the acceptance criteria
• Choose the appropriate sample methodology to
  minimize the business risk

36
Change Control

• Once process validation is completed you have
  established a state of control
• Any changes made should consider
• Impact to product design risk (design
  verification testing design validation testing)
• Impact to process design risk (equipment design
  decisions)
• Impact to control of risk within process

37
Conclusions

• Risk management in process validation
• Provides the opportunity to
• challenge existing assumptions
• think out of the box
• Acknowledges the risks that we have taken
  (product and business)
• Provides a common language
• Provides a framework for consistency
• Formalizes what we have done
• Contributes to safer products

38
Questions
39
Thank You !
Jim Handzo Innovative Spinal Technologies 1 508
452-3517 jhandzo_at_istspine.com www.istspine.com
Fran Akelewicz Practical Solutions 1
215 337-9238 franakelewicz_at_comcast.net www
.practicalsolns.net