A Road Map to COTS Computer System Validation

1
A Road Map to
COTS Computer System Validation based on a
HPLC, as example
Ulf Segerstéen Pharma Quality Europe AB
SARQ, 3rd of October, 2002
2
21 CFR Part 11 effective Incipit requires ....
3
Guidance Content

• GUIDANCE KEY ELEMENTS
• System Requirements Specifications (5.1)
• Documentation of Validation Activity (5.2)
• Equipment Installation (5.3)
• Dynamic Testing (5.4)
• Static Verification Techniques (5.5)
• Extent of Validation (5.6)
• Independence of Review (5.7)
• Change Control (5.8)
• SPECIAL CONSIDERATIONS COTS products and Internet

Example of Guidance Independence of Review
4
COTS (Commercial Off-The Shelf) products
5
New Trend - Partnership by-Outsourcing Validation
Activities Whats in it for the Customer ?

• Staying focused on his business - be more
  cost-effective.
• Staying on top of the requirements - stay
  compliant rather than spending time solving
  regulatory issues.
• Sharing the latest knowledge - dont be active in
  all fields and save time for the core business.
• Being prepared for whats to come - get resources
  when needed, dont let them be an added cost to
  the Products.
• Being part of the solution NOT part of the
  problem - give theorganization confidence to
  handle changes and challenges.
• Always having access to a Partner to discuss and
  solve issues with.

Whats in it for the Partner ?

• Client staying focused on his business makes
  easier for the Partner to support him.
• Client staying updated on the requirements gives
  the Partner flexibility to act more proactively.
• Sharing the latest knowledge makes the Client and
  the Partner understand each other, reducing
  misunderstandings and increasing efficiency.
• Being prepared for what is to come helps the
  Partner to keep prices down and gives the Client
  more accurate budget control.
• Being part of the solution, NOT part of the
  problem, helps both parties in solving and
  anticipating potential problems.

6
The Lifecycle Activities for a COTS, using a
Project perspective
SPECIFY
TEST
START UP
SUPPORT IMPROVE
PLAN TO BEPROACTIVE
CONTROL
7
The Lifecycle Activities for a COTS, cont.
SPECIFY
TEST
START UP
URS (PROCESS) EVALUATION VALIDATION
PLAN MASTER INDEX (MI)
USER GUIDE SYSTEM ORG. DOCs
VALIDATIONREPORT SOPS (Approved)
PROCESS Q FAT SAT TEST PLAN DRAFT SOPs
8
The Lifecycle Activities for a COTS, cont.

• User Requirement Specification based on
• the lab Process including the HPLC-equipment,
  HW-platform, SW-application, printer, expected
  filestorage and user interactions.
• requirements for compliance to CFR 21 Part 11
  (ER ES) and other applicable GxP regulations
• testable requirements

SPECIFY
URS (PROCESS) EVALUATION VALIDATION
PLAN MASTER INDEX (MI)
9
COTS End User Requirements Specifications
Define what you need (all factors)
Define what predicate rule needs
Define what Part 11 needs
10
The Lifecycle Activities for a COTS, cont.

• Evaluation based on
• the CS fulfilling the Process that it is to be
  used within.
• Risk Assessment Index, based on complexity of
  the CS and on the criticality for failure during
  the process.RISK MANAGEMENTRISK APPROACHRISK
  ACTIVITIES
• the Suppliers expected role, process and
  responsibilities during the Project for
  developing testing (FAT), installing and
  testing at the customers site (SAT), supporting
  the CS over time and license agreements.
• fulfillment of compliance to CFR 21 Part 11 (ER
  ES) and other valid GxP regulations
• referenced other Pharmaceutical customers using
  the system.

SPECIFY
URS (PROCESS) EVALUATION VALIDATION
PLAN MASTER INDEX (MI)
11
The Lifecycle Activities for a COTS, cont.
Evaluation of WHAT approach to take
GOLDEN RULE
VALIDATION EFFORT SHOULD BE MAXIMUN FOR HIGH
CRITICAL AND COMPLEX SYSTEMS. EFFORT MIGHT BE
REDUCED THROUGH AUDITS IN CASE OF SW STANDARDS
PROCESS
CRITICALITY
DATA
COMPLEXITY
SYSTEM
CATEGORY
12
Criticality Assessment

• High
• Software application whose features and
  functions have a direct impact on the quality,
  performance and efficacy of drug products
• Medium
• Software used for business process analysis,
  information and documentation systems that poses
  some business risk, or can have an indirect
  impact on drug products
• Low
• Packaged Software used for business purposes
  that poses no business risk

13
Risk Assessment Evaluation

• The Risk Assessment Index (RAI) is a rationale to
  evaluate the criticality and complexity of
  computerized systems.
• System Complexity is higher when system
• Performs detailed algorithm or calculations
• Interfaces with other computerized systems
• performs extensive data input checking
• processes numerous types of transactions
• requires extensive support to be maintained
• involves a large number of users
• GxP Criticality is an index of the impact of the
  system on pharmaceutical product or on raw data
  safety and traceability

System Complexity
RAI definition
GxP Criticality
(see next page)
14
Risk Assessment Evaluation

• that brings to RAI definition

15
RAI used for evaluating HOW to perform the
Supplier Evaluation
EVALUATION THROUGH REFERENCES
RAI0-2
RISKS
EVALUATION THROUGH EXPERIENCES
RAI3-4
REQUEST FOR INFORMATION
RAI5-6
3RD PARTY AUDIT
RAI7-8
SPECIFIC FIRM AUDIT
RAI9
COSTS
16
COTS Functional Testing of SW dependent on
Supplier Documentation available
REDUCE VALIDATION EFFORT
DEVELOPMENT DOCS
17
The Lifecycle Activities for a COTS, cont.
SPECIFY

• Validation Plan based on
• Validation Strategy which is based on the
  Evaluation.
• available Validation Methods, Tools and
  knowledge
• a Validation Process that are documented as a
  matrix of all Validation Activities (VA), based
  on RAI that are expected to be executed for a
  HPLC (COTS). Rational given for VAs that are not
  to be performed. Correspondent User roles in the
  Project and Supplier roles to these activities in
  the project are used in the same matrix.
• statement for compliance to CFR 21 Part 11 (ER
  ES) and other valid GxP regulations
• required Supplier-, System Documentation and
  User Organization and SOPs for supporting and
  maintaining the CS.

URS (PROCESS) EVALUATION VALIDATION
PLAN MASTER INDEX (MI)
18

• Project
• Documentation
• User Requirements

• Validation Documentation
• Validation Plan
• Validation Protocols and Records
• Validation Report

R E A L I Z E

• Maintenance
• Documentation
• Registration
• Support Agreements for HW and SW
• SOPs

Master Index (MI)
O P E R A T E
Change Control Documents
Escrow
IQ Log
User Documentation
Periodic Review
Validation Documentation
Backup Log
19
The Lifecycle Activities for a COTS, cont.
SPECIFY
TEST
Prerequisites to move to Test phase Reviewed and
Approved URSApproved EVALUATION
REPORT Reviewed and Approved VALIDATION
PLAN Standard Index produced for all documents
URS (PROCESS) EVALUATION VALIDATION
PLAN MASTER INDEX (MI)
20
TESTING IS THE KEYSTONE OF THE VALIDATION
PROCESS...
INDEPENDENCE FROM DEVELOPMENT
21
The perfect path ..
SYSTEM TEST
FAT/ SAT
OQ/PQ
22
Life Test Cycle Model
URS User Requirement Specification
Performance/Process Qualification Risk
Analysis
Functional Specification
Operation
Qualification Risk Analysis
Design Specification
Installation Qualification

Source Code / Source Code Testing
SUPPLIER
23
The Lifecycle Activities for a COTS, cont.
TEST

• Test Plan based on
• Validation Plan and RAI of the CS.
• FAT, which would include review of Supplier
  Evaluation, Test, System User documentation
  from the Supplier
• SAT, which would include witness during IQ and
  OQ on site together with Supplier. These tests
  should normally include test for compliance to
  CFR 21 Part 11.
• PROCESS QUALIFICATION, which would include
  performing the complete lab process as required
  by URS and User Guide, should also include CS
  Administration tests.
• DRAFT SOPs could be in one document for this
  type of systems and should be available at PQ.

PROCESS Q FAT SAT TEST PLAN DRAFT SOPs
24
The Lifecycle Activities for a COTS, cont.
TEST
START UP
Prerequisites to move to Start Up
phase Reviewed and Approved PROCESS Q based on
DRAFT versions of the User Guide and
SOPsApproved FAT SAT REPORT Reviewed and
Approved TEST PLANFiled in the Master Index
(MI)
PROCESS Q FAT SAT TEST PLAN DRAFT SOPs
25
The Lifecycle Activities for a COTS, END
START UP
START UP THROUGH CS DECISION based on USER
GUIDE approved and Users trained SYSTEM
ORGANISATION trained on CS and System
Documentation available. VALIDATION REPORT, no
remaining corrections, all documents in MI
approved. (included in MI)
USER GUIDE SYSTEM ORG. DOCs
VALIDATIONREPORT SOPS (Approved)
26
Example of a Validation Tool to a COTS
TOOL
VALIDATION TOOL 1-CS Req. SOP 2-Validation Plan
Report 3-Test Plan Report 4-Validation Plan
Report
REUSABLE
APPENDIX

• 1-CS Req. SOP
• URS/FS/DS and user guide
• Training
• CS Security and authorization (incl. Backup and
  Contingency)
• Maintenance
• Change Control
• Periodic Review

• 2-VALIDATION PR
• Matrix with
• Activities incl. CFR 21 Part 11-declaration
• Responsibility
• Expected Output
• 4-Validation Report
• Reported Output
• Deviations
• Summary
• Approval/Rejection

• 2-Test Plan Report
• Ver. 1
• Matrix with
• Test case
• Expected Result
• 4-Test Plan Report
• Ver. 2
• Reported Result
• Deviations/Re-test
• Summary Recommend.

27
Saving Effort by Reusing the Tool
28
FinallyTo be in Compliance meansCoordination
(Policy Standards) Cooperation (sharing
knowledge support) Capacity (make realistic
Plans for big changes) Competence (get trained
to gain competence) Consistency (use same
measurements tools)
29

• We envision to conveniently engineer
• innovative technology in order to
• synergistically facilitate value-added
• leadership skills to stay competitive in
• tomorrow's world
• Dilbert