PROFICIENCY TESTING A BRIEF OVERVIEW

1
PROFICIENCY TESTINGA BRIEF OVERVIEW

• CPT Anne Sterling
• LTC Paul Mann

2
OUTLINE

• WHAT IS PT? WHY DO WE DO IT?
• WHO DOES IT HOW OFTEN?
• HOW IS IT EVALUATED?
• WHAT DO RESULTS MEAN?
• WHAT HAPPENS WHEN WE FAIL?
• HOW TO INVESTIGATE PT

3
PROFICIENCY TESTING

• Is a form of external quality control
• Is a mechanism to ensure standardized testing
  across clinical laboratories and evaluate your
  labs performance in comparison to peer groups
  performance
• - Uses commercially available materials
  evaluations
• - CAP Surveys

4
WHY DO WE DO IT?

• CLIA 88
• CLIP (CCLM)
• CAP
• TJC
• GOOD LABORATORY PRACTICES
• - Develops confidence in the accuracy and
  reliability of results

5
WHO IT APPLIES TO?

• CLIA 88
• - ALL TESTING LOCATIONS PERFORMING
• MODERATE OR HIGH COMPLEXITY TESTING
• CLIP
• - ALL TESTING LOCATIONS PERFORMING MODERATE OR
  HIGH COMPLEXITY TESTING Waived (Minimal
  Complexity) labs are required to be enrolled in a
  PT program if one is commercially available.

6
WHO IS EXEMPT?

• CLIA 88
• - WAIVED PPM SITES
• CLIP
• - nobody
• LOCAL POLICY
• - Can be more stringent than CLIA/CLIP. This is
  what TJC/CAP will hold you to.

7
WHAT IT APPLIES TO ?

• CLIA 88
• - ALL REGULATED ANALYTES
• CLIP
• - ALL ANALYTES

8
Regulated vs UnregulatedAnalytes

• Regulated Analytes Listed in 42 CFR Part 493
  Subpart I
• Unregulated analytes those not listed in the CFR

9
HOW OFTEN?

• CAP SURVEYS
• - THREE TIMES A YEAR (MINIMUM OF 5 SAMPLES FOR
  ALL REGULATED ANALYTES)
• Unregulated analytes usually have fewer
  challenges, fewer samples per challenge
• FOR TESTS WITH NO COMMERCIAL SURVEYS AVAILABLE
• - Alternate method (Split Testing) will be
  performed at least every 6 months

10
HOW IS IT EVALUATED?
11
e-Lab Solutions

• The best way to view CAP evaluations is to use
  e-Lab Solutions
• Request access from your institutions
  administrator

12
THREE COMPONENTS TO PT PERFORMANCE

• the actual result
• the target value
• the evaluation interval - or acceptable error -
  for that specimen.

13
THREE DIFFERENT TYPES OF TARGET VALUES

• method group (peer group) means
• means from another group or an all-results mean
• values derived from an external source (for
  example, reference laboratory consensus or
  definitive/reference methods).

14
FOUR GENERAL TYPES OF EVALUATION INTERVALS

• fixed intervals (e.g., 4 mmol/L)
• fixed percentages (e.g., 10 of the target
  value)
• a combination of these two (e.g., 6 mg/dL or 10
  of the target value, whichever is greater)
• intervals based on the group standard deviation
  (SD) (e.g., 2 SD).

15
WHAT DO THE RESULTS MEAN?

• WOW!!
• IF YOUR IN, YOUR GOOD
• - GUESS AGAIN
• IF YOUR OUT, YOUR INCOMPETENT
• - PROBABLY, BUT MAYBE NOT

16
CAP Evaluation What Should You Review?

• Evaluate each analyte and specimen for
  negative/positive bias, trends or shifts
• Evaluate ungraded challenges

17
EVALUATE RESULTS

• QUANTITATIVE
• - MEAN /- 2 SD
• - EVALUATE BIAS
• - EVALUATE CV
• - EVALUATE TRENDS
• QUALITATIVE
• - INTENDED RESPONSE
• - MAJORITY OF RESULTS

18
PT Grading Policy

• Minimum passing score of 80,
• 100 for ABO/Rh Compatibility testing
• Passing grades on PT surveys with 5 challenges
  will require 4 correct, 4 challenges will require
  3 correct, 3 or fewer challenges will require all
  to be correct.
• Challenge Failures
• Analyte Failures

19
Proficiency Testing Exception Summary (PTES)

• Unsatisfactory PT performance
• Failure to attain the minimum satisfactory score
  for an analyte, test, subspecialty, or specialty
  for a testing event.
• Unsuccessful PT performance (2 of 3 testing
  events)
• Failure to attain the minimum satisfactory score
  for an analyte, test, subspecialty, or specialty
  for two consecutive or two of three consecutive
  testing events.
• Critical PT performance (3 of 4 testing events)
• Failure to attain the minimum satisfactory score
  for an analyte, test, subspecialty, or specialty
  for three consecutive or three of four
  consecutive testing events. A laboratory must
  immediately cease testing for that analyte or the
  discipline.

20
WHAT HAPPENS WHEN WE FAIL?
21
UNSATISFACTORY

• BE PROACTIVE
• - determine why
• - prevent 2nd unsatisfactory performance
• INITIATE CORRECTIVE ACTION
• PREVENT LIMITATION OF SERVICES

22
2nd PT Failure
23
UNSUCCESSFUL

• STOP TESTING
• INVESTIGATE DETERMINE CAUSE OF FAILURE
• EVALUATE PATIENT RESULTS
• INITIATE CORRECTIVE ACTION
• TRAIN STAFF
• EVALUATE CORRECTIVE ACTION (verify on 2 separate
  occasions)
• DOCUMENTATION
• REQUEST APPROVAL TO RESUME TESTING

24
Do a Root Cause Analysis

• A rigorous systematic approach to answering
• - What happened Why?

25
How do we do Root Cause Analysis?

• Said simply, Root Cause Analysis is asking why
  the problem occurred, and then continuing to ask
  why that happened until we reach the fundamental
  process element that failed.

26
RCA Goals

• Find out
• What happened?
• Why did it happen?
• What do you do to prevent it from happening
  again?
• How do we know we made a difference?

27
Team Approach to RCA

• QA
• Supervisor
• Bench Techs
• Director

28
Involve Techs?

• Techs know what happens at the microprocess
  (bench) level
• Will give them a sense of accomplishment/contribut
  ion
• Involve in both the investigation and solution
  development
• Techs will be the ones that implement the
  solution

29
4Ms of a RCA

• Materials
• Defective Survey Material
• Wrong Survey Material
• Machine / Equipment
• Instrument Malfunction
• Maintenance
• Calibration
• Quality Control

• Methods
• SOP written , current, available
• SOP adequate
• SOP followed
• Man (Management)
• Training
• Competency
• Result Entry
• Review Process

30
RCA Methods

• How do you determine the cause?
• Examine the original test print outs to ensure
  accurate entry
• Verify QC and look at Levy-Jennings graphs to see
  if there was a problem (drift or shift) that was
  not caught
• Re-test CAP specimen to see if the same result is
  achieved
• Split test with other instruments (other
  facility)
• Verify staff competency

31
CLASSIFY THE PROBLEM

• Unacceptable results may be classified as
  follows
• Clerical error
• Methodological problem
• Technical problem
• Problem with proficiency testing materials
• Problem with evaluation of results
• No explanation after investigation

32
Reasons for Failure
33
INITIATE CORRECTIVE ACTIONS

• PROVE ACCURACY PRECISION OF METHOD
• - RECALIBRATION
• -PROCEDURAL UPDATE
• - SURVEY MATERIAL HANDLING PROCESSING
• RETRAINING
• NEW METHOD/INSTRUMENT

34
EVALUATION OF PATIENT RESULTS

• Review patient data from the time of the
  unacceptable PT result, to determine whether the
  problem could have affected patient care
• If so, appropriate follow-up action should be
  documented

35
EVALUATE CORRECTIVE ACTIONS

• Ensure the validity of patient results by
  verifying on two consecutive occasions that the
  corrective action taken has resolved the problem
  
• This maybe done by reanalysis and/or retesting of
  frozen or additional PT material, purchase of
  supplemental PT material, or blind, split-sample
  testing of patient material with another
  certified laboratory

36
DOCUMENTATION

• Document investigation, conclusions, and
  corrective actions taken
• Maintain documentation for at least 2 years to
  include worksheets, instrument tapes, reporting
  forms, evaluation reports, participant summaries,
  and documentation of follow-up, as applicable.
  (5 years for Immunohematology) from the date of
  event.

37
(No Transcript)
38
Director Review

• The laboratory director will review the
  effectiveness of the corrective actions and, if
  satisfied, will document his/her recommendation
  whether to resume testing.

39
Approval Process

• Unsatisfactory Local
• Unsuccessful RMC
• Critical CCLM

40
Critical

• 3rd of 4 PT events
• Failed to address the problem, identify the
  cause, and correct to avoid future problems

41
REQUEST APPROVAL TO RESUME TESTING

• MEDICAL DIRECTORS APPROVAL
• - 1 of 2 or 1 of 3 unsatisfactory performance
• RMC APPROVAL
• - 2 of 2 or 2 of 3 unsatisfactory performance
• MEDCOM APPROVAL
• - 3 of 3 or 3 of 4 unsatisfactory performance

42
Actions Laboratories Should Take when a PT Result
is not Graded

• Code 11 unable to analyze
• Code 20 No appropriate target/response cannot
  be graded
• Code 21 Specimen Problem
• Code 22 Result is outside the method/instrument
  reportable range
• Code 24 Incorrect response due to failure to
  provide a valid response code
• Code 25 Inappropriate use of antimicrobial

43
Actions Laboratories Should Take when a PT Result
is not Graded contd

• Code 26 Educational Challenge
• Code 27 Lack of participant or referee
  consensus
• Code 28 Response qualified with a greater than
  or less than sign unable to quantitate
• Code 30 Scientific committee decision
• Code 33 Specimen determined to be
  unsatisfactory after contacting the CAP
• Code 40 Results for this kit were not received

44
Education Challenges Code 26

• Phase II
• Is there evidence of evaluation and, if
  indicated, corrective action in response to
  "unacceptable" results on the proficiency testing
  reports and results of the alternative
  performance assessment system?
• NOTE The evaluation must document the
  specific reason(s) for the "unacceptable"
  result(s) and actions taken to reduce the
  likelihood of recurrence. This must be done
  within one month after the program receives its
  evaluation. In addition, each ungraded
  challenge, each educational challenge, and each
  episode of nonparticipation must be reviewed and
  corrective action instituted as appropriate.

45
Actions Laboratories Should Take when a PT Result
is not Graded contd

• Code 41 Results of this kit were received past
  the due date
• Code 42 No credit assigned due to absence of
  response
• Code 44 This drug is not included in our test
  menu. Use of this code counts as a correct
  response

46
CONCLUSION

• EVERY EFFORT SHOULD BE MADE TO FIND THE CAUSE(S)
  OF AN UNACCEPTABLE PT RESULT.
• ACTIONS TO IMPROVE THE LABORATORY SYSTEM WILL
  MINIMIZE THE RISK OF RECURRENCE AND POTENTIALLY
  IMPROVE THE QUALITY OF PATIENT RESULTS.

47
QUESTIONS?