Assessor seminar feedback

1
Topics

• Assessor seminar feedback
• Laboratory feedback
• Sampling of tests for assessment
• New/revised requirements of 17025
• Statistics on non-conformities
• Empirical in-house methods

2
Feedback - last assessor seminar

• Fulfilling assessors need and expectation
• Duration
• Gain from the seminar

3
Feedback - last assessor seminar

• 127(110) assessors attended
• 84(48) evaluation forms returned
• Fulfilling assessors need and expectation
• 0 (0) not really, 23 (31) adequately, 57 (65)
  well, 20 (4) very well
• Duration
• 0 (0) too long, 96 (98) suitable, 4 (2) too
  short

4
Feedback - gain

• 112 comments received
• Grading of non-conformities (9)
• Views and experience of assessors (17)
• Latest development of HKAS (24)
• Technical issues, eg PT, MU, etc (6)

5
Feedback suggestions on future activities

• 24 suggestions received
• More assessor workshops
• Longer discussion
• More technical seminars

6
Laboratory Feedback

• September 2005
• All accredited organisations
• Accredited labs 128
• Return (labs) 55 43

7
Laboratory Feedback- Degree of rigor

• Substantially relaxed 0
• Slightly relaxed 29.5
• Remained unchanged 67.2
• Slightly tightened 3.3
• Substantially tightened 0

8
Laboratory Feedback- Degree of rigor

• Depending on the availability of technology
  available in the market, some equipment,
  specifications or requirements stated in testing
  standard that cannot be fulfilled should be duly
  accepted. Surely it can be reviewed again per the
  advancement of technology.

9
Laboratory Feedback- Degree of rigor

• More flexibility to be given for quoting
  accreditation to a standard with modification for
  those modified method verified with same level of
  performance characteristics.

10
Response
11
Laboratory Feedback- Degree of rigor

• Consistency on technical/quality requirements
  between different assessors should be aligned.
• Sometimes, the corrective actions taken for
  non-conformities raised in the previous
  assessment may not be accepted in the next
  assessment.

12
Response
13
Response
14
Laboratory Feedback- Degree of rigor

• It can help to maintain lab standard.
• I like officer be tighten on the quality system.

15
Laboratory Feedback- Degree of rigor

• It seems that the assessment is too concentrated
  on minor points rather than major issues.

16
Response
17
ISO/IEC 170252005

• Published on 15 May 2005
• 8th edition of HOKLAS 003 published in October
  2005
• Full implementation by 15 May 2007 by accredited
  laboratories
• No extra assessment to be conducted
• Amendment mainly on Section 4 (Management
  requirements)

18
HOKLAS 003 8th edition

• Section 1 Introduction - Updated
• Section 2 Normative references New
• Section 3 Terms and definitions New
• Sections 4 5 revised as per ISO/IEC
  170252005
• Section 5 Technical requirements only two
  revised/new clauses

19
Section 5 HOKLAS 003 8th edition

• Clause 5.2.2 Personnel
• . The effectiveness of the training actions
  taken shall be evaluated.
• Clause 5.9.2 Assuring the quality of test and
  calibration results
• Quality control data shall be analysed and, where
  they are found to be outside pre-defined
  criteria, planned action shall be taken to
  correct the problem and to prevent incorrect
  results from being reported..

20
HOKLAS 003 8th edition

• Section 1 Introduction
• Compliance with regulatory and safety
  requirements on the operation of laboratories is
  not covered by this International Standard.
• Section 5.3.H
• It should be emphasized that it is the
  responsibility of the laboratory to ensure that
  its operation is in compliance with all
  applicable regulations, particularly those
  related to laboratory safety.

21
HOKLAS 003 8th edition

• Section 4.9.H Control of nonconforming testing
  and/or calibration work
• Examples
• environmental conditions exceeded limits
• duration of conditioning of test specimens not
  complying with specifications
• max. holding times of samples exceeded
• tests performed using instruments with expired
  calibration
• QC acceptance limits exceeded
• performance in PT unsatisfactory

22
HOKLAS 003 8th edition

• Section 4.13.H Control of records
• The retention period of at least 3 yrs for
  equipment records and lab. operation procedures
  shall be counted from the date on which the use
  of the equipment or the operation procedures have
  been discontinued respectively. Similarly, the
  retention period of at least 3 yrs for personnel
  records shall be counted from the date of
  departure of the staff member concerned.

23
Sampling of tests for assessment

• ISO/IEC 170112004
• Conformity assessment
• General requirements for accreditation bodies
  accrediting conformity assessment bodies

24
Sampling of tests for assessment

• Cl. 7.5.6 Preparation for assessment
• The accreditation body shall establish procedures
  for sampling (if applicable) where the scope of
  the CAB covers a variety of specific conformity
  assessment services. The procedures shall ensure
  that the assessment team witness a representative
  number of examples to ensure proper evaluation of
  the competence of the CAB

25
Sampling of tests for assessment

• Cl. 7.6.1 Document and record review
• The assessment team shall review all relevant
  documents and records supplied by the CAB to
  evaluate its system, as documented, for
  conformity with the relevant standard(s) and
  other requirements for accreditation
• Cl. 7.7.3 On-site assessment
• The assessment team shall witness the performance
  of a representative number of staff of the CAB to
  provide assurance of the competence of the CAB
  across the scope of accreditation.

26
Sampling of tests for assessment

• HKAS Management Manual App 9.4A5
• Emphasis should be given to new activities to be
  accredited and activities and aspects which are
  known to be problematic, e.g. activities with
  poor proficiency testing results or performed in
  accordance with standards which are known to have
  different interpretations, subjects of
  complaints, aspects where many non-conformities
  were identified in previous assessments, etc.

27
Sampling of tests for assessment

• HKAS 008 Guide for HKAS Assessors and Lead
  Assessors Clause 7.3.1 Observing of the
  Organisations Practice
• The assessment team should witness some or all of
  the tests within the scope of accreditation.
• Even if a particular test is too lengthy to be
  performed in its entirety during the visit, it
  should be possible for at least the most critical
  phases of the test to be performedDepending
  upon the circumstances, samples or specimens used
  in such tests may be conveniently drawn from
  the normal work throughput or provided by a
  member of the assessment team through arrangement
  with the team leader.

28
Sampling of tests for assessment

• Difficult to have hard and fast rules
• Random sampling is not the most appropriate
• Priority for those not assessed before
• At least select a representative test in each
  test area

29
Sampling of tests for assessment

• Like driving test select those that need
  special skills and knowledge
• Like an examination, select those tests which are
  most indicative of labs competence
• Like cooking class, if the test is solely for
  demonstration purpose, skip those non-critical
  steps
• More than one test can be performed at a time
• Prior arrangement with the lab is helpful

30
Laboratory internal validation and verification
Laboratory requirement Verification
(Secondary validation) Validation Validatio
n Validation (Primary validation)
Existing information Fully validated standard
methods (have been studied in a collaborative
trail) Standard methods amplifications and
modifications e.g. new instrument Standard
methods outside their intended
scope Laboratory developed and non-standard
methods
31
Definition

• Validation is the confirmation, through provision
  of objective evidence, that the particular
  requirements for a specific intended use or
  application have been fulfilled (ISO/IEC 17025
  clause 5.4.5.1 and ISO 90002000)
• Verification is the confirmation, through
  provision of objective evidence, that specified
  requirements have been fulfilled. (ISO 90002000)
• Objective evidence is data supporting the
  existence or verity of something (ISO90002000)

32
Standard methods shall be verified for

• 1. Procedures conformity with the standard
  method
• 2. Availability and suitability of equipment and
  reference materials/standard, reagents, etc
• 3. Appropriateness of environmental conditions
• 4. Competence of testing staff
• 5. Capability of lab of achieving the defined
  method performance characteristics

33
Main validation parameters

• Applicability
• Selectivity
• Calibration and linearity
• Trueness
• Precision

34
Main validation parameters

• Recovery
• Working concentration and measurement range
• Limit of Detection (LOD) and Quantitation (LoQ)
• Matrix variation
• Measurement Uncertainty

35
Common insufficiencies

• 1. Method bias not established or demonstrated
• 2. Precision data derived from short term
  results and do not cover major variations
• 3. Detection limits wrongly determined
• 4. Major components omitted in measurement
  uncertainty estimation

36
Empirical Methods

• Definition the quantity estimated is simply the
  result found on following the stated
  procedure.Hence, method bias is zero and matrix
  variation is irrelevant. However, laboratory bias
  need to be assessed. Apart from these, method
  validation similar to other rational methods.
  Empirical methods are often defined by standard
  methods or legislation.

37
In-house Methods

• Definition methods other than standard methods
  and which are designed/developed by the
  laboratory.
• Full method validation required

38
In-house Empirical Methods

• For some measurands (analytes), their values are
  defined by the procedure used, which may be given
  in standard methods, in legislation or by users
  of testing service. Many commonly encountered
  measurands determined by empirical methods have
  established meanings, e.g. BOD, COD, SS, VS, etc.

39
In-house Empirical Methods

• For these measurands, in-house empirical methods
  need to show comparability with the those defined
  in well recognised standard methods.
• Very often, in-house empirical methods are
  modified standard methods. Ensure the
  modifications do not affect the values obtained,
  based on professional judgement and validation
  data (e.g. CRM, PT, comparison with relevant
  standard methods, etc.).

40
In-house Empirical Methods

• May need to define the measurands or give key
  experimental conditions in test reports, e.g.
  extraction conditions for metals (temperature,
  duration, acid strength, etc.), drying
  temperature of suspended solids, etc.
• Customer specified in-house empirical methods
  precision need to be demonstrated