MEASUREMENT UNCERTAINTY

1
MEASUREMENT UNCERTAINTY
Abdul Rashid Hj. Zainal Abidin National
Metrology Laboratory SIRIM Berhad Shah Alam 21
April 2005
Workshop on Significance of Testing and
Conformity Assessment In Global Trade Challenges
for Research Technology Organizations
2
OBJECTIVES

• Background
• Concepts Definitions
• Measurement Traceability Requirements
• Measurement Uncertainty Evaluation

3
GLOBAL MARKET PLACE

• Global objective
• Mutual trust in the reliability and comparability
  of test and calibration results over space and
  time

? What constitutes acceptable evidence of
traceability of test and calibration results?
4
GLOBAL MRAs
5
CIPM MRA/APLAC MRA
Test and Measuring Equipment
Accredited calibration laboratories
National Metrology Institutes
APLAC MRA
CIPM-MRA
6
VERTICAL TRACEABILITY
SI UNITS
NML National physical standards
Accredited calibration laboratory reference
standards
In-house calibration laboratory working standards
Measuring and test equipment of the enterprise
product
7
BASIC OPERATIONS OF METROLOGY
      SI Definition
1 ? 1 m2.kg.s-3.A-2  
System/Standard used   Cross-capacitor
and impedance bridges       
Wire wound Standard resistors    
Wire wound Standard
resistors Sensor technology        
relative uncertainty
Realization
? 2.10-7
QH-effect
Reproduction
? 10-9
Maintenance
? 10-8
Dissemination
? 5.10-7
? 10-7
Application
8
SI BASE UNITS
MECHANICS
OPTICS
c
NA
?
CHEMISTRY
?
e
THERMODYNAMICS
ELECTRICITY
9
DEFINITIONS OF THE SI BASE UNITS
10
DERIVED UNITS FROM BASE UNITS
11
THE INTERNATIONAL SYSTEM OF UNITS
12
ISO/IEC 170251999/S1 TRACEABILITY REQUIREMENTS
Clause 5.6.2.1.1 For calibration laboratories,
the programme for calibration of equipment shall
be designed and operated so as to ensure that
calibrations and measurements made by the
laboratory are traceable to the International
System of Units (SI) (Systeme international
dunites)
13
SOURCES OF TRACEABILITY TO THE SI UNITS

• National Metrology Laboratory directly
• National Metrology Laboratory of another country
• A calibration laboratory accredited by SAMM
• A calibration laboratory that is accredited by an
  accrediting body that is a signatory to the APLAC
  MRA
• Direct reference to a primary standard or to a
  natural constant, the value of which in terms of
  the relevant SI unit is known and recommended by
  the General Conference of Weights and Measures
  (CGPM)

14
TRACEABILITY TO NON-SI UNITS

• Clause 5.6.2.1.2
• There are certain calibrations that currently
  cannot be
• strictly made in SI units. In these cases
  calibration
• shall provide confidence in measurements by
  establishing
• traceability to appropriate measurement standards
  such as
• the use of certified reference materials
  provided by a
• competent supplier to give a reliable
  physical or
• chemical characterization of a material
• the use of specified methods and/or consensus
  standards
• that are clearly described and agreed by
  all parties concerned.
• Participation in a suitable programme of
  interlaboratory
• comparisons is required where possible.

15
DEFINITION OF TRACEABILITY (VIM 6.10 1993)
property of the result of a measurement or the
value of a measurement standard whereby it can
be related to stated references, usually
national or international measurement standards,
through an unbroken chain of comparisons all
having stated uncertainties
16
ELEMENTS OF TRACEABILITY

• an unbroken chain of comparisons
• unbroken line of succession back as far as the
  national/international standard
• uncertainty of measurement
• using agreed methods (ISO GUM), overall
  uncertainty of the whole chain
• documentation
• each step in the chain must be documented, all
  results recorded
• competence
• providers in the chain must provide evidence of
  technical competence
• (e.g. accreditation)
• reference to SI units
• the chain must, WHERE possible, end at SI unit
• calibration intervals
• calibrations must be repeated at appropriate
  intervals

17
DEFINITION OF UNCERTAINTY (VIM 3.9 1993)
parameter, associated with the result of a
measurement, that characterizes the dispersion
of the values that could reasonably be
attributed to the measurand
Example of the parameter standard deviation or
multiples of it
18
DEFINITION OF UNCERTAINTY (VIM 3.9 1993)
Example for voltage measurement
V 100.00250 V 0.00050 V - the result of the
measurement is the best estimate of the value
of the measurand - all components of uncertainty
contribute to the dispersion - a measurement
result is not considered valid and not
acceptable without uncertainty statement
19
WHY UNCERTAINTY?

• Measurement based decisions
• To check products against specifications
• To estimate taxes derived from national
  resources
• Need to have some indication of the quality of
  the results
• Confidence in data obtained from outside
  sources
• Comparability of results
• One useful measure is measurement
  uncertainty

20
WHY UNCERTAINTY?
There is no perfect measurement. Uncertainty of
measurement is the doubt that exists about the
result of any measurement. 100 V 1 V at a
level of confidence of 95 This means that we
are 95 sure that the voltage is between 99 V and
101 V.
21
INTERLAB COMPARISON
22
INTERLAB COMPARISON
Laboratory Comparison Results
Outlier
Value Measured
Reference values
Participants
23
ASSESSING COMPLIANCE 1
Case A
Case B
Case C
Case D
Specified Upper limit
measured result
uncertainty interval
Specified lower limit
24
ASSESSING COMPLIANCE 2
Case G
Case H
Case E
Case F
Specified Upper limit
Specified lower limit
25
REPORTING RESULTS TO ISO/IEC 17025
Clause 5.10.3 Test Reports c) where applicable,
a statement on the estimated uncertainty of
measurement information on uncertainty is needed
in test reports when it is relevant to the
validity or application of the test results,
when a clients instruction so requires, or when
the uncertainty affects compliance to a
specification limit
26
REPORTING RESULTS TO ISO/IEC 17025
Clause 5.10.4 In addition to the requirements
listed in 5.10.2, calibration certificates shall
include the following, where necessary for the
interpretation of calibration results b) the
uncertainty of measurement and/or a statement
of compliance with an identified metrological
specification or clauses thereof
27
RANDOM ERROR
Random error typically arises from
unpredictable variations of influence quantities.
These random effects give rise to variations in
repeated observations of the measurand. The
random error of a measurement result cannot
be compensated for, but it can usually be reduced
by increasing the number of observations.
28
SYSTEMATIC ERROR
Component of error which, in the course of a
number of analyses of the same measurand, remains
constant or varies in a predictable way. It is
independent of the number of measurements made
and cannot therefore be reduced by increasing the
number of measurements The result of a
measurement should be corrected for all
recognised significant systematic effects.
29
ERROR AND UNCERTAINTY
Error the difference between an individual
result and the true value of the measurand -
single value Uncertainty A range of values The
value of the uncertainty cannot be used to
correct a measurement result.
Correction - Error
30
INTERNATIONAL VOCABULARY (vim)
Precise definition of METROLOGICAL terms
31
GUIDE TO THE EXPRESSION OF UNCERTAINTY IN
MEASUREMENT
Published in 1993 by ISO in collaboration with
BIPM, IEC, IFCC, IUPAC, IUPAP and OIML Formally
established general rules for evaluating and
expressing uncertainty in measurement across a
broad spectrum of measurements.
32
GUIDES FOR EVALUATION OF UNCERTAINTY OF
MEASUREMENT
33
PURPOSE OF ISO GUM

• Purpose of GUM
• to established general rules procedure for
  evaluating and expressing uncertainty of
  measurement
• intended to be applicable to a broad spectrum of
  measurement at various levels of accuracy
• to provide a basis for international comparison
  of measurement results
• For use within standardization, calibration,
  laboratory accreditation, and metrology services
• Not only a scientific but also a standardization
  issue

34
UNCERTAINTY ESTIMATION PROCESS
Main Steps of Evaluation

• 1. Modeling the measurement
• 2. Evaluating standard uncertainty for components
  according to type A or type B
• 3. Determining combined standard uncertainty
• 4. Determining expanded uncertainty
• 5. Reporting uncertainty

35
SOURCES OF MEASUREMENT UNCERTAINTY

• incomplete definition of the measurand
• imperfect realization of the definition
• imperfect mathematical model
• sampling method
• uncertainties in values of measurement standards
  and reference materials
• uncertainties in constant of other parameters
  obtained from other sources
• environmental factors
• random variation in repeated observations
• instrument resolution
• etc

36
SOURCES OF UNCERTAINTIES
Model
Model
Purity
V
Temperature
Calibration
Repeatability
c(Cd)
Readability
Readability
m(tare)
m(gross)
Linearity
Linearity
Repeatability
Repeatability
Sensitivity
Sensitivity
Calibration
m
Calibration
Cause and Effect Analysis for Cd Standard
Preparation
37
CLASSIFICATION OF UNCERTAINTY COMPONENTS

• Classification of uncertainty components
  according to method of evaluation
• Type A components those that are evaluated by
  statistical analysis of a series of observations
• Type B components those that are evaluated by
  other means
• Both are based on probability distributions
• standard uncertainty of each input estimate is
  obtained from a distribution of possible values
  of input quantity both based on the state of our
  knowledge
• Type A founded on frequency distributions
• Type B founded on a priori distributions

38
TYPE A EVALUATION

• For component of uncertainty arising from random
  effect
• Applied when multiple independent observations
  are made under the same conditions
• Data can be from repeated measurements, control
  chars, curve fit by least-squares method etc
• Obtained from a probability density function
  derived from an observed frequency distribution
  (usually Gaussian)

39
TYPE A EVALUATION

• Best estimate of the expected value of a input
  quantity - arithmetic mean
• Distribution of the quantity - experimental
  standard deviation
• spread of the distribution of the means -
  experimental standard deviation of the mean
• Type A standard uncertainty
• degrees of freedom

40
TYPE B EVALUATION

• evaluated by scientific/professional judgement
• based on all available knowledge
• previous measurement data
• experience/general knowledge of the
  behavior/properties of instruments/materials
• calibration certificates
• manufacturers specifications
• reference data from handbooks

41
TYPE B EVALUATION

• Best estimate ?
• Standard uncertainty of the best estimate, ?
• Degrees of freedom

42
TYPE B EVALUATION

• Normal distribution if the value is obtained
  from a calibration certificate, standard
  uncertainty and degrees of freedom need to be
  retrieved from cal certificate
• Standard uncertainty u(xi) U/k(U - reported
  expanded uncertainty in a certificate,k -
  coverage factor) k 1.96 95 C.L . k
  2 95.45 (appox 95) C.L. (2?) k 3 99.7
  C.L. (3?)
• degrees of freedom check from t-distribution
  table

43
LAW OF PROPAGATION OF ERROR FOR COMBINED
UNCERTAINTY
The combined standard uncertainty is an estimated
standard deviation and characterizes the
dispersion of the values that could reasonably be
attributed to the measurand Y
Combined Standard Uncertainty
Correlation term
Random/independent variability
44
EXPANDED UNCERTAINTY

• quantity defining an interval about the result of
  a measurement that may be expected to encompass a
  large fraction p of the distribution
  characterized by that result and its combined
  standard uncertainty
• p - coverage probability or level of confidence
  of the interval

45
DETERMINING THE EXPANDED UNCERTAINTY

• U k uc(y)
• k - coverage factor
• chosen from the desired level of confidence
• under certain conditions, normal distribution can
  be assumed due to Central Limit Theorem
• for normal distribution chosen from
  t-distribution table according to effective
  degrees of freedom
• Level of confidence
• Most cal labs adopt 95 which gives k ? 2 for
  effective degrees of freedom ? 30

46
DETERMINING the k-factor
t-distribution from the ISO GUM
47
REPORTING

• should include
• result of measurement
• expanded uncertainty with coverage factor and
  level of confidence specified
• description of measurement method and reference
  standard used
• uncertainty budget
• example of uncertainty statement
• e.g. The expanded uncertainty of measurement is
  ____, estimated at a level of confidence of
  approximately 95 with a coverage factor k ____.

48
TOOLS FOR UNCERTAINTY OF MEASUREMENT CALCULATION
49
FUTURE DEVELOPMENT

• Mainstream GUM
• Propagation of uncertainties
• (widely used but some limitations)
• Propagation of distributions
• Numerical methods Monte Carlo Simulation

50
TRACEABILITY REQUIREMENTS IN NEW AND EMERGING
AREAS

• Chemistry
• Health care
• Food safety and testing
• Fraud, forensics, anti-doping, e-commerce
  security
• Biotechnology and biology
• Noise, dust, vibration, taste, smell, appearance
• Environmental monitoring
• Monitoring of climate change
• Nanometrology
• Laboratory Medicine

51
CONCLUSION

• Uncertainty of test and calibration results is
  critical to laboratory
• compliance to ISO/IEC 170251999
• It is strongly linked to the concept of
  traceability
• An essential information for conformity
  assessment
• A tool to quantify or estimate the quality of the
  results provided
• by the laboratories
• A tool to control and improve the measurement and
  testing processes
• Measurement and calibration results shall be
  traceable to
• the SI units where possible
• Traceability of measurement standards and results
  is
• critical to successful participation in a PT and
  MA programme
• If evaluated properly, is an indicator of the
  quality of a test and the technical
  competency of the lab

52
NATIONAL METROLOGY LABORATORY
THANK YOU