Summary of Experimental Uncertainty Assessment Methodology

1
Summary of Experimental Uncertainty Assessment
Methodology

• F. Stern, M. Muste, M-L. Beninati, W.E. Eichinger

2
Table of Contents

• Introduction
• Test Design Philosophy
• Definitions
• Measurement Systems, Data-Reduction Equations,
  and Error Sources
• Uncertainty Propagation Equation
• Uncertainty Equations for Single and Multiple
  Tests
• Implementation Recommendations

3
Introduction

• Experiments are an essential and integral tool
  for engineering and science
• Experimentation procedure for testing or
  determination of a truth, principle, or effect
• True values are seldom known and experiments have
  errors due to instruments, data acquisition, data
  reduction, and environmental effects
• Therefore, determination of truth requires
  estimates for experimental errors, i.e.,
  uncertainties
• Uncertainty estimates are imperative for risk
  assessments in design both when using data
  directly or in calibrating and/or validating
  simulation methods

4
Introduction

• Uncertainty analysis (UA) rigorous methodology
  for uncertainty assessment using statistical and
  engineering concepts
• ASME (1998) and AIAA (1999) standards are the
  most recent updates of UA methodologies, which
  are internationally recognized
• Presentation purpose to provide summary of EFD
  UA methodology accessible and suitable for
  student and faculty use both in classroom and
  research laboratories

5
Test design philosophy

• Purposes for experiments
• Science technology
• Research development
• Design, test, and product liability and
  acceptance
• Instruction
• Type of tests
• Small- scale laboratory
• Large-scale TT, WT
• In-situ experiments
• Examples of fluids engineering tests
• Theoretical model formulation
• Benchmark data for standardized testing and
  evaluation of facility biases
• Simulation validation
• Instrumentation calibration
• Design optimization and analysis
• Product liability and acceptance

6
Test design philosophy

• Decisions on conducting experiments governed by
  the ability of the expected test outcome to
  achieve the test objectives within allowable
  uncertainties
• Integration of UA into all test phases should be
  a key part of entire experimental program
• Test description
• Determination of error sources
• Estimation of uncertainty
• Documentation of the results

7
Test design philosophy
8
Definitions

• Accuracy closeness of agreement between measured
  and true value
• Error difference between measured and true value
• Uncertainties (U) estimate of errors in
  measurements of individual variables Xi (Uxi) or
  results (Ur) obtained by combining Uxi
• Estimates of U made at 95 confidence level

9
Definitions

• Bias error b fixed, systematic
• Bias limit B estimate of b
• Precision error e random
• Precision limit P estimate of e
• Total error d b e

10
Measurement systems, data reduction equations,
error sources

• Measurement systems for individual variables Xi
  instrumentation, data acquisition and reduction
  procedures, and operational environment
  (laboratory, large-scale facility, in situ) often
  including scale models
• Results expressed through data-reduction
  equations
• r r(X1, X2, X3,, Xj)
• Estimates of errors are meaningful only when
  considered in the context of the process leading
  to the value of the quantity under consideration
• Identification and quantification of error
  sources require considerations of
• Steps used in the process to obtain the
  measurement of the quantity
• The environment in which the steps were
  accomplished

11
Measurement systems and data reduction equations

• Block diagram showing elemental error sources,
  individual measurement systems, measurement of
  individual variables, data reduction equations,
  and experimental results

12
Error sources

• Estimation assumptions 95 confidence level,
  large-sample, statistical parent distribution

13
Uncertainty propagation equation

• Bias and precision errors in the measurement of
  Xi propagate through the data reduction equation
  r r(X1, X2, X3,, Xj) resulting in bias and
  precision errors in the experimental result r
• A small error (?Xi) in the measured variable
  leads to a small error in the result (?r) that
  can be approximated using Taylor series expansion
  of r(Xi) about rtrue(Xi) as
• The derivative is referred to as sensitivity
  coefficient. The larger the derivative/slope,
  the more sensitive the value of the result is to
  a small error in a measured variable

14
Uncertainty propagation equation

• Overview given for derivation of equation
  describing the error propagation with attention
  to assumptions and approximations used to obtain
  final uncertainty equation applicable for single
  and multiple tests
• Two variables, kth set of measurements (xk, yk)

The total error in the kth determination of r
(1)
sensitivity coefficients
15
Uncertainty propagation equation

• We would like to know the distribution of dr
  (called the parent distribution) for a large
  number of determinations of the result r
• A measure of the parent distribution is its
  variance defined as

(2)

• Substituting (1) into (2), taking the limit
  for N approaching infinity, using definitions of
  variances similar to equation (2) for b s and e
  s and their correlation, and assuming no
  correlated bias/precision errors

(3)

• ss in equation (3) are not known estimates for
  them must be made

16
Uncertainty propagation equation

• Defining
• estimate for
• estimates for the
  variances and covariances (correlated bias
  errors) of
  the bias error distributions
• estimates for the
  variances and covariances ( correlated precision
  errors) of the
  precision error distributions

equation (3) can be written as
Valid for any type of error distribution

• To obtain uncertainty Ur at a specified
  confidence level (C), a coverage factor (K) must
  be used for uc

• For normal distribution, K is the t value from
  the Student t distribution.
• For N ? 10, t 2 for 95 confidence level

17
Uncertainty propagation equation

• Generalization for J variables in a result r
  r(X1, X2, X3,, Xj)

sensitivity coefficients
Example
18
Uncertainty equations for single and multiple
tests

• Measurements can be made in several ways
• Single test (for complex or expensive
  experiments) one set of measurements (X1, X2,
  , Xj) for r
• According to the present methodology, a test is
  considered a single test if the entire test is
  performed only once, even if the measurements of
  one or more variables are made from many samples
  (e.g., LDV velocity measurements)
• Multiple tests (ideal situations) many sets of
  measurements (X1, X2, , Xj) for r at a fixed
  test condition with the same measurement system

19
Uncertainty equations for single and multiple
tests

• The total uncertainty of the result

(4)

• Br same estimation procedure for single and
  multiple tests
• Pr determined differently for single and
  multiple tests

20
Uncertainty equations for single and multiple
tests bias limits

• Br

• Sensitivity coefficients

• Bi estimate of calibration, data acquisition,
  data reduction, conceptual bias errors for Xi..
  Within each category, there may be several
  elemental sources of bias. If for variable Xi
  there are J significant elemental bias errors
  estimated as (Bi)1, (Bi)2, (Bi)J, the bias
  limit for Xi is calculated as
• Bik estimate of correlated bias limits for Xi
  and Xk

21
Uncertainty equations for single test precision
limits

• Precision limit of the result (end to end)

t coverage factor (t 2 for N gt 10) Sr the
standard deviation for the N readings of the
result. Sr must be determined from N readings
over an appropriate/sufficient time interval

• Precision limit of the result (individual
  variables)

the precision limits for Xi
Often is the case that the time interval is
inappropriate/insufficient and Pis or Prs must
be estimated based on previous readings or best
available information
22
Uncertainty equations for multiple tests
precision limits

• The average result

• Precision limit of the result (end to end)

t coverage factor (t 2 for N gt 10)
standard deviation for M readings of the result

• The total uncertainty for the average result
• Alternatively can be determined by RSS of
  the precision limits of the individual variables

23
Implementation

• Define purpose of the test
• Determine data reduction equation r r(X1, X2,
  , Xj)
• Construct the block diagram
• Construct data-stream diagrams from sensor to
  result
• Identify, prioritize, and estimate bias limits at
  individual variable level
• Uncertainty sources smaller than 1/4 or 1/5 of
  the largest sources are neglected
• Estimate precision limits (end-to-end procedure
  recommended)
• Computed precision limits are only applicable for
  the random error sources that were active
  during the repeated measurements
• Ideally M ? 10, however, often this is no the
  case and for M lt 10, a coverage factor t 2 is
  still permissible if the bias and precision
  limits have similar magnitude.
• If unacceptably large Ps are involved, the
  elemental error sources contributions must be
  examined to see which need to be (or can be)
  improved
• Calculate total uncertainty using equation (4)
• For each r, report total uncertainty and bias and
  precision limits

24
Recommendations

• Recognize that uncertainty depends on entire
  testing process and that any changes in the
  process can significantly affect the uncertainty
  of the test results
• Integrate uncertainty assessment methodology into
  all phases of the testing process (design,
  planning, calibration, execution and post-test
  analyses)
• Simplify analyses by using prior knowledge (e.g.,
  data base), concentrate on dominant error sources
  and use end-to-end calibrations and/or bias and
  precision limit estimation
• Document
• test design, measurement systems, and data
  streams in block diagrams
• equipment and procedures used
• error sources considered
• all estimates for bias and precision limits and
  the methods used in their estimation (e.g.,
  manufacturers specifications, comparisons against
  standards, experience, etc.)
• detailed uncertainty assessment methodology and
  actual data uncertainty estimates

25
References

• AIAA, 1999, Assessment of Wind Tunnel Data
  Uncertainty, AIAA S-071A-1999.
• ASME, 1998, Test Uncertainty, ASME PTC
  19.1-1998.
• ANSI/ASME, 1985, Measurement Uncertainty Part
  1, Instrument and Apparatus, ANSI/ASME PTC
  19.I-1985.
• Coleman, H.W. and Steele, W.G., 1999,
  Experimentation and Uncertainty Analysis for
  Engineers, 2nd Edition, John Wiley Sons, Inc.,
  New York, NY.
• Coleman, H.W. and Steele, W.G., 1995,
  Engineering Application of Experimental
  Uncertainty Analysis, AIAA Journal, Vol. 33,
  No.10, pp. 1888 1896.
• ISO, 1993, Guide to the Expression of
  Uncertainty in Measurement,", 1st edition, ISBN
  92-67-10188-9.
• ITTC, 1999, Proceedings 22nd International Towing
  Tank Conference, Resistance Committee Report,
  Seoul Korea and Shanghai China.