Measurement Uncertainties and Inconsistencies

1
Measurement Uncertainties and Inconsistencies

• Dr. Richard Young
• Optronic Laboratories, Inc.

2
Introduction

• The concept of accuracy is generally understood.
• an accuracy of 1.
• What does this mean?
• 99 inaccurate?

3
Introduction

• The confusion between the concept and the numbers
  has lead national laboratories to abandon the
  term accuracy.
• Except in qualitative terms e.g. high accuracy.
• The term now used is uncertainty.
• an uncertainty of 1.

4
Introduction

• Sometimes
• Users do not know the uncertainty of their
  results.
• They interpret any variations as inconsistencies.

5
Uncertainty vs. Inconsistency

• Laboratories give different values, but the
  difference is within their combined
  uncertainties
• Pure chance.
• Laboratories give different values, and the
  difference is outside their combined
  uncertainties
• Inconsistency.

6
What is uncertainty?

• an uncertainty of 1.
• But is 1 the maximum, average or typical
  variation users can expect?
• Uncertainty is a statistical quantity based on
  the average and standard deviation of data.

7
Statistics

• There are three types of lies lies, damned lies
  and statistics.
• -attributed to Benjamin Disraeli

The difference between statistics and experience
is time. -Richard Young
Statistics uses past experience to predict likely
future events.
8
Statistics

• We toss a coin
• It is equally likely to be heads or tails.
• We toss two coins at the same time
• There are 4 possible outcomes
• Head Head
• Head Tail
• Tail Head
• Tail Tail

9
Statistics

• Now let us throw 10 coins.
• There are 1024 possibilities (210).
• What if we threw them 1024 times, and counted
  each time a certain number of heads resulted

10
Statistics

• Although the outcome of each toss is random
• ...not every result is equally likely.
• If we divide the number of occurrences by the
  total number of throws
• We get probability.

11
Statistics

• Here is the same plot, but shown as probability.
• Probability is just a number that describes the
  likelihood between
• 0 never happens
• 1 always happens

12
Statistics

• Gauss described a formula that predicted the
  shape of any distribution of random events.
• Shown in red
• It uses just 2 values
• The average
• The standard deviation

13
Statistics

• Now throw 100 coins

We have an average 50
The Gaussian curve fits exactly.
And the familiar bell-shaped distribution.
14
Confidence

• Now throw 100 coins

Since the total probability must 1, the standard
deviation marks off certain probabilities.
15
Confidence

• Now throw 100 coins

Since the total probability must 1, the standard
deviation marks off certain probabilities.
About 67 of all results lie within ? 1 standard
deviation.
I am 67 confident that a new throw will give
between 45 and 55 heads.
16
Confidence

• Now throw 100 coins

Since the total probability must 1, the standard
deviation marks off certain probabilities.
About 95 of all results lie within ? 2 standard
deviations.
I am 95 confident that a new throw will give
between 40 and 60 heads.
17
Real Data

• Real data, such as the result of a measurement,
  is also characterized by an average and standard
  deviation.
• To determine these values, we must make
  measurements.

18
Real Data

• NVIS radiance measurements are unusual.
• The signal levels at longer wavelengths can be
  very low close to the dark level of the system.
• The signal levels at longer wavelengths dominate
  the NVIS radiance result.
• The uncertainty in results close to the dark
  level can be dominated by PMT noise.
• Therefore Variations in NVIS results can be
  dominated by PMT noise.

19
Real Data

• The net signal from the PMT is used to calculate
  the spectral radiance.
• Dark current, which is subtracted from each
  current reading during a scan, contains PMT
  noise.
• Scans at low signals contain PMT noise.

20
Real Data

• PMT noise present in each of these current
  readings does not have the same effect on
  results
• A high or low dark reading will raise or lower
  ALL points.
• Current readings during scans contain highs and
  lows that cancel out to some degree.

21
Real Data
Excel average() ? 2E-12
Excel stdev() ? 1E-13
22
Real Data
23
Real Data
24
Real Data
25
Calculations

• We can describe the effects of noise on class A
  NVIS radiance mathematically
• ?s is the standard deviation of the noise
• C(?) is the calibration factors
• GA(?) is the relative response of class A NVIS

Signal averaging
Dark subtraction
26
Calculations

• A similar equation, but using NVIS class B
  response instead of class A, can give the
  standard deviation in NVISb radiance.
• The standard deviations should be scaled to the
  luminance to give the expected variations in
  scaled NVIS radiance.

27
Calculations

• Noise can be reduced by multiple measurements.
• If we generalize the equation to include multiple
  dark readings (ND) and scans (S)

Brain overload
28
Spreadsheet

• Moving on to the benefits

Introducing
The Spreadsheet