Technical Review Panel

1

• Technical Review Panel
• Evaluation of
• Forecast Icing Product (FIP) -
• Severity

2
Technical Review Panel Composition

• An FAA Representative
• Normally from the AWRP when the technology in
  question is a product of AWRP
• Warren Fellner, ATO-P Aviation Weather Research
  Program
• An NWS Representative
• Normally from the Office of Science and
  Technology
• Cecilia Miner, NWS Office of Climate, Water and
  Weather Services
• A disinterested third person
• A scientist knowledgeable in the subject matter
  but not directly involved in the development of
  the product
• Gordon Brooks, Air Force Weather Agency

3
TRP Mandate

• Advise the AWTT Board on the scientific validity
  of the product. Consider
• Scientific Validity. Verify that the products
  basis is consistent with sound scientific
  practices and that it does not involve
  assumptions which are invalid in the targeted NAS
  domain.
• Objective verification. Evaluate verification
  results which come from a comparison of the
  products output with truth.

4
TRP Evaluation

• Scientific validity
• The physical basis for FIP-Severity was found to
  be sound
• Objective verification
• Goal Independently evaluate FIP-Severity
• Methodology Pilot Reports were used to evaluate
  the quality of FIP-Severity. FIP-Severity was
  also compared to the Current Icing Potential
  (CIP) and AIRMETs.

5
TRP Evaluation (cont.)

• Objective verification results summary
• Overall the FIP severity algorithm is consistent
  and performs at least as well as the AIRMETs for
  MOG PIREPs over all four lead times.
• The FIP Severity field is able to discriminate
  between the No icing (Icing Intensity 0)
  severity category and the positive icing severity
  categories for all lead times.
• The forecasting performance of the FIP severity
  field was consistent over all valid times as well
  as over different geographic regions.

6
TRP Evaluation (cont.)

• Reasonably calibrated for lower probabilities,
  but underforecasts rare events that occur at the
  higher probabilities
• Results are consistent with the CIP calibration
  study.
• Additional calibration effort needed prior to D4
  to further enhance accuracy over differing
  circumstances such as across seasons.

7
TRP Conclusions

• Unanimous agreement that the FIP-Severity is
• Scientifically valid
• Sufficiently accurate
• Recommend AWTT Board D3 (experimental status)
  approval from a scientific/technical perspective

8

• BACKUP SLIDE

9
Objective Verification Results

• PODy Percentage of Yes observations that were
  correctly forecast (Best 1)
• PODn Percentage of No observations that were
  correctly forecast (Best 1)
• TSS Level of discrimination between Yes and No
  observations (Best value 1, no skill 0)
• Percent Area Percent of the total possible
  area that had a Yes forecast