GOES-R%20Products%20and%20Their%20Algorithms

1
GOES-R Products and Their Algorithms

• Jaime Daniels, AWG Deputy Program Manager
• Mitchell D. Goldberg, AWG Program Manager
• Walter Wolf, Algorithm Integration Manager
• Zhaohui Cheng, Quality Assurance/EVM Manager
• Kenny Lowe, AWG Program Support
• AWG Application/Development Teams
• NOAA/NESDIS Center for Satellite Applications and
  Research

2
OUTLINE

• The Algorithm Working Group (AWG) A Brief
  Introduction
• AWG Progress in GOES-R Level-2 Product
  Development and Validation
• Product Examples, Algorithm Highlights, and
  Operational Applications
• Summary and Future Work

3
The AWG A Brief Introduction

• Purpose and make-up
• Roles and responsibilities
• Processes and standards

4
GOES-R Algorithm/Product Readiness
Underpinning Research Development (new applications Day 2 Products) Risk Reduction Pre Post Launch Sensor Calibration and Validation Calibration Working Group Operational Algorithm Readiness Development and Transition to Operations Algorithm Working Group Sustained Post Launch Validation and Reactive Science Maintenance Algorithm Working Group Proving Grounds and User Readiness Proving Ground Risk Reduction
Key components of a successful satellite
program Established GOES-R programs/activities
and working groups that directly support these
components
4

5
Algorithm Working Group
PURPOSE To select, develop, test, validate, and
demonstrate Level-2 algorithms that meet the
GOES-R FPS requirements and provide them to the
GOES-R Ground Segment. Provide sustained life
cycle validation and Level-2 product enhancements

• Leverages nearly 100 scientists from NOAA, NASA,
  DOD, EPA, and NOAAs Cooperative Institutes
  (University partners)
• Applies first-hand knowledge of algorithms
  developed for POES, GOES, DMSP, AIRS, MODIS,
  MetOP and Space Weather.
• Leverages other programs experiences (GOES,
  POES, MODIS, AIRS, IASI, NPOESS and other
  prototype instruments and international systems)
• Seeks to facilitate algorithm consistency across
  satellite platforms -- prerequisite for GEOSS
  (maximize benefits and minimizes integration)

6
AWG Teams

• Product Application Teams Plan and execute the
  activities to assess, select, develop, validate,
  and deliver level-2 product algorithms
• Product Development Teams Code, host, and test
  candidate level-2 product algorithms in a
  scalable operational demonstration environment
  and develop validation tools
• Proxy Team Responsible for the development of
  high-quality GOES-R instrument simulated and
  proxy data sets for GOES-R product algorithm
  development, testing and validation
• Integration Team Establishes requirements,
  standards, infrastructure, architecture,
  integrates software from the product development
  teams, and prepares deliveries to Ground Segment
  Project

7
Product Application Teams
GOES-R Products Mapped to Algorithm Application
Teams

• Imagery (Tim Schmit)
• Soundings (Tim Schmit, Chris Barnet)
• Winds (Jaime Daniels)
• Clouds (Andy Heidinger)
• Aviation (Ken Pryor, Wayne Feltz)
• Hydrology (Robert Kuligowski)
• Land Surface (Bob Yu)
• Cryosphere (Jeff Key)
• Radiation Budget (Istvan Lazslo)
• Lightning (Steve Goodman)
• Space Environment (Steven Hill)
• SST and Ocean Dynamics (Alexander Ignatov)
• Aerosols / Air Quality / Atmospheric Chemistry
  (Shobha Kondragunta)
• Proxy Data (Fuzhong Weng)
• Cal/Val (sensor) (Changyong Cao)
• Algorithm Integration (Walter Wolf)

8
AWG Follows a High Maturity Process and Adheres
to Established Standards

• Standards
• Algorithm Theoretical Basis Document
• Metadata (FGDC guidelines)
• Interface Control
• System Description
• Users Manual
• Fortran Programming
• C and C Programming
• Test Plan
• Algorithm Implementation Instructions
• Latency Reports

• Processes
• Initial Requirements Analysis
• Algorithm Design Review
• Critical Design Review
• Test Readiness Review
• Code Unit Test Review
• Algorithm Readiness Review
• EVM Reporting (monthly)

(Development gates)
AWG adherence to its established processes and
standards in its algorithm development activity
reduces risk associated with the development of
the Level-2 product algorithms and their delivery
to the GOES-R program
9
Algorithm Selection ProcessConsiderations for
algorithm selection

• Leverage Will or can the algorithm take
  advantage of new capabilities offered by new
  GOES-R instruments? Growth potential exist?
• Robustness Assess the strengths weaknesses and
  associated risks of algorithms. Will algorithm
  generate products that meet requirements and user
  needs under all/most conditions?
• Continuity Use of heritage algorithms (to
  maintain climate record, for example)
• Synergy With LEO product algorithms with
  international community

Outcome of algorithm selection process is a suite
of algorithms for GOES-R instruments that are
expected to be computationally efficient, robust,
easy to implement and maintain, and meet their
respective requirement specifications
10
OUTCOMEAWG Deliverables

• Algorithm Packages (APs)
• Algorithm Theoretical Basis Documents (ATBD)
• Instrument proxy datasets
• Product output datasets (for comparison)
• Algorithm Interfaces and Ancillary Data
  Description (AIADD) document
• Schedule of Deliveries
• September 2008 As-Is ATBDs
• September 2009 80 APs for Baseline Products
• September 2010 80 APs for Option 2 Products
• 100 APs for Baseline Products
• September 2011 100 APs for Option 2 Products

11
High Confidence in ABI Algorithms Meeting
Requirements

• Algorithms from MODIS and current GOES program
  are being leveraged
• EUMETSAT SEVIRI instrument serves as excellent
  proxy data source
• Current GOES and MODIS instruments serve as ABI
  proxy datasets
• High fidelity simulated datasets for ABI have
  been generated
• Government and University expertise from relevant
  current programs

ABI
SEVIRI
Similar spectral channel experience provides
confidence the algorithms will be delivered with
minimal program risk while meeting the required
accuracies
12
AWG Progress in GOES-R Level-2 Algorithm
Development and Validation

• Baseline and Option-2 Products
• Algorithm Development and Validation Strategies
• Proxy and Validation Data Sources

13
GOES-R BASELINE OPTION-2 PRODUCT SUMMARY
BASELINE Products
OPTION 2 Products

• Cloud Layer/Heights
• Cloud Ice Water Path
• Cloud Liquid Water
• Cloud Type
• Convective Initiation
• Turbulence
• Low Cloud and Fog
• Enhanced V/Overshooting Top
• Aircraft Icing Threat
• SO2 Detections (Volcanoes)
• Visibility
• Upward Longwave Radiation (TOA)
• Downward Longwave Radiation (SFC)
• Upward Longwave Radiation (SFC)
• Total Ozone
• Aerosol Particle Size
• Surface Emissivity
• Surface Albedo
• Vegetation Index

• Clouds and Moisture Imagery (KPP)
• Clear Sky Mask
• Cloud Top Pressure and Height
• Cloud Top Phase
• Cloud Top Temperature
• Cloud Particle Size Distribution
• Cloud Optical Path
• Temperature and Moisture Profiles
• Total Precipitable Water
• Stability Parameters (Lifted Index)
• Aerosol Detection
• Aerosols Optical Depth
• Derived Motion Winds
• Hurricane Intensity
• Fire/Hot Spot Characterization
• Land and Sea Surface Temperature
• Volcanic Ash
• Rainfall Rate
• Snow Cover

Advanced Baseline Imager (ABI)
Advanced Baseline Imager (ABI)
GLM
14
Algorithm Development StrategyA wide variety of
instrument proxy datasets have been assembled
and are being used
Real ABI PROXY Data Sources
Simulated ABI Proxy Data Sources
(FD, CONUS, Meso)
Case Studies
10.35um (Hurricane Lili)
11.2 um (HurricaneKatrina)
AWG Proxy and Product Application Teams have
assembled a wide variety of instrument proxy and
simulated datasets to use for algorithm
development, testing, and validation activities
15
High Fidelity Simulated ABI Datasets
GOES-12 Band 4 (10.7um) BT(K) GOES-R
Simulated ABI Band 14 (11.2um) BT(K)
AWG Proxy Team at CIMSS has provided high
fidelity simulated datasets that are important
for algorithm development and validation
activities
16
(No Transcript)
17
Algorithm Validation Strategy A wide variety of
reference (Ground Truth) datasets have been
assembled and are being used
AWG Product Application Teams and the Proxy Data
Team have assembled a wide variety of Ground
Truth datasets to use for Level-2 product
validation activities
18
Algorithm Development andValidation
StrategiesAn iterative process
Algorithm Iterations

• As algorithms mature
• Better estimates of product performance
• Increased confidence that on-orbit product
  performance will meet specs
• Increased confidence that user needs are met

Seasonal conditions represented Wide variety of
atmospheric and surface conditions are represented
MORE IS BETTER!
AWG is responsible for Level-2 product accuracy
and precision specifications, and has therefore,
worked to establish robust pre-launch validation
strategies for each product
19
Algorithm Testing and ValidationThe Outcome
AWG
Proxy Team
GOES-R Ground Segment Project (GSP)
Algorithm Integration Team
Product Application Teams
GSP delivers Algorithm Packages to Harris

• Each Product Application Team delivers to the
  AIT
• Three internal deliveries of algorithm that lead
  up to the Algorithm Packages having a maturity
  level of at least 80
• Two internal deliveries of algorithm that lead
  up to the Algorithm Packages having a maturity
  level of 100

• AIT delivers to the GSP
• Algorithm packages at a maturity level of 80 or
  greater
• Algorithm packages at maturity level of 100

GOES-R Ground Segment Vendor (Harris)
Outcome of algorithm testing and validation
process is a suite of algorithms for GOES-R
instruments that have been demonstrated to be
computationally efficient, robust, easy to
implement and maintain, and meet the their
respective requirement specifications.
20
AWG Algorithm FrameworkPrototype developed by
AIT

• Test bed for algorithm development and
  performance testing.
• Enables testing of code, algorithm integration,
  compilers, use of common ancillary data and
  forward models
• Used for verification of Level-2 algorithm
  performance (accuracy precision specifications)
  

See AIT Poster GOES-R AWG Product Processing
System Framework
21
GOES-R ABI Level-2 Baseline Product Examples and
Algorithm Highlights

• Product illustrations
• Algorithm highlights
• ABI attributes leveraged
• Operational applications

22
Cloud ProductsCourtesy of the Cloud Application
Team

23
Clear Sky Mask (Cloud Detection)

• Algorithm Highlights
• Designed to maximize flexibility and use
• Mask built with multiple individual tests that
  could be turned on and off by the downstream
  applications.
• Uses new ABI Bands (1.38um, 1.6um)
• Design allows for additional of new tests easily
  as warranted.
• Individual tests were taken from various cloud
  masks developed by the team.
• Determination of test thresholds accomplished
  through an analysis of CALIPSO data.
• Operational Applications
• Used extensively by downstream level-2 product
  algorithms
• Identifies clear pixel radiances for NWP data
  assimilation

23
23
23
24
Cloud Height

• Algorithm Highlights
• An optimal estimation approach is used to
  estimate cloud temperature, cloud emissivity and
  a cloud microphysical index.
• Algorithm currently uses the 11, 12 and 13.3mm
  channels.
• Cloud pressure and height are computed from NWP
  profiles.
• Special processing occurs in the presence of
  multi-layer cloud and clouds in the presence of
  inversions.
• Operational Applications
• Aviation Terminal Aerodrome Forecasts (TAFs)
• Supplements Automated Surface Observing System
  (ASOS) with upper-level cloud information
• Cloud initialization
• Assimilation into mesoscale NWP models

24
24
24
24
24
25
Cloud Phase

• Algorithm Highlights
• An Infrared only algorithm that exploits the rich
  IR information (7.4, 8.5, 11.2, and 12.3 mm)
  provided by the ABI
• Exploits recent improvements in fast clear-sky
  radiative transfer models and ancillary data
  (land cover, surface emissivity
• Makes advanced use of spatial information
• Operational Applications
• Prerequisite for other cloud property retrievals
  (height, temp)
• Climate prediction
• Aviation forecasting (Aircraft icing)

Clear
Spare
Thin Ice
Thick Ice
Multilayered Ice
Mixed Phase
Liquid Water
Super-Cooled Water
25
25
25
25
25
26
Land ProductsCourtesy of the Land Application
Team

27
Land Surface Temperature

• Algorithm Highlights
• Regression-based algorithm that uses the 11.2 and
  12.3 mm channels
• Split-window algorithm has significant heritage
  (geo leo)
• Leverages ABIs higher spatial resolution data
• Operational Applications
• Fog forecasting
• Frost/freezing temperature forecasting
• Assimilation into land surface models
• Assimilation into mesoscale and climate NWP
  models
• Climate prediction

27
27
27
28
Fire/Hot Spot Characterization

• Algorithm Highlights
• Heritage lies with the GOES operational Wildfire
  Automated Biomass Burning Algorithm (WF_ABBA)
• Dynamic, multi-spectral, thresholding contextual
  algorithm
• Utilizes the 0.64, 3.9, 11.2 and 12.3 mm channels
• Leverages ABIs higher spatial and temporal
  resolution data
• Operational Applications
• Fire weather forecasting
• Air quality forecasting

Fire mask product in Bolivia derived from MODIS
observations 07 September 2007
28
28
28
28
28
29
Derived Motion Wind ProductCourtesy of the
Winds Application Team

30
Derived Motion Winds

• Algorithm Highlights
• Heritage in targeting, tracking, and QC
  algorithms lie with current NESDIS operational
  winds algorithms
• Wind height assignment will rely on utilization
  of pixel level cloud heights generated upstream
  via algorithms delivered by AWG cloud application
  team
• Leverages ABIs higher spatial and temporal
  resolution data
• Operational Applications
• Weather Forecasting
• Assimilation into mesoscale and global NWP models
• Aviation (flight routing)

30
30
30
30
30
31
Temperature Moisture Soundings, Precipitable
Water, Atmospheric StabilityCourtesy of the
Soundings Application Team

32
Total Precipitable Water (TPW)Lifted Index (LI)
(mm)
TPW

• Algorithm Highlights
• 1D-variational physical retrieval algorithm that
  has heritage with MODIS and current operational
  GOES sounder physical retrieval algorithms
• Regression-based initial guess T/Q profiles
• Utilizes NWP forecast T/Q profiles
• Utilizes the 6.15, 7.0, 7.4, 8.5, 9.7, 10.35,
  11.2, 12.3, and 13.3 mm bands)
• Exploits recent improvements in fast clear-sky
  radiative transfer models and ancillary data
  (surface emissivity)
• Operational Applications
• Nowcasting
• Gulf of Mexico return flow
• Southwest US monsoon
• QPF (heavy rain, flash flooding)
• Convective potential and morphology
• Fog potential
• Situational awareness in pre-convective
  environments for potential watch/warning
  scenarios
• NWP
• Assimilation into regional and mesoscale NWP
  models (TPW)

(deg C)
LI
32
32
32
32
32
33
Sea Surface TemperatureCourtesy of the Sea
Surface Temperature Application Team

34
Sea Surface Temperature
SST product derived from MSG2/SEVIRI observations
for 28 March 2008

• Algorithm Highlights
• Hybrid approach that combines the advantages of
  regression (heritage approach) with a physical
  retrieval approach (optimal estimation)
• Utilizes the 3.9, 8.5, 10.35, 11.2, 12.3mm bands
• Exploits recent improvements in fast clear-sky
  radiative transfer models
• Leverages increased ABI temporal resolution
• Operational Applications
• Assimilation into atmospheric and oceanic models
• Climate monitoring/forecasting
• NOAA Coast Watch Program
• Harmful Algal Bloom monitoring
• Sea turtle tracking
• Vessel positioning
• Upwelling identification
• Commercial fisheries management
• NOAAs Coral Reef Watch Program
• Coral bleach warnings and assessments

Inversion Algorithm
Regression Algorithm
34
34
34
34
34
35
Rainfall Rate/QPECourtesy of the Hydrology
Application Team

36
Rainfall Rate/QPE

• Algorithm Highlights
• Self-Calibrating Multivariate Precipitation
  (SCaMPR) retrieval algorithm
• VIS/IR-based algorithm that is dynamically
  calibrated against satellite-based microwave (MW)
  derived rain rates (SSM/I, AMSU, AMSR-E and TRMM)
• Calibration is continuously updated to reflect
  time changes in MW-IR relationship
• Dynamic channel selection
• Leverages ABIs higher spatial and temporal
  resolution data
• Operational Applications
• Flash flood forecasting
• Nowcasting
• Assimilation into hydrologic models

Rainfall rate derived from Meteosat-8 SEVIRI 1600
UTC on 24 August 2006
mm/hr
36
36
36
36
36
37
Solar InsolationCourtesy of the Radiation
Budget Application Team

38
Downward Solar Insolation SFCReflected Solar
Insolation TOA
Solar Insolation derived from Terra MODIS 1020
UTC on 24 August 2006

• Algorithm Highlights
• Hybrid algorithm that combines the merits of
  candidate NASA (direct path) and STAR/UMD
  (indirect path) algorithms
• Physically-based retrieval by using a Look-Up
  Table (LUT) representation of the RTM
• Based on the NASA/CERES, NOAA/GOES and GEWEX/SRB
  heritages
• Leverages ABIs higher temporal resolution data
  and spectral coverage (VIS/near IR)
• Operational Applications
• Climate studies
• Surface (land and ocean) energy budget models
• - Assimilation into
• - Independent verification of
• Crop modeling
• Fire risk assessment
• Earth energy budget studies

Reflected TOA
Downward SFC
38
38
38
38
38
39
Snow CoverCourtesy of the Cryosphere
Application Team

40
Snow Cover
MODIS Color Composite (Colorado Rockies) 30
April 2007

• Algorithm Highlights
• Retrieves sub-pixel fractional snow cover and
  grain size estimates via computationally
  efficient spectral mixture modeling
• Heritage derived from MODIS-based fractional snow
  cover and grain size (MODSCAG) algorithm
• Leverages ABIs higher temporal resolution data
  and spectral coverage (VIS/near IR)
• Operational Applications
• Assimilation into NOAAs NOHRC snow model
• Hydrologic forecasts and warnings, including
  river and flood forecasts
• Stream-flow forecasting/modeling
• Snowpack monitoring, analysis
• Water management
• Climate studies

40
40
40
40
40
Retrieved Fractional Snow Cover
41
Aerosol Detection, Aerosol Optical
DepthCourtesy of the Aerosols, Air Quality,
Atmospheric Chemistry Application Team

42
Aerosol Detection
MODIS Color Composite (British Columbia, Canada
Fire Event) 19 August 2003

• Algorithm Highlights
• Spectral threshold algorithm that uses spectral
  (wavelength dependent) characteristics of
  surface, aerosols, and clouds to identify
  aerosols
• Heritage derived from AVHRR and MODIS-based
  aerosol detection algorithms
• Synergy with VIIRS aerosol retrieval algorithm
• Leverages ABIs higher spectral (VIS/near IR
  portion) coverage data
• Operational Applications
• Air quality forecasting
• Air quality assessment and management
• Climate studies

Smoke Flag
42
42
42
42
42
Retrieved Fractional Snow Cover
43
Volcanic AshCourtesy of the Aviation
Application Team

44
Volcanic Ash
SEVIRI Color Composite (Karthala Volcano
Eruption) on 24 November 2005

• Algorithm Highlights
• Detects volcanic ash and estimates its height and
  mass loading
• The 8.5 mm, 11.2 mm, and 12.3,11.2 mm channel
  pairs are used to detect volcanic ash
• An optimal estimation approach is used to
  estimate ash cloud temperature, emissivity and
  microphysical index.
• Ash cloud height determined from NWP profiles.
• Mass loading estimated from computed optical
  depth and effective particle size
• Leverages ABIs new 8.5 mm channel, along with
  11.2 mm and 12.3 mm for sensitivity to cloud
  microphysics (including composition)
• Operational Applications
• Aviation safety
• Health safety
• Climate studies

44
44
44
44
44
0 2 4 6 8 10 12
14
45
Summary and Future Work

• AWG has made significant progress
• Established processes and standards are in place
  and being executed
• Algorithm processing framework is in place
• All baseline product algorithms have successfully
  gone through development review gates (ADR, CDR,
  TRR)
• Successfully delivered baseline product algorithm
  packages (80) to the GOES-R Ground Segment
  Project (Sept 30, 2009)
• Complete our scheduled algorithm development
  activities
• Baseline (100) Option-2 (80) algorithm
  deliveries (Sept 2010)
• Option-2 (100) algorithm deliveries (Sept 2011)
• Support algorithm implementation activity being
  done by the Harris Team
• Support User Readiness and Training Activities
• GOES-R Proving Ground
• GOES-R Risk Reduction