PERTURBATION VS' ERROR CORRELATION ANALYSIS PECA

1
THORPEX THe Observing system and Predictability
Experiment A World Weather Research Program
INTERNATIONAL PROGRAM OBJECTIVES SCIENCE
GOAL Promote research leading to new techniques
in Observations (OBS), Data
assimilation (DA), Forecasting (FCST), and
Socioeconomic Applications (SA) SCIENTIFIC
RESEARCH MUST ENABLE SERVICE GOALS SERVICE GOAL
Accelerate improvements in utility of forecasts
for high impact weather THORPEX ANSWER
Develop new paradigm for weather forecasting
through Enhanced collaboration on
international level between research
operations
2
STATUS OF PLANNING
International Science Plan Completed
accepted (Dec. 2005), 51 pages International
Implementation Plan (TIP) Completed accepted
(Febr. 2005), 93 pages US THORPEX Plan First
Draft ready in Aug. 2004 Under revision NOAA
THORPEX Science Implementation Plan Completed
in June 2003, 52 pages
3
NOAA SERVICE GOAL ACCELERATE IMPROVEMENTS IN
3-14 DAY FORECASTS3-7 day precipitation
forecasts8-14 day probabilistic daily weather
forecastsNOAA SCIENCE OBJECTIVE REVOLUTIONIZE
NWP PROCESS
TRADITIONAL NWP Each discipline developed
on its own Disjoint steps in forecast
process Little or no feedback One-way flow of
information Uncertainty in process ignored

• NEW NWP
• Sub-systems developed in coordination
• End-to-end forecast process
• Strong feedback among components
• Two-way interaction
• Error/uncertainty accounted for

SOCIOEC.
SOCIOEC.
SYSTEM
SYSTEM
INTEGRATED, ADAPTIVE, USER CONTROLLABLE
4
DIRECT LINK BETWEEN NOAA THORPEX SCIENCE AND
IMPLEMENTATION PLAN (NTSIP-2002) ANDTHORPEX
INTERNATIONAL SCIENCE PLAN THORPEX
IMPLEMENTATION PLAN (TIP)
TIP
TIP OBSERVING SYSTEM
TIP DATA ASSIMILATION
NTSIP
SOCIOECON.
SYSTEM
CROSS-CUTTING ACTIVITIES
GLOBAL INTERACTIVE FORECAST SYSTEM (GIFS)
TIP PREDICTABILITY DYNAMICAL PROCESSES
TIP SOCIAL ECONOMIC APPLICATIONS
5
STATUS - NOAA

• Grant program
• 12 studies funded at slightly reduced level
• 2 on Observing System
• 5 on Data Assimilation
• 3 on Predictability/Forecasting
• 1 on Socio-Economic Applications
• 1 on Cross-Cutting
• IPY Planning
• Pre-Proposal (Short-Term Arctic Predictability,
  STAP) submitted, Jan. 2005
• Predictability of weather, sea ice, ocean wave,
  and land surface in the Arctic on 3-90 days
• One of 10 NOAA submissions
• Coordinated with a group of 10 PIs (JCSDA, SIMMS,
  OAR, OGP, NIC, NWS, SIP)
• Full plan due June 2005
• Need to coordinate with US/NA/International
  THORPEX plans/efforts
• North American Ensemble Forecast System (NAEFS)

6
NORTH AMERICAN ENSEMBLE FORECAST SYSTEM

• Joint research/development with Met. Services of
  Canada Mexico
• Combine Canadian / US ensemble systems
• Seamless forecasts over borders and lead times
• Share development and computer resources
• Better represent forecast system related
  uncertainty
• Timeline
• May 2003 Kick-off Workshop
• Nov 2003 Research, Development Implementation
  Plan complete
• Sept 2004 Initial Operational Capability
• Nov 2004 Inauguration Ceremony 2nd Workshop
• High level management, press coverage
• Link with GEOSS
• Example for GEOSS empowered applications
• Phased operational implementation
• March 2006, 2007, 2008

7
ISSUES

• Long term budget planning
• Need to bring budget and RDI (research,
  development implementation) planning in sync
• Mismatch between science plan/high level promises
  and available resources
• Current 1.3M (2.3M for 06?) budget does not allow
  full NOAA participation
• Cannot take advantage of IPY-related
  opportunities
• Cannot fulfill GEOSS expectations (THORPEX IS
  GEOSS for global weather forecasting)
• Cannot take full advantage of other international
  collaborative efforts
• Cannot achieve NOAAs goal of promoting global
  environmental safety
• Immediate concerns
• Need funding for transition of grant-based
  research into operations
• Entering 2nd year of grant research
• Must find a solution, otherwise good efforts may
  be wasted
• Operational Test Center must be established
  (jointly with JCSDA)
• Need funding for directed research/development
  (by NOAA Labs others)
• To fill gaps in program (new fcst system will be
  as good as its weakest point allows)
• Foster interagency collaboration
• IPY offers good opportunity (and challenge)

8
BACKGROUND
9
NOAA THORPEX ORGANIZATIONAL CHART
NOAA THORPEX USWRP Sub-Committee Louis
Uccellini (Chair) NWS Michael Uhart OWAQ Marie
Colton ORA/NESDIS Gregg Mandt NWS Jim Kimpel
(OAR) NOAA THORPEX Science Steering
Committee Zoltan Toth Chair, Program Manager
NOAA/NWS Observations Data
Assimilation Jaime Daniels NOAA/NESDIS Craig
Bishop NRL David Emmitt SWA L.-P.
Riishojgaard JCSDA Chris Velden CIMSS Dezso
Devenyi NOAA/FSL Forecasting/Predictability S
ocioeconomic Applications Jim Hansen MIT
Rebecca Morss NCAR Jeff Whitaker/T. Hamill
NOAA/CDC James Wilczak NOAA/ERL George
Kiladis NOAA/AL
10
THORPEX A GLOBAL ATMOSPHERIC RESEARCH
PROGRAM NOAA LONG-TERM RESEARCH PROGRAM PLAN
Based largely on work of NOAA THORPEX Planning
Meeting (October 21-22 2002)

• NOAA NWS
• Zoltan Toth
• Naomi Surgi
• NOAA OAR
• Melvyn Shapiro
• Jeff Whitaker

• Outside NOAA
• Craig Bishop NRL
• David Carlson NCAR
• Ron Gelaro NASA
• Rebecca Morss NCAR
• John Murray NASA
• Chris Snyder NCAR

With further input from NOAA THORPEX Science and
Implementation Team
Acknowledgements D. Rogers, L. Uccellini, S.
Lord, J. Gaynor, W. Seguin
11
THORPEX A GLOBAL ATMOSPHERIC RESEARCH
PROGRAM NOAA LONG-TERM RESEARCH PROGRAM PLAN
INTRODUCTION New forecast paradigm SCIENCE
PLAN Major Themes Open Science
Questions Research and Development
Tasks IMPLEMENTATION PLAN Work
Plan Deliverables Performance
measures Education/Outreach Path to
Operations APPENDIX Link with NOAA Strategic
Goals NWS STIP Process
12
NOAA THORPEX PROGRAM OVERVIEW

• ANSWER SCIENCE QUESTIONS
• Advance basic knowledge,
• directed explicitly toward NWP applications
• Each task must be conceived as part of overall
  program
• DEVELOP NEW METHODS
• Sub-system development
• Academic research
• Cross-cutting activities
• Academic operational centers
• Infrastructure / Core tasks
• Facilitate other activities - Strong agency
  involvement
• Operational Test Facility
• Real time test and demonstration
• RECOMMEND/PREPARE OPERATIONAL IMPLEMENTATION
• Integral part of program
• Strong participation by operational centers

13
SCIENCE QUESTIONS ACTIVITIES

• Observing system (OBS)
• Data assimilation (DA)
• Forecast procedures (FCST)
• Socio-economic Applications (SA)
• Cross-cutting activities
• Core tasks

14
SCIENCE QUESTIONS ACTIVITIES - 1

• OBSERVING SYSTEM
• New in-situ and remote instruments/platforms to
  complement existing network
• Adaptive observing instruments/platforms
• For large data sets
• Super-obing etc prior to OR within data assimil.
• (Joint work with data assimilation)
• Obs. error estimation (correlated/uncorrelated)

15
SCIENCE QUESTIONS ACTIVITIES - 2

• Observing system
• DATA ASSIMILATION
• Improve techniques
• Forward models, transfer codes
• Thinning of data
• Treatment of data with correlated errors
• Advanced methods to use case dependent covariance
• 4DVAR research, e.g., continual update of error
  covariance
• Ensemble based techniques
• Treatment of model errors
• Adaptive observing techniques
• Quick use of targeted data (pre-emptive
  forecasting)
• Methods in the presence of
• Strong non-linearities
• Model error
• Effectiveness of targeted data in
  analyses/forecasts
• Effect on climatological applications of data

16
SCIENCE QUESTIONS ACTIVITIES - 3

• Observing system
• Data assimilation
• FORECAST PROCEDURES
• Initial ensemble perturbations (Joint with data
  assimilation)
• Role of non-modal behavior
• Separate model related error from initial value
  errors
• Systematic vs. random errors
• Atmospheric features most affected
• Critical model features responsible for different
  errors
• Improve model formulation to reduce errors
  (Coupling techniques)
• Techniques to account for remaining uncertainty
  in ensembles
• Adaptive modeling and ensemble techniques

17
SCIENCE QUESTIONS ACTIVITIES - 4

• Observing system
• Data assimilation
• Forecast procedures
• SOCIO-ECONOMIC APPLICATIONS
• Probabilistic forecasting
• Statistical post-processing
• New procedures for intermediate and end users
• Add-on costs of new THORPEX NWP process
• Cost of data from multi-use satellite platforms
  (Joint with Observtns.)
• Incremental societal/economic benefits of new NWP
  process
• New NWP verification measure
• Societal aspects of new adaptive NWP procedures
• Equitable use of NWP resources

18
CROSS-CUTTING ACTIVITIES

• Integrating NWP procedures from four sub-systems
• Observing System Simulation Experiments (OSSEs)
• Data needs of NWP
• What variables/resolution/accuracy required
• Instrument/platform neutral assessment
• What instruments/platforms can provide data needs
• Existing and new in-situ remote platforms
• Adaptive component to complement fixed network
• Most cost effective solution
• Relative value of improvements in four
  sub-systems
• Improvements in which sub-system offer best
  return?
• Reallocation of resources
• Test of proposed operational configurations
• Major field program if needed
• Cost/benefit analysis - Select most cost
  effective version

19
CORE TASKS

• Needed for efficient research planned
  operations
• Strong agency involvement
• THORPEX data base (observations, forecasts)
• Information Technology challenge
• High data volume
• Transmission
• Storage of data
• Foster collaboration in critical areas
• Workshops (Societal and economic impacts)
• Joint proposals Interdisciplinary collaboration
• Critical in past programs like FASTEX
• Test-bed Pathway from research to operations
• Formal procedure for researchers to follow
• Melting pot for new ideas
• Venue for cross-cutting activities

20
NOAA THORPEX OBJECTIVES

• 1) Develop new forecast procedures leading to
• Improved operational NWP forecasts and
• Develop/adapt cost/benefit tools to measure
  resulting societal impact
• ULTIMATE MEASURE OF SUCCESS
•  
• The overall success of the NOAA THORPEX program
  will be measured in a unique and comprehensive
  way. The program will be considered successful if
  the newly developed cost/benefit analysis tools
  (point 3 above) indicate that the forecast
  improvements (point 2) due to the new THORPEX
  procedures (point 1) can be achieved
  operationally in a cost-effective manner. That
  is, the incremental economic and societal
  benefits associated with the use of the new
  THORPEX forecast procedures outweigh their
  implementation and maintenance costs.
•  

21
LINK WITH NOAA MISSION GOAL

• NOAAS 3rd MISSION GOAL sounds like excerpt
  from THORPEX doc.
• NOAA will provide integrated observations,
  predictions, and advice for decision makers to
  manage environmental resources.
• Mission strategies and measures of success
• directly correspond with
• THORPEX Sub-program areas
• NOAA MISSION STRATEGY THORPEX FORECAST
  COMPONENTS
• Monitor and Observe Observations
• Understand and Describe Data Assimilation
• Assess and Predict Forecasting
• Engage, Advise, and Inform Socio-economic
  Applications
• Different Line Offices responsible for various
  forecast components
• NEED FOR NEW MATRIX MANAGEMENT CONCEPT FOR
  INTEGRATION

22
LINK WITH NWS STIP PROCESS

• National Weather Service (NWS)
• NOAAs operational weather forecast
  provider
• NWS Science and Technology Infusion Plan (STIP)
• Operational requirements should motivate all
  service oriented research
• Research must have thread to operations
• Credible path to operational implementation
• SCIENTIFIC RESEARCH MUST ENABLE SERVICE GOALS
• THORPEX seeks advanced knowledge on two fronts
• Nature (atmospheric and related processes)
• Forecast procedures (OBS, DA, FCST SA
  techniques)
• Integrating knowledge from two areas leads to new
  forecast paradigm of
• INTEGRATED, ADAPTIVE, AND USER CONTROLLABLE FCST
  PROCESS

23
NOAAS INVOLVEMENT IN THORPEX

• 1998-99 Discussions started with involvement of
  NOAA scientists
• Apr 2000 First International Meeting
• Mar 2002 First Workshop, International Science
  Steering Committee formed
• Aug 2002 NOAA Tiger Team Meeting
• Oct 2002 NOAA THORPEX Planning Meeting
• Nov 2002 1st Draft NOAA THORPEX Science and
  Implementation Plan
• Jan 2003 NOAA THORPEX Science and Implementation
  Committee formed
• Feb 2003 Pacific TOST Experiment (PTOST)
• Jun 2003 First NOAA THORPEX Announcement of
  Opportunity
• Sep 2003 25 Full Proposals received

24
4.2 THORPEX ORGANIZATION PROPOSED INTERNATIONAL
MANAGEMENT STRUCTURE

• Solid plan
• Reflects expected organization of work

ICSC In charge
Advisory bodies Science, Operations, Users
Executive Board (Headed by IPO Director)
Regions Asia, Europe, NA, etc
Working Groups Science, Demos, GIFS
25
THORPEX INTL ORGANIZATIONAL FLOWCHART
Core Research
Deliverable
Support
Observing System
Data Assim. Obs. Strateg.
Global Interactive Forecast System
Data Management and Policy
Predict- ability
TIGGE Data Base
Socio-Econ. Applications
Facilitates Res. Demo
26
THORPEX ORGANIZATION NATIONAL INTERNATIONAL
LINKS
INTERNATIONAL LEVEL
US NATIONAL LEVEL
REGIONAL LEVEL
International Core Steering Committee ICSC,
Chair M. Beland
US Interagency Coordination US Rep. R.
Rosen Alt. L. Uccellini
International Leadership
National Leadership
NOAA NTEC L. Uccellini
NSF J. Moyers
NASA R. Birk J. Kay
ONR S. Cheng
THORPEX Executive Board (TEB) / IPO ?
International Exec. Leaders
Agency Execs
Regional Committees Europe, Asia, etc
NA Committee US Rep D. Parsons
Agency Sci/Imp
NOAA Sci/Impl Z. Toth
NSF Program Man. L. Gates
Science, Demo, GIFS Working Groups ????
Intl. Science Implementation
NASA Sci/Impl T. Miller
ONR Sci/Impl. C. Reynolds
27
NOAA THORPEX ORGANIZATIONAL CHART
NTEC MEMBERS J. Hayes, D-OST/NWS J. Kimpel,
OAR M. Colton, D-ORA/NESDIS
NOAA Senior Management
CHAIR L. Uccellini, D-NCEP/NWS
Interagency Coordination
NOAA Line Offices
STAFF Z. Toth, NTSIC Chair M. Ralph F.
Toepfer, PPBES
NTSIC MEMBERS Obs. Systems J. Daniels (ORA), D.
Emmitt (SWA), C. Velden (U. Wisc/SIMMS) Data
Assim C. Bishop (NRL), L.-P. Riishojgaard
(NASA/JCSDA) Forecasting J. Hansen (MIT), G.
Kiladis (AL), D. Devenyi (FSL), J. Whitaker/T.
Hamill (CDC) SA Applications R. Morss (NCAR), J.
Wilczak (ETL)
NOAA Res. Labs AL, CDC, ETL, FSL, ORA
Regional Sci. Com. Science/Demo Working Groups
CHAIR Z. Toth, EMC/NCEP
NOAA Joint Inst. JCSDA, CIMMS, SIP
NWS-NCEP Operational Implementations F. Toepfer
Consistent with D. Rogers Memo of 7/9/2003,
establishing NTEC NTSIC
International Forecasting Activities NOAA THORPEX
Administrative Assistant
28
NOAA THORPEX ORGANIZATIONAL CHART ROLES AND
RESPONSIBILITIES
NOAA THORPEX EXECUTIVE COMMITTEE (NTEC)

• 1) Provide overall guidance
• 2) Secure necessary funding
• 3) Make all funding decisions
• 1) Develop and update NOAA THORPEX Science and
  Implementation Plan
• 2) Evaluate proposals submitted to NOAA THORPEX
  Research Grant Program (AO)

• Leads committee, makes final decisions if it
  lacks consensus
• Interfaces with USWRP/ International Executive
  Commit.
• Interfaces with Chair of NTSIC
• 1) Leads and coordinates NOAA THORPEX Program as
  directed by NTEC
• 2) Prepares draft budget
• Recommends funding for
• Research grants
• Operational implementation
• Infrastructure
• International commitments, etc

CHAIR
NOAA THORPEX SCIENCE AND IMPLEMENTATION COMMITTEE
(NTSIC)
CHAIR