ESMF Development Status and New Directions

1
ESMF Development Status and New Directions
Cecelia DeLuca ESMF 5th Community
Meeting Baltimore, MD May 23, 2006
Climate
Data Assimilation
Weather
2
ESMF Development Status

• ESMF Version 2.2.2
• Component and coupling layer stable
• Components can run concurrently or sequentially,
  single or (limited) multiple executable support
• Support for configuring ensembles
• Logically rectangular grids with regular and
  arbitrary distributions can be represented and
  regridded
• Other parallel methods - e.g. halo,
  redistribution, low-level comms implemented
• Utilities such as time manager, logging, and
  configuration manager usable and adding features
• Fortran interfaces and Reference Manual, some C
  interfaces
• Test suite, examples, demo, and component
  templates distributed with source code
• Internal release 3.0.0 new index space layer
• supports regridding of many more kinds of grids

3
Highest Priorities

• Usability
• Simplifying build and software interfaces
• Improving documentation
• Completing, fixing bugs in, and optimizing
  existing functions
• Adding better tutorial materials and realistic
  code examples
• More grids! Support for representing and
  regridding general curvilinear coordinate grids,
  unstructured grids, and observational data
  streams
• Numerical accuracy of methods
• Performance optimization
• Customer support and training

4
Actions

• Design and implementation of more general grid,
  data, and decomposition data structures
• Many additions to user and developer
  documentation, especially in the area of grids
• Optimization of multi-field sends
• Development of realistic use test cases
• Addition of hands-on exercises to tutorials
• Build rework
• New hires with mathematics and modeling
  backgrounds

5
Results

• Internal Release ESMF Version 3.0.0 (May 2006)
• Very general multi-patch data structures
• Sparse matrix multiply for regridding can handle
  curvilinear coordinate grids and unstructured
  grids
• New data structures will be the building blocks
  for a variety of methods on cubed sphere,
  tripole, and other grids
• Greater user satisfaction with build
• Growing interest from groups working in related
  physical domains
• Increase in user contributions to source

6
Contributions to Core

• Past
• Configuration Manager (Arlindo da Silva, NASA
  GSFC)
• SCRIP Regridding (Phil Jones, LANL)
• I/O (WRF)
• Current
• Representation and regridding of 3D grids
• Bob Oehmke, University of Michigan (est. Dec 06)
• C interfaces
• Tim Campbell, NRL Stennis (est. Sept 06)
• Representation and manipulation of observational
  data streams
• Will Sawyer, NASA GMAO (est. July 06)
• Upcoming
• Direct intercomponent data transfers via State
  put and get
• Alan Sussman, University of Maryland (est. Dec
  06)
• Support for grids that are composites of general
  curvilinear coordinates, e.g., cubed sphere
• Ed Hill, MIT (est. Dec 06)

7
Supported Platforms and theESMF Test Suite

• IBM AIX (32 and 64 bit addressing)
• SGI IRIX64 (32 and 64 bit addressing)
• Compaq OSF1 (64 bit addressing)
• Linux Intel (32 and 64 bit addressing, with mpich
  and lam)
• Linux PGI (32 bit addressing, with mpich)
• Linux NAG (32 bit addressing, with mpich)
• Linux Absoft (32 bit addressing, with mpich)
• Linux Lahey (32 bit addressing, with mpich)
• Mac XLF (32 bit addressing, with lam)
• Mac Absoft (32 bit addressing, with lam)
• SGI Altix (64 bit addressing)
• Cray X1 (64 bit addressing)
• User-contributed G95
• Almost - NEC

The ESMF Test Suite consists of 2000 unit tests,
17 system tests, 40 examples. The Suite is run
nightly on the platforms above. The ESMF Test
Suite is bundled with the ESMF source code so
that users can run it themselves for
verification, debugging, and porting.
8
ESMF Requirements

• Support for representing, partitioning,
  communicating with, and regridding data based on
  general curvilinear coordinates and unstructured
  grids
• Advanced support for data assimilation systems,
  including data structures for observational data
  and adjoints for ESMF methods
• Looser couplings support for multiple
  executable and Grid-enabled versions of ESMF
• I/O including asynchronous I/O
• Support for nested, moving grids and adaptive
  grids
• Support for regridding in three dimensions and
  between different coordinate systems
• Advanced optimization and load balancing

9
Current Development Challenges

• Finish implementation of general array data
  structures
• Consolidation of new index space data structures
  with old
• Rework of grid data structures for more
  generality
• Reorganization of development team
• Ongoing development of training program
  (including hire of training program lead)

10
New Directions

• Extension into new domains and coupling to other
  frameworks
• Space weather
• Air quality
• Hydrology
• Increased support for C/C
• Extension into a new coupling paradigm (multiple
  executable, put/get anywhere)
• Extension into end-to-end problem solving
  environments and interaction with data systems
  (Curator project)

11
Space Weather Coupling SWMF
Single Executable
ESMF App Driver
ESMF Coupler
ESMF Earth System Component
ESMF SWMF Component
SWMF Driver

• The entire SWMF is wrapped as an ESMF component
• The SWMF component can communicate data to one or
  more Earth system components
• The application is run using an ESMF driver
• The application runs as a single executable

12
Space Weather Coupling CISM
Multiple executables Work just starting
ESMF App Driver
ESMF Coupler
CISM Model
ESMF Component 1 (e.g. land)
ESMF Component 2 (e.g. atm)
InterComm data transfer and sequencing

• InterComm is used for data transfers between an
  ESMF application and a CISM model
• These are in separate executables
• InterComm is wrapped in ESMF_StatePut() and
  ESMF_StateGet() commands so that it presents a
  standard interface on the ESMF side
• Data transfers can be from anywhere, to anywhere

13
More Framework Combinations

• PRISM and ESMF
• PRISM is a European climate framework
• ESMF and PRISM are working together through a
  NASA grant (PI, Max Suarez) and the World Climate
  Research Program (WCRP) to develop common
  conventions
• MOM4 ocean model at GFDL was used to demonstrate
  how one model can be instrumented to run in both
  PRISM and ESMF
• Common Component Architecture and ESMF
• CCA is a general high performance computing
  framework that covers many different domains, and
  offers web and Grid service interfaces
• Collaborators include LANL, ANL, LLNL, ORNL, etc.
• ESMF components can be readily adapted to run in
  CCA drivers, demo presented at SC2003
• OpenMI and ESMF
• OpenMI is a hydrology framework written in C
  (Windows C-like language)
• Exploring coupling using multiple executable
  capability

14
This time next year

• New index space layer fully developed
• Redesign of grid layer and additional methods for
  manipulation of multi-patch, general curvilinear
  coordinate grids and unstructured grids
• Additional training materials and documentation
• Basic IO in place