ESMF Status

1
ESMF Status
Climate
NCAR/LANL CCSM
Data Assimilation
GFDL FMS Suite
NASA GMAO Analysis
Weather
GMAO Seasonal Forecast
MITgcm
Cecelia DeLuca Interagency Working Group
Meeting November 30, 2004

NCEP Forecast
2
Outline

• What is ESMF?
• ESMF Open Source Development
• Development Status
• Portability and Performance
• Funding Status
• Questions/Discussion

3
What is ESMF?

• ESMF provides tools for turning model codes into
  components with standard interfaces and standard
  drivers
• ESMF provides data structures and common
  utilities that components use
• to organize codes
• to improve performance portability
• for common services such as data communications,
  regridding, time management and message logging

4
Why Use ESMF?

• Interoperability ESMF enables modeling groups
  to exchange codes, compare codes, extend codes,
  and use codes in different contexts
• Reuse ESMF prevents redundant development of
  common utilities and offers robust toolkits with
  many features and good documentation
• Ease of Use ESMF standard component interfaces
  enable modelers to understand new codes quickly,
  and consistent look and feel across the entire
  framework helps users learn new functions quickly
  
• Performance Portability the ESMF virtual
  machine provides excellent portability, good
  performance, and a consistent interface to
  diverse architectures

5
Application Example GEOS-5

• Each box is an ESMF component
• Every component has a standard interface so that
  it is swappable
• New components can easily be added to the system
• Data in and out of components are packaged as
  state types
• Coupling interfaces are standardized but coupling
  methods are customizable
• Coupling tools include regridding and
  redistribution methods

6
Outline

• What is ESMF?
• ESMF Open Source Development
• Development Status
• Portability and Performance
• Funding Status
• Questions/Discussion

7
ESMF is a Community Effort

• The ESMF collaboration has roots in the
  grass-roots Common Modeling Infrastructure
  Working Group, and grew out of Earth system
  modelers desire for shared software
• Users define development priorities
• Users actively test and evaluate the framework
  design and implementation
• 15 of ESMF source code is from user
  contributions (IO from WRF, resource file manager
  from GMAO, regridding from Los Alamos)

8
Open Source Development

• Open source license (GPL)
• Open repositories web-browsable CVS repositories
  for source code and contributions
• Open development priorities and schedule
  priorities set based on user meetings, telecons,
  and mailing list discussions, web-browsable task
  lists
• Open testing 1000 tests are bundled with the
  ESMF distribution and can be run by users
• Open port status results of nightly tests on
  many platforms are web-browsable
• Open metrics test coverage, lines of code,
  requirements status are updated regularly and
  web-browsable

9
Open Source Constraints

• ESMF does not allow unmoderated check-ins to its
  main CVS repository (though there is minimal
  check-in oversight for the contributions
  repository)
• ESMF has a co-located, line managed Core Team
  whose members are dedicated to framework
  implementation and support it does not rely on
  a volunteer labor
• ESMF actively sets priorities based on user needs
  and feedback
• ESMF requires that contributions follow project
  conventions and standards for code and
  documentation
• ESMF schedules regular releases and meetings

The above are necessary for development to
proceed at the pace desired by sponsors and
users, and to provide the level of quality and
customer support necessary for codes in this
domain
10
Outline

• What is ESMF?
• ESMF Open Source Development
• Development Status
• Portability and Performance
• Funding Status
• Questions/Discussion

11
ESMF Status

• Overall architecture well-defined and
  well-accepted
• Components, States, Fields, low-level
  communications mature
• Logically rectangular grids with regular and
  arbitrary distributions implemented
• On-line parallel regridding (bilinear, 1st order
  conservative) completed and optimized
• Other parallel methods, e.g. halo,
  redistribution, low-level comms implemented
• Utilities such as time manager, logging, and
  configuration manager usable and adding features
• Virtual machine with uniform interface to shared
  / distributed memory implemented, hooks for load
  balancing implemented

12
ESMF Near-Term Priorities(through June 2004)

• Concurrent components
• Wholly irregular grid distributions
• Grid merges
• Optimized regridding and low-level
  communications, including non-blocking
  communications
• More grids (including unstructured, tripolar,
  displaced pole, spectral, cubed sphere) and more
  regridding methods (bicubic, 2nd order
  conservative)
• Fractional times and real times
• Multi-file and other options for logging
• Reworked design and implementation of array /
  grid / field interface and array-level
  communications
• Asynchronous IO and read/write grid specifications

13
ESMF Long-Term Priorities

• Improve portability, performance, and error
  handling, and expand and improve documentation,
  tutorial materials, and training program
• Develop and assimilate contributions of new
  functionality into the ESMF software (nested and
  adaptive grids, data assimilation support
  including adjoints, MPMD, improved IO)
• Modify the collaboration environment and project
  organization so that it is effective with
  multiple sponsors and a larger number of
  collaborators
• Expand the program of collaboration with CCA,
  PRISM and other national and international
  infrastructure initiatives
• Begin design and implementation of Earth System
  Modeling Environment (ESME)

14
Some Metrics

• Core Team currently has 2 testers, .5 performance
  analyst, 5 developers, 1 admin/web support, 1
  manager
• 273 ESMF interfaces implemented, 250 fully or
  partially tested, 91 fully or partially tested.
• 142,000 SLOC, 26,000 lines of text
• 63 open bugs, 316 closed bugs
• 785 downloads

15
ESMF Current Challenges

• Quality and correctness of source code,
  especially numerical methods recent addition of
  tester contributed by NASA Goddard
• Process for design and interface review
• Lead for development of advanced grids
• Requirements database and requirements tracking
  new software packages being explored
• Clarity and completeness of documentation and
  training program materials in short term,
  possible addition of casual staff for technical
  editing

16
NASA Deliverable Schedule and Metrics

• Public delivery of prototype ESMF v1.0 in May
  2003
• Completion of first coupling demonstrations using
  ESMF in March 2004
• Delivered ESMF v2.0 in June 2004
• Delivery of ESMF v2.1.0 in December 2004
  (includes concurrency)
• All project codes scheduled to achieve partial
  adoption (use of the ESMF component layer and
  coupling) by November 2004
• All project codes scheduled to achieve full
  adoption (use of the component layer and coupling
  plus 3 or more utilities) by June 2005

17
Outline

• What is ESMF?
• ESMF Open Source Development
• Development Status
• Portability and Performance
• Funding Status
• Questions/Discussion

18
ESMF Platform Support

• SGI
• Altix
• IBM
• Compaq
• Linux (Intel, PGI, NAG, Absoft, Lahey)
• Mac xlf
• X1

19
ESMF Component Overhead

• Measures overhead of ESMF superstructure in NCEP
  Spectral Statistical Analysis (SSI), 1 overall
• Run on NCAR IBM
• Runs done by JPL staff, confirmed by NCEP
  developers

20
ESMF Regridding Performance, Initialization

• Comparison with the Argonne Model Coupling
  Toolkit (MCT) bundled with CCSM
• Run on NCAR IBM
• Runs done by JPL staff, not yet confirmed by
  Argonne developers

21
ESMF Regridding Performance, Run Time

• Comparison with the Argonne Model Coupling
  Toolkit (MCT) bundled with CCSM
• Run on NCAR IBM
• Runs done by JPL staff, not yet confirmed by
  Argonne developers

22
Outline

• What is ESMF?
• ESMF Open Source Development
• Development Status
• Portability and Performance
• Funding Status
• Questions/Discussion

23
Shared Overhead Cost
Travel, meeting, and other expenses related to
project administration and management computing
time.
24
NASA
Current development indicates that for about
six external staff trying to adopt or contribute
to ESMF it is useful to have 1 FTE dedicated to
development/support/integration (over and above
general support) on the core team.
25
NSF
26
NOAA
27
DoD
28
DoE
29
Resulting Allocation
30
Outline

• What is ESMF?
• ESMF Open Source Development
• Development Status
• Portability and Performance
• Funding Status
• Questions/Discussion