What is ESMF and what does it mean to adopt it

1
What is ESMF and what does it mean to adopt it?
NCAR/LANL CCSM
Climate
Data Assimilation
GFDL FMS Suite
NASA GMAO Analysis
GMAO Seasonal Forecast
Weather
MITgcm
3rd ESMF Community Meeting Cecelia DeLuca
cdeluca_at_ucar.du Nancy Collins nancy_at_ucar.edu
Jon Wolfe jwolfe_at_ucar.edu
NCEP Forecast
2
Outline

• What is ESMF?
• ESMF users
• A closer look at ESMF
• Adoption strategies
• A couple suggestions for how to get started

3
ESMF Architecture

• ESMF provides an environment for assembling
  geophysical components into applications, with
  support for ensembles and hierarchies.
• ESMF provides a toolkit that components use to
• increase interoperability
• improve performance portability
• abstract common services

4
Hierarchies and Ensembles

• ESMF encourages applications to be assembled
  hierarchically
• Coupling interfaces are standard at each layer
• Components can be used in different contexts

ESMF supports ensembles with multiple instances
of components running sequentially (and soon,
concurrently)
Ensemble Forecast
Seasonal Forecast
assim_atm
sea ice
ocean
assim_atm
assim_atm
assim_atm
assim
atmland
atm
land
coupler
physics
dycore
5
ESMF Data Classes

• Model data is contained in a hierarchy of
  multi-use classes. The user can reference a
  Fortran array to an Array or Field, or retrieve a
  Fortran array out of an Array or Field.
• Array holds a Fortran array (with other info,
  such as halo size)
• Field holds an Array, an associated Grid, and
  metadata
• Bundle collection of Fields on the same Grid
• State contains States, Bundles, Fields, and/or
  Arrays
• Component associated with an Import and Export
  State

6
ESMF Utilities

• Communication libraries
• Regridding library (parallelized, on-line SCRIP)
• Message logging
• Calendar management clocks, alarms, calendars
• Configuration (replaces namelists)
• IO currently netCDF
• Performance profilng (not implemented yet)

7
Outline

• What is ESMF?
• ESMF users
• A closer look at ESMF
• Adoption strategies
• A couple suggestions for how to get started

8
What kind of user are you?
Package code to run many places under many
systems, support, documentation
Provide components to others without large
overhead
Provider
Support routines, utilities, communication
code, leverage code developed elsewhere
Full framework, inter-platform,
inter-institution solutions
User
Individual/Small Group
Large Group/Institution
9
What do you get from ESMF?

• Different running configurations
• Standalone
• Coupled to data only (dead components)
• Coupled to live components
• Strategy for structuring complicated applications
• Different communication strategies
• Different data decompositions
• Utilities

10
What do you get (cont.)

• Standard component interfaces
• Coupler-building utilities

11
Outline

• What is ESMF?
• ESMF users
• A closer look at ESMF
• Adoption strategies
• A couple suggestions for how to get started

12
ESMF Superstructure Classes

• Gridded component
• Models, data assimilation systems - real code
• Coupler component
• Data transformations between Gridded Components
• State - Data sent between Components
• AppDriver Generic driver

13
ESMF Components

• Your code goes here
• Framework specifies interface, user supplies code
  which is to be called
• Initialize/Run/Finalize subroutine paradigm
• Can create nested subcomponents

14
ESMF Gridded Components

• Does the computational work
• Depending on how the current code is structured,
  may be possible to wrap without structural
  changes
• Or might use ESMF conversion for excuse to make
  structural changes!

15
ESMF Gridded Components (cont)

• States for import/export
• Sequential mode (concurrent coming soon)
• All import data available at start
• All export data ready at end
• Registration routine (SetServices) to tell the
  framework what routines youre supplying

16
ESMF Coupler Components

• States for import/export
• Does the transformation needed to take data from
  one Component and make it fit the requirements of
  another Component
• Not automatic - needs to be customized for each
  new configuration
• Expected to be thin, however - making use of the
  transformation routines in ESMF

17
ESMF States

• Description/reference to other ESMF data objects
• All data passed between Components is in the form
  of States and States only
• Data is referenced so does not need to be
  duplicated
• Can be Bundles, Fields, Arrays, States, or
  name-placeholders

18
ESMF Infrastructure Data Classes

• Grids Coordinates and grid decomposition
• Arrays Hold data
• Fields include Grid, Array, and metadata
• Bundles Fields packed together for convenience,
  data locality, latency reduction

19
ESMF DataMap Classes

• These classes give the user a systematic way of
  expressing interleaving and memory layout, also
  hierarchically (partially implemented)
• ArrayDataMap relation of array to decomposition
  and grid, row / column major order, complex type
  interleave
• FieldDataMap interleave of vector components
• BundleDataMap interleave of Fields in a Bundle

20
ESMF TimeMgr

• Clocks - timesteps, time intervals
• Calendars - many different varieties
• Alarms - periodic, fixed time
• Can be used independent of other parts of ESMF

21
ESMF Parallelism

• VM - hardware and OS abstraction
• DELayout - data decomposition
• Grids (ESMF_GridDistribute)

22
Virtual Machine (VM)

• VM handles resource allocation
• Elements are Persistent Execution Threads or PETs
• PETs reflect the physical computer, and are
  one-to-one with Posix threads or MPI processes
• Parent Components assign PETs to child Components
• The VM communications layer does simpleMPI-like
  communications between PETs (alternative
  communication mechanisms are layered underneath)

23
DELayout

• Handles decomposition
• Elements are Decomposition Elements, or DEs
  (decomposition thats 2 pieces in x by 4 pieces
  in y is a 2 by 4 DELayout)
• DELayout maps DEs to PETs, can have more than one
  DE per PET (for cache blocking, user-managed
  OpenMP threading)
• Simple connectivity or more complex connectivity,
  with weights between DEs - users specify
  dimensions where greater connection speed is
  needed
• Array, Field, and Bundle methods perform inter-DE
  communications

24
ESMF Communications

• Halo
• Updates edge data for consistency between
  partitions
• Redistribution
• No interpolation, only changes how the data is
  decomposed
• Regrid
• Based on SCRIP
• Methods include bilinear, conservative,
  higher-order conservative
• Bundle, Field, Array-level interfaces

25
ESMF Quickstart

• Directory with the shell of an application
• 2 Gridded Components
• 1 Coupler Component
• 1 Top level Gridded Component
• 1 AppDriver main program

26
ESMF Quickstart Application
AppDriver (generic main program)
Top level Gridded Component
GridComp 1
CplComp
GridComp 2
27
Outline

• What is ESMF?
• ESMF users
• A closer look at ESMF
• Adoption strategies
• A couple suggestions for how to get started

28
Adoption Strategies Top Down

• Wrap existing application as ESMF Component
• Wrap existing data with Fields and put them into
  States
• Can run as a Component at this point
• Integrate Field level communication
• Use Time Manager, Config, LogErr

29
Adoption Strategies Bottom Up

• Use ESMF utilities as needed
• Incorporate the Time Manager, Config
• Use Grids, Arrays for internal data
• Use VM, DELayout routines for data decomposition
• Wrap data structures as Fields
• Use data communication routines like Halo, Regrid

30
How can you learn more?

• Web site
• Examples
• Documentation
• Interoperability experiments
• Users forum

31
What do you need to get started?

• Supported build platforms include SGI, IBM,
  Compaq, many Linuxes
• C (gcc ok) and Fortran 90/95 compiler
• MPI (or mpiuni substitute lib)
• Understand how to run a job

32
Where do you go from here?

• Download ESMF 2.0 and compile it
• or find someone who already has it on your
  platform of choice!
• Run the demo
• Run the quickstart application
• Look at the examples
• Start writing code!