Ocean Models in Cactus

1
Ocean Models in Cactus

• Experiences with MOM and THCM
• Fokke Dijkstra
• Aad van der Steen (HPCG)
• Henk Dijkstra (IMAU)

2
Contents

• Intro
• Cactus
• Problems
• Howto
• THCM
• MOM

3
Computational framework

• Computational science project
• Astrophysics, Plasmaphysics, Oceanography,
  Mathematics, Computer Science
• Sharing knowledge, models etc.
• Oceanographic model coupling

4
Why Cactus?

• General
• C and Fortran interoperability
• Parallelisation model
• Active development
• Flexible structure

5
Ocean Modelling

• Grid based
• Many models available MOM, MICOM, POP,...
• PRISMFrontier ProjectESMF
• FMS

6
(No Transcript)
7
(No Transcript)
8
Existing Code

• Do not reinvent wheel
• Users familiar with code
• Rewriting lot of work
• Acceptance
• Limits use of framework?

9
Cactus

• Structure of flesh and thorns
• Generic types
• Thorn described by .ccl files
• Grid oriented

10
schedule.ccl

• Global structure of thorn
• Thorn divided into pieces
• Time bins

STARTUP loop EVOL ANALYSIS TERMINATE
11
interface.ccl

• Data structures
• Grid functions over grid
• Automatically distributed

CCTK_REAL fields TYPEARRAY SIZEnx,ny,nz
TimeLevels2 u,v,w,p,t,s
12
param.ccl

• Input for thorn
• Name
• Description
• Range

CCTK_INT nsteps Number of steps 0100
Maximum 100 steps 10
13
Problems Languages

• Calls between C and Fortran
• Strings
• Logicals
• Function aliasing

14
Problems Structure

• Modules
• Dimensions
• Cactus grid based, solvers matrix based
• Sparse matrices
• Schedule

15
Problems Parallelisation

• Automatic data distribution
• No data distribution?
• 1D/2D/3D distributions?
• Choices needed

16
Problems

• Fortran 90 modules
• cpp
• -r8, -i4 flags
• Single double precision constants

17
Howto Gradual approach

• Stay close to original code
• Check correctness every now and then
• Easy things first
• Monolithic Thorn ? Cactus Thorn

18
Howto Types

• Cactus types CCTK_REAL, CCTK_INT
• Change real into CCTK_REAL
• Do not change explicitly defined types (real4
  and similar)

? Easy
19
Howto Input

• Cactus parameter.ccl files
• Change thorn input to use DECLARE_CCTK_PARAMETERS
• Conflicting names
• Strings, logicals, changing variables

? Easy
20
Howto Structure

• Divide program into thorns if necessary
• Function aliasing
• Change main program into parts for schedule.ccl

? Doable
21
Howto Data structures

• Data structures of program
• Time levels
• Keep meaning of data clear
• Introduce Cactus Grid Functions/Arrays at top
• Work with them downwards

? Difficult
22
THCM

• Implicit model
• Fields ? matrix vector
• Compressed row storage
• Now integrated

23
THCM

• Types
• Thorns
• Parameters
• Schedule
• Fields

24
MOM

• MOM 3.1 ifdefs
• MOM 4 modular, Clear structure
• Integrated as monolithic thorn
• Types
• Input
• Function aliasing

25
Flexible Modelling system

• Similarities with Cactus
• Parallelisation model similar to Cactus
• Replacing FMS by Cactus should be possible
• Work just begun

26
Conclusions

• Cactus promising framework
• Easy application of other thorns
• Porting existing code is difficult
• Structural conflicts limit integration

27
Acknowledgments

• Cactus group (Gabrielle Allen, Tom Goodale, David
  Rideout, Ed Seidel)
• IMAU (Jeroen Gerrits, Lianke te Raa)
• NWO for funding