IO in Cactus 4'0

1
IO in Cactus 4.0

• Thomas Radke
• Albert-Einstein-Institute
• tradke_at_aei-potsdam.mpg.de

2
Outline

• Overview on Cactus IO Infrastructure- What is an
  IO method ?- What is provided by the flesh ?
• What kind of IO is available in Cactus ?
• How do I actually get some output,whats there
  to control it ?
• Usage of Checkpointing / Recovery

3
IO Infrastructure in Cactus 4.0
Flesh
Registration Scheduling of IO methods
IOFlexIO
IOHDF5
IOASCII
IOPanda
IOBasic
IOUtil

• Set of basic IO parameters
• Registration Scheduling of Recovery Methods

4
What is an IO method ?

• provided by IO thorns
• called by the flesh and/or application thorns
• consists of 4 routines

OutputGH(GH)
Loops over all variables on this GHand outputs
them if necessary
OutputVarAs(GH, var, alias)
Does unconditional output of variable var
TimeToOutput(GH, var)
Called by flesh for a thorns routineregistered
at CCTK_ANALYSISand triggered by var
TriggerOutput(GH, var)
5
Registration of IO methods

• register a new IO method with the flesh by
  giving it a unique name

IOMethod CCTK_RegisterIOMethod (method_name)

• register the necessary routines for this IO
  method

CCTK_RegisterIOMethodOutputGH (IOMethod,
OutputGH) CCTK_RegisterIOMethodOutputVarAs
(IOMethod, OutputVarAs) CCTK_RegisterIOMethodTime
ToOutput (IOMethod, TimeForOutput) CCTK_RegisterI
OMethodTriggerOutput (IOMethod, TriggerOutput)
6
Interaction with IO methods

• Flesh provides routines for application thorns
  to interact with IO methods

CCTK_OutputGH(GH)
Invoke all registered IO methodsand output all
variables on this GH
CCTK_OutputVar(GH, var)
CCTK_OutputVarAs(GH, var, alias)
Do output on variable var with all IO methods
CCTK_OutputVarByMethod(GH, method_name, var)
CCTK_OutputVarAsByMethod(GH, var, method_name,
alias)
Output variable var by calling specific IO method
method_name
7
IO Skeleton Thorn IOUtil

• Provides basic parameters common for all kinds
  of IO methods
• Sets some meaningful default values for getting
  output
• Parameters are used by other IO thorns as
  default values or overridden by their own
  private parameters for more specific control

IOUtil needs to be compiled into Cactusand
included into ActiveThornsif any output is
requested
8
IOUtil Parameters

• Implemention IO

IOltparametergt ltvaluegt

• General parameters

Output when INT out_every -1
Output where STRING outdir .
Output info BOOLEAN verbose no
9
IOUtil Parameters (continued)

• 3D output parameters

Output mode for parallel IO
KEYWORD out3D_mode proc np onefile
proc
INT out3D_procs 8
BOOLEAN out3D_unchunked no
BOOLEAN out3D_septimefiles no
Downsampling of 3D variables
INT out3D_downsample_xyz 1
Output floating point data
BOOLEAN out3D_single no
10
Thorn IOBasic
Provides 2 IO methods
SCALAR - outputs the value of scalar
variables as well as
the results of reduction operations
on grid functions
minimum
maximum
L1 and L2 norm INFO - prints
runtime information to screen
(minimum and maximum values
of grid functions)
11
IOBasic Parameters

• Implemention IOBasic

Output what STRING outScalar_vars,
outInfo_vars
Output when INT outScalar_every,
outInfo_every defaults to
IOout_every
Output where STRING outdirScalar
defaults to IOoutdir
Output style KEYWORD outScalar_style
xgraph gnuplot xgraph
12
Thorn IOASCII
Provides IO method

• IOASCII_1D - one-dimensional output
  of grid functions
  - slices in x,y,z, and diagonal
  directions depending
  on the symmetry

13
IOASCII Parameters

• Implemention IOASCII

Output what STRING out1D_vars
Output when INT out1D_every
defaults to IOout_every
Output where STRING outdir1D
defaults to IOoutdir
1D slices BOOLEAN out1D_xyzd yes
Output style KEYWORD out_style xgraph
gnuplot xgraph
14
Thorn IOFlexIO

• Provides 2 IO methods

IOFlexIO_2D - two-dim. slices of 3D grid
functions IOFlexIO_3D - three-dim. output of
variables

• based on John Shalfs FlexIO library
  (http//bach.ncsa.uiuc.edu/FlexIO) - simple
  uniform API for storing multidim. scientific
  data in different file formats

• creates datafiles in IEEEIO binary file format
  - complies to IEEE 754 - optimized for
  writing - platform-independent -
  crash-safe - readers available for Amira,
  AVS, IDL

15
IOFlexIO Parameters

• Implemention IOFlexIO

Output what STRING out2D_vars, out3D_vars
Output when INT out2D_every, out3D_every
defaults to IOout_every
Output where STRING outdir2D, outdir3D
defaults to IOoutdir
Optimized 3D output BOOLEAN out3D_reuse_filehan
dles yes
16
Visualizing IOFlexIO 3D outputwith Amira
17
Thorn IOHDF5

• Provides IO method IOHDF5_3D for
  three-dimensional output of variables

• based on HDF5 (http//hdf.ncsa.uiuc.edu/whatishdf
  5.html) - future standard for storing
  scientific data - overcomes limitations of
  HDF4 - simple but powerful data storage
  model - selection of hyperslabs -
  automatic datatype conversion - support for
  parallel IO gt Metacomputing ! - readers
  available for Amira, AVS

18
IOHDF5 Parameters

• Implemention IOHDF5

Output what STRING out3D_vars
Output when INT out3D_every
defaults to IOout_every
Output where STRING outdir3D
defaults to IOoutdir
19
Using HDF5 in Cactus

• HDF5 is not delivered with the Cactus
  distribution

• Building Cactus 4.0 with HDF5 make
  ltconfigurationgt-config HDF5yes
  HDF5_DIRltdirgt

• Using parallel IO extensions of HDF5
  IOout3D_mode onefile IOunchunked
  yes

High IO performance, no need for recombining !
20
Checkpointing Recoveryin Cactus 4.0
Checkpointing - dump the current state of
simulation to a
checkpoint file Recovery - set initial
state of new simulation
from a checkpoint file

• flesh provides scheduling points
• IOUtil declares common parameters for
  checkpointing
• IO thorns (IOFlexIO, IOHDF5) register
  checkpoint/recovery routines with the flesh
  and IOUtil

21
Checkpointing Recovery Parameters

• General checkpointing parameters from IOUtil

Checkpoint when INT checkpoint_every -1
BOOLEAN checkpoint_ID no
Checkpoint where STRING checkpoint_file
checkpoint
STRING checkpoint_ID_file
checkpoint_ID STRING
checkpoint_dir .
Checkpoint how INT checkpoint_keep 1
BOOLEAN checkpoint_keep_all no

• uses same output mode as for 3D data

22
Checkpointing Recovery Parameters (continued)

• Recovery parameters from IOUtil

Recover from BOOLEAN recover no
STRING recover_file checkpoint
STRING recovery_dir .

• Checkpointing parameters from other IO thorns

Enable checkpointing
BOOLEAN IOFlexIOcheckpoint no
BOOLEAN IOHDF5checkpoint no