Introduction to Charm Machine Layer

1
Introduction toCharm Machine Layer

• Gengbin Zheng
• Parallel Programming Lab
• 4/3/2002

2
Machine Layer Overview

• Startup and initialization
• Low level communication
• CmiFreeSendFn, etc
• CmiSyncBroadcastAllFn(), etc
• SMP support
• Multiple worker threads and one comm thread
• Thread creation and state initialization

3
Supported Platforms

• Charm version OS Communication
  Compiler
• ------------------- ---------
  ------------------- --------------------
• net-linux PC Linux
  UDP/Myrinet GNU compiler
• net-linux-ia64 IA64 Linux
  UDP/Myrinet GNU compiler
• net-axp Alpha
  UDP GNU compiler
• net-cygwin Win32/cygwin UDP
  GNU compiler
• net-win32 Win32 UDP
  MS Visual C
• net-sol Solaris
  UDP GNU compiler
• net-irix IRIX
  UDP GNU compiler
• net-hp HP-UX
  UDP GNU compiler
• net-ppc-darwin MacOS X UDP
  GNU C compiler
• mpi-linux PC Linux
  MPI GNU compiler
• mpi-linux-ia64 IA64 Linux MPI
  GNU compiler
• mpi-axp Alpha
  MPI GNU compiler
• mpi-linux-axp Alpha Linux MPI
  GNU compiler
• mpi-origin Origin2000 MPI
  C compiler
• mpi-sp AIX
  MPI AIX xlC Compiler
• origin2000 Origin2000 shared-mem
  SGI C compiler
• t3e Cray T3E
  shared-mem Cray C compiler

4
Communication Architecture
5
Machine Layer Files Layout

• Configuration files for each main platform
• conv-mach.h
• conv-common.h
• conv-mach-opt.h from
• smp, scyld, bluegene, gm, vmi, tcp
• cc, cxx, kcc, cc64, acc, ecc
• Implementations
• common.net, common.mpi, common.shmem
• machine.c

6
How build works

• build charm net-linux gm smp kcc bluegene
• Sort gm, smp and bluegene
• Mkdir net-linux-bluegene-gm-smp-kcc
• Cat conv-mach-kccbluegenegmsmp.h to
  conv-mach-opt.h
• Cat conv-mach-kccbluegenegmsmp.sh to
  conv-mach-opt.sh
• Gather files from common.net, etc (Makefile)
• Make charm under
• net-linux-bluegene-gm-smp-kcc/tmp

7
Parallel Program Startup

• Net version - nodelist

Rsh/ssh (IP, port)
Rsh/ssh (IP, port)
Charmrun node
compute node
compute node
my node (IP, port)
Broadcast all nodes (IP, port)
8
Converse life

• ConverseInit
• Global variables initialization
• Start worker threads
• ConverseRunPE for each Charm PE
• Per thread initialization
• Loop into scheduler CsdScheduler()

9
Message formats

• Net version
• define CMK_MSG_HEADER_BASIC CmiUInt2
  d0,d1,d2,d3,d4,d5,hdl,d7

• MPI version
• define CMK_MSG_HEADER_BASIC CmiUInt2 rank,
  root, hdl,xhdl,info,d3

info
handler
xhandler
rank
root
d3
10
SMP support

• MPI-smp as an example
• Create threads CmiStartThreads
• Worker threads work cycle
• See code in mahcine-smp.c
• Communication thread work cycle
• See code in machine-smp.c