Introduction to parallel programming

1
Introduction to parallel programming
2
Welcome to parallel programming!

• Under PVM - Parallel Virtual Machine - you will
  run your tasks several times faster on
  environment, working together as parallel
  cluster, than on a PC.

3
Choice of decomposition method

• Data decomposition
• Algorithm decomposition
• Algorithm and data decomposition

4
Decomposition
5
Flowchart of serial C program

• Includes
• Functions

Main(int argc,char argv)

• Matrix's initialization
• Calculations

• Results
• Exit

6
Making parallel PVM program from serial C program
Master
Slave

• Includes
• Functions

• Includes
• Functions

Main(int argc,char argv)
Main(int argc,char argv)

• Matrix's initialization
• Start PVM

Receiving matrixs from master

• Spawn slaves
• Sending matrixs to slaves

Calculations

• Receiving results from slaves
• Exit from PVM

• Sending results to master
• Exit from PVM

7
Run programs under PVM and XPVM
8
EXAMPLE - DATA DECOMPOSITION

• ? (aB) 2 bB 2

B
B
MASTER 4 SLAVES
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
EXAMPLE - ALGORITHM DECOMPOSITION

• (A B) 2 ? A - B

(A B) 2
B
B
?(A,B)
? A - B
13
EXAMPLE - COMPLEX DECOMPOSITION (matrixs
multiplication)
B
B
Matrix-matrix multiplication algorithm based on
two-dimensional decompositions. Each step
involves three stages (a) an A submatrix is
broadcast to other tasks in the same row (b)
local computation is performed (c) the B
submatrix is rotated upwards within each column.
14
RESULTING GRAF
15
MAIN PVM ROUTINES
16
pvm_mytid
int pvm_mytid() pvm_mytid() returns the tid of
the calling task. Each PVM task has a unique
(tid) Task ID which is assigned to it when it is
created.
pvm_parent
int pvm_parent() pvm_parent() returns the tid
of the calling tasks parent, the task which
spawned it. In the case where the task wasnt
spawned PvmNoParent (-1) is returned.
17
pvm_spawn()

• int pvm_spawn(char task, char argv, int flag,
  char where,
• int ntask, int tids)
• Spawns a task which is a executable with optional
  command
• line arguments. Can spawn several copies on
  several machines
• depending on the flag argument. The tids of
  spawned tasks are
• returned in tids and the number of spawned tasks
  is returned
• by the function. The combination flag and where
  define where
• the tasks will be spawned
• PvmTaskDefault - spawn on any machine
• PvmTaskHost - spawn on the machine named in where
• PvmTaskArch - spawn on the architecture named in
  where
• PvmTaskDebug - spawn the task in a debugger
• PvmHostCompl - spawn on any machine except where

18
Receiving a message

• int pvm_recv(int source, int tag)
• -1 is a wildcard value that receives from any
  source or tag. Returns the id of the message
  buffer used.
• int pvm_upk()
• int pvm_upkstr(char str) - unpack a string from
  the message buffer.
• int pvm_upkint(char ip, int size, int stride) -
  unpack size integers, insert them in every other
  stride place in ip.
• int pvm_upkfloat(char ip, int size, int stride)
  - unpack size floats, insert them in every other
  stride place in ip.
• int pvm_bufinfo(int buf, int bytes, int tag,
  int tid) - using buf the buffer id received from
  pvm_recv(), get the size bytes, tag and tid of
  the received message.

19
Sending a massage

• int pvm_initsend(int encoding) - init a buffer
  for sending a message. Encoding defines how the
  data is packed, use PvmDataDefault for XDR
  encoding.
• pvm_pk()
• int pvm_pkstr(char str) - pack a string into the
  buffer.
• int pvm_pkint(int ip, int size, int stride) -
  pack an array of size integers, pack every other
  stride integer.
• int pvm_pkfloat(char ip, int size, int stride) -
  unpack size floats, insert them in every other
  stride place in ip.
• int pvm_send(int target, int tag) - send the
  buffer to tid target. Identify the message by tag.

20
pvm_exit
int pvm_exit() pvm_exit() tells, that the task
is leaving PVM.