Title: MPI Message Passing Interface Basics
1MPI(Message Passing Interface)Basics
2MPI(Message Passing Interface)
- Message passing library standard developed by
group of academics and industrial partners to
foster more widespread use and portability. - Defines routines, not implementation.
- Several free implementations exist.
3- MPI designed
- To address some problems with earlier
message-passing system such as PVM. - To provide powerful message-passing mechanism and
routines - over 126 routines - (although it is said that one can write
reasonable MPI programs with just 6 MPI
routines).
4Unsafe Message PassingExample
Intended behavior
Process 0
Process 1
Destination
send(,1,)
send(,1,)
lib()
Source
recv(,0,)
lib()
recv(,0,)
5Unsafe Message PassingExample
Possible behavior
Process 0
Process 1
Destination
send(,1,)
send(,1,)
lib()
Source
recv(,0,)
lib()
recv(,0,)
6- Message tags not sufficient to deal with such
situations -- library routines might use same
tags. - MPI introduces concept of a communicator -
which defines the scope of communication.
7MPI Communicators
- Defines a communication domain - a set of
processes that are allowed to communicate between
themselves. - Communication domains of libraries can be
separated from that of a user program. - Used in all point-to-point and collective MPI
message-passing communications.
8Default Communicator MPI_COMM_WORLD
- Exists as first communicator for all processes
existing in the application. - A set of MPI routines exists for forming
communicators. - Processes have a rank in a communicator.
9MPIProcess Creation and Execution
- Purposely not defined - Will depend upon
implementation. - Only static process creation supported in MPI
version 1. All processes must be defined prior to
execution and started together. - Originally SPMD model of computation.
10SPMD Computational Model
- main (int argc, char argv)
- MPI_Init(argc, argv)
- .
- .
- MPI_Comm_rank(MPI_COMM_WORLD, myrank) /get
rank/ - if (myrank 0)
- master() / routine for master to execute /
- else
- slave() / routine for slaves to execute /
- .
- .
- MPI_Finalize()
11MPI Point-to-Point Communication
- Uses send and receive routines with message tags
(and communicator) - Wild card message tags available
12MPI Blocking Routines
- Return when locally complete
- when location used to hold message can be used
again or altered without affecting message being
sent. - Blocking send will send message and return
- does not mean that message has been received,
just that process free to move on without
adversely affecting message.
13Parameters of blocking send
14Parameters of blocking receive
15Example
- To send an integer x from process 0 to process 1,
- int x
- MPI_Comm_rank(MPI_COMM_WORLD,myrank) / find
rank / - if (myrank 0)
- MPI_Send(x, 1, MPI_INT, 1, msgtag,
MPI_COMM_WORLD) - else if (myrank 1)
- MPI_Recv(x,1,MPI_INT,0,msgtag,MPI_COMM_WORLD,sta
tus)
16MPI Nonblocking Routines
- Nonblocking send - MPI_Isend()
- will return immediately even before source
location is safe to be altered. - Nonblocking receive - MPI_Irecv()
- will return even if no message to accept.
17Nonblocking Routine Formats
- MPI_Isend(buf,count,datatype,dest,tag,comm,request
) - MPI_Irecv(buf,count,datatype,source,tag,comm,
request) - Completion detected by MPI_Wait() and MPI_Test().
- MPI_Wait() waits until operation completed,
returns then. - MPI_Test() returns with flag set indicating
whether operation completed at that time. - Need to know whether particular operation
completed. - Determined by accessing request parameter.
18Example
- To send an integer x from process 0 to process 1
and allow process 0 to continue, - MPI_Comm_rank(MPI_COMM_WORLD, myrank)/ find
rank / - if (myrank 0)
- MPI_Isend(x,1,MPI_INT,1,msgtag,MPI_COMM_WORLD,
req1) - compute()
- MPI_Wait(req1, status)
- else if (myrank 1)
- MPI_Recv(x,1,MPI_INT,0,msgtag,MPI_COMM_WORLD,sta
tus)
19Send Communication Modes
- Standard Mode Send - Not assumed that
corresponding receive routine has started. Amount
of buffering not defined by MPI. If buffering
provided, send could complete before receive
reached. - Buffered Mode - Send may start and return before
a matching receive. Necessary to specify buffer
space via routine MPI_Buffer_attach(). - Synchronous Mode - Send and receive can start
before each other but can only complete together. - Ready Mode - Send can only start if matching
receive already reached, otherwise error. Use
with care.
20- Each of the four modes can be applied to both
blocking and nonblocking send routines. - Only the standard mode is available for the
blocking and nonblocking receive routines. - Any type of send routine can be used with any
type of receive routine.
21Collective Communication
- Involves set of processes, defined by an
intra-communicator. Message tags not present.
Principal collective operations - MPI_Bcast() - Broadcast from root to all other
processes - MPI_Gather() - Gather values for group of
processes - MPI_Scatter() - Scatters buffer in parts to group
of processes - MPI_Alltoall() - Sends data from all processes to
all processes - MPI_Reduce() - Combine values on all processes to
single value - MPI_Reduce_scatter() - Combine values and scatter
results - MPI_Scan() - Compute prefix reductions of data on
processes
22Example
- To gather items from group of processes into
process 0, using dynamically allocated memory in
root process - int data10 /data to be gathered/
- MPI_Comm_rank(MPI_COMM_WORLD, myrank) / find
rank / - if (myrank 0)
- MPI_Comm_size(MPI_COMM_WORLD, grp_size) /find
group size/ - buf (int )malloc(grp_size10sizeof(int))
/allocate memory/ -
- MPI_Gather(data,10,MPI_INT,buf,grp_size10,MPI_INT
,0,MPI_COMM_WORLD) - MPI_Gather() gathers from all processes,
including root.
23Barrier
- As in all message-passing systems, MPI provides a
means of synchronizing processes by stopping each
one until they all have reached a specific
barrier routine.
24Barrier Concept
25Using Library Routines
26(No Transcript)
27Measuring Execution Time
- To measure execution time between point L1 and
point L2 in the code, might have - .
- L1 time(t1) / start timer /
- .
- .
- L2 time(t2) / stop timer /
- .
- elapsed_t difftime(t2, t1) /t2 - t1/
- printf(Elapsed time 5.2f,elapsed_t)
- MPI provides MPI_Wtime() for returning time (in
seconds).
28Executing MPI programs
- MPI version 1 standard does not address
implementation and did not specify how programs
are to be started and each implementation has its
own way.
29- Several MPI implementations, such as MPICH and
LAM MPI, use command - mpirun -np prog
-
- where is the number of processes and prog is
the program. - Additional argument specify computers (see
later).
30MPI-2
- The MPI standard, version 2 does recommend a
command for starting MPI programs, namely - mpiexec -n prog
- where is the number of processes and prog is
the program.
31Sample MPI Programs
32Hello World
- include "mpi.h"
- include ltstdio.hgt
- int main(int argc,char argv)
- MPI_Init(argc, argv)
- printf("Hello World\n")
- MPI_Finalize()
- return 0
33Hello WorldPrinting out rank of process
- include "mpi.h"
- include ltstdio.hgt
- int main(int argc,char argv)
- int myrank, numprocs
- MPI_Init(argc, argv)
- MPI_Comm_rank(MPI_COMM_WORLD,myrank)
- MPI_Comm_size(MPI_COMM_WORLD,numprocs)
- printf("Hello World from process d of d\n",
myrank, numprocs) - MPI_Finalize()
- return 0
34Question
- Suppose this program is compiled as helloworld
and is executed on a single computer with the
command - mpirun -np 4 helloworld
- What would the output be?
35Answer
- Several possible outputs depending upon order
processes are executed. - Example
- Hello World from process 2 of 4
- Hello World from process 0 of 4
- Hello World form process 1 of 4
- Hello World form process 3 of 4
36- Adding communication to get process 0 to print
all messages - include "mpi.h"
- include ltstdio.hgt
- int main(int argc,char argv)
- int myrank, numprocs
- char greeting80 / message sent from
slaves to master / - MPI_Status status
- MPI_Init(argc, argv)
- MPI_Comm_rank(MPI_COMM_WORLD,myrank)
- MPI_Comm_size(MPI_COMM_WORLD,numprocs)
- sprintf(greeting,"Hello World from process d of
d\n",rank,size) - if (myrank 0 ) / I am going print
out everything / - printf("s\n",greeting) / print
greeting from proc 0 / - for (i 1 i lt numprocs i) /
greetings in order / - MPI_Recv(geeting,sizeof(greeting),MPI_CHAR,i,1,M
PI_COMM_WORLD, - status)
- printf(s\n", greeting)
-
- else
37MPI_Get_processor_name()
- Return name of processor executing code (and
length of string). Arguments - MPI_Get_processor_name(char name,int resultlen)
- Example
- int namelen
- char procnameMPI_MAX_PROCESSOR_NAME
- MPI_Get_processor_name(procname,namelen)
returned in here
38- Easy then to add name in greeting with
- sprintf(greeting,"Hello World from process d of
d on s\n", rank, size, procname)
39Pinging processes and timingMaster-slave
structure
- include ltmpi.hgt
- void master(void)
- void slave(void)
- int main(int argc, char argv)
- int myrank
- printf("This is my ping program\n")
- MPI_Init(argc, argv)
- MPI_Comm_rank(MPI_COMM_WORLD, myrank)
- if (myrank 0)
- master()
- else
- slave()
-
- MPI_Finalize()
- return 0
40Master routine
- void master(void)
- int x 9
- double starttime, endtime
- MPI_Status status
- printf("I am the master - Send me a message when
you receive this number d\n", x) - starttime MPI_Wtime()
- MPI_Send(x,1,MPI_INT,1,1,MPI_COMM_WORLD)
- MPI_Recv(x,1,MPI_INT,1,1,MPI_COMM_WORLD,status)
- endtime MPI_Wtime()
- printf("I am the master. I got this back d \n",
x) - printf("That took f seconds\n",endtime -
starttime)
41Slave routine
- void slave(void)
- int x
- MPI_Status status
- printf("I am the slave - working\n")
- MPI_Recv(x,1,MPI_INT,0,1,MPI_COMM_WORLD,status)
- printf("I am the slave. I got this d \n", x)
- MPI_Send(x, 1, MPI_INT, 0, 1, MPI_COMM_WORLD)
42Example using collective routinesMPI_Bcast()MPI
_Reduce()
- Adding numbers in a file.
43- include mpi.h
- include ltstdio.hgt
- include ltmath.hgt
- define MAXSIZE 1000
- void main(int argc, char argv)
- int myid, numprocs
- int dataMAXSIZE, i, x, low, high, myresult,
result - char fn255
- char fp
- MPI_Init(argc,argv)
- MPI_Comm_size(MPI_COMM_WORLD,numprocs)
- MPI_Comm_rank(MPI_COMM_WORLD,myid)
- if (myid 0) / Open input file and
initialize data / - strcpy(fn,getenv(HOME))
- strcat(fn,/MPI/rand_data.txt)
- if ((fp fopen(fn,r)) NULL)
- printf(Cant open the input file s\n\n,
fn) - exit(1)
-
44Compiling/Executing MPI ProgramsPreliminaries
- Set up paths
- Create required directory structure
- Create a file listing machines to be used
(hostfile)
45- Before starting MPI for the first time, need to
create a hostfile - Sample hostfile
- terra
- venus //Currently not used, commented out
- leo1
- leo2
- leo3
- leo4
- leo5
- leo6
- leo7
- leo8
46Compiling/executing (SPMD) MPI program
- For LAM MPI version 6.5.2. At a command line
- To start MPI
- First time lamboot -v hostfile
- Subsequently lamboot
- To compile MPI programs
- mpicc -o file file.c
- or mpiCC -o file file.cpp
- To execute MPI program
- mpirun -v -np no_processors file
- To remove processes for reboot
- lamclean -v
- Terminate LAM
- lamhalt
- If fails wipe -v lamhost
47Compiling/Executing Multiple MPI Programs
- Create a file, say called appfile, specifying
programs - Example
- 1 master and 2 slaves, appfile contains
- n0 master
- n0-1 slave
- To execute
- mpirun -v appfile
- Sample output
- 3292 master running on n0 (o)
- 3296 slave running on n0 (o)
- 412 slave running on n1
48Parallel Programming Home Page
- http//www.cs.uncc.edu/par_prog
- Gives step-by-step instructions for compiling and
executing programs, and other information.