MPI Message Passing Interface Basics - PowerPoint PPT Presentation

1 / 48
About This Presentation
Title:

MPI Message Passing Interface Basics

Description:

MPI Message Passing Interface Basics – PowerPoint PPT presentation

Number of Views:181
Avg rating:3.0/5.0
Slides: 49
Provided by: barry199
Category:

less

Transcript and Presenter's Notes

Title: MPI Message Passing Interface Basics


1
MPI(Message Passing Interface)Basics
2
MPI(Message Passing Interface)
  • Message passing library standard developed by
    group of academics and industrial partners to
    foster more widespread use and portability.
  • Defines routines, not implementation.
  • Several free implementations exist.

3
  • MPI designed
  • To address some problems with earlier
    message-passing system such as PVM.
  • To provide powerful message-passing mechanism and
    routines - over 126 routines
  • (although it is said that one can write
    reasonable MPI programs with just 6 MPI
    routines).

4
Unsafe Message PassingExample
Intended behavior
Process 0
Process 1
Destination
send(,1,)
send(,1,)
lib()
Source
recv(,0,)
lib()
recv(,0,)
5
Unsafe Message PassingExample
Possible behavior
Process 0
Process 1
Destination
send(,1,)
send(,1,)
lib()
Source
recv(,0,)
lib()
recv(,0,)
6
  • Message tags not sufficient to deal with such
    situations -- library routines might use same
    tags.
  • MPI introduces concept of a communicator -
    which defines the scope of communication.

7
MPI Communicators
  • Defines a communication domain - a set of
    processes that are allowed to communicate between
    themselves.
  • Communication domains of libraries can be
    separated from that of a user program.
  • Used in all point-to-point and collective MPI
    message-passing communications.

8
Default Communicator MPI_COMM_WORLD
  • Exists as first communicator for all processes
    existing in the application.
  • A set of MPI routines exists for forming
    communicators.
  • Processes have a rank in a communicator.

9
MPIProcess Creation and Execution
  • Purposely not defined - Will depend upon
    implementation.
  • Only static process creation supported in MPI
    version 1. All processes must be defined prior to
    execution and started together.
  • Originally SPMD model of computation.

10
SPMD Computational Model
  • main (int argc, char argv)
  • MPI_Init(argc, argv)
  • .
  • .
  • MPI_Comm_rank(MPI_COMM_WORLD, myrank) /get
    rank/
  • if (myrank 0)
  • master() / routine for master to execute /
  • else
  • slave() / routine for slaves to execute /
  • .
  • .
  • MPI_Finalize()

11
MPI Point-to-Point Communication
  • Uses send and receive routines with message tags
    (and communicator)
  • Wild card message tags available

12
MPI Blocking Routines
  • Return when locally complete
  • when location used to hold message can be used
    again or altered without affecting message being
    sent.
  • Blocking send will send message and return
  • does not mean that message has been received,
    just that process free to move on without
    adversely affecting message.

13
Parameters of blocking send
14
Parameters of blocking receive
15
Example
  • To send an integer x from process 0 to process 1,
  • int x
  • MPI_Comm_rank(MPI_COMM_WORLD,myrank) / find
    rank /
  • if (myrank 0)
  • MPI_Send(x, 1, MPI_INT, 1, msgtag,
    MPI_COMM_WORLD)
  • else if (myrank 1)
  • MPI_Recv(x,1,MPI_INT,0,msgtag,MPI_COMM_WORLD,sta
    tus)

16
MPI Nonblocking Routines
  • Nonblocking send - MPI_Isend()
  • will return immediately even before source
    location is safe to be altered.
  • Nonblocking receive - MPI_Irecv()
  • will return even if no message to accept.

17
Nonblocking Routine Formats
  • MPI_Isend(buf,count,datatype,dest,tag,comm,request
    )
  • MPI_Irecv(buf,count,datatype,source,tag,comm,
    request)
  • Completion detected by MPI_Wait() and MPI_Test().
  • MPI_Wait() waits until operation completed,
    returns then.
  • MPI_Test() returns with flag set indicating
    whether operation completed at that time.
  • Need to know whether particular operation
    completed.
  • Determined by accessing request parameter.

18
Example
  • To send an integer x from process 0 to process 1
    and allow process 0 to continue,
  • MPI_Comm_rank(MPI_COMM_WORLD, myrank)/ find
    rank /
  • if (myrank 0)
  • MPI_Isend(x,1,MPI_INT,1,msgtag,MPI_COMM_WORLD,
    req1)
  • compute()
  • MPI_Wait(req1, status)
  • else if (myrank 1)
  • MPI_Recv(x,1,MPI_INT,0,msgtag,MPI_COMM_WORLD,sta
    tus)

19
Send Communication Modes
  • Standard Mode Send - Not assumed that
    corresponding receive routine has started. Amount
    of buffering not defined by MPI. If buffering
    provided, send could complete before receive
    reached.
  • Buffered Mode - Send may start and return before
    a matching receive. Necessary to specify buffer
    space via routine MPI_Buffer_attach().
  • Synchronous Mode - Send and receive can start
    before each other but can only complete together.
  • Ready Mode - Send can only start if matching
    receive already reached, otherwise error. Use
    with care.

20
  • Each of the four modes can be applied to both
    blocking and nonblocking send routines.
  • Only the standard mode is available for the
    blocking and nonblocking receive routines.
  • Any type of send routine can be used with any
    type of receive routine.

21
Collective Communication
  • Involves set of processes, defined by an
    intra-communicator. Message tags not present.
    Principal collective operations
  • MPI_Bcast() - Broadcast from root to all other
    processes
  • MPI_Gather() - Gather values for group of
    processes
  • MPI_Scatter() - Scatters buffer in parts to group
    of processes
  • MPI_Alltoall() - Sends data from all processes to
    all processes
  • MPI_Reduce() - Combine values on all processes to
    single value
  • MPI_Reduce_scatter() - Combine values and scatter
    results
  • MPI_Scan() - Compute prefix reductions of data on
    processes

22
Example
  • To gather items from group of processes into
    process 0, using dynamically allocated memory in
    root process
  • int data10 /data to be gathered/
  • MPI_Comm_rank(MPI_COMM_WORLD, myrank) / find
    rank /
  • if (myrank 0)
  • MPI_Comm_size(MPI_COMM_WORLD, grp_size) /find
    group size/
  • buf (int )malloc(grp_size10sizeof(int))
    /allocate memory/
  • MPI_Gather(data,10,MPI_INT,buf,grp_size10,MPI_INT
    ,0,MPI_COMM_WORLD)
  • MPI_Gather() gathers from all processes,
    including root.

23
Barrier
  • As in all message-passing systems, MPI provides a
    means of synchronizing processes by stopping each
    one until they all have reached a specific
    barrier routine.

24
Barrier Concept
25
Using Library Routines
26
(No Transcript)
27
Measuring Execution Time
  • To measure execution time between point L1 and
    point L2 in the code, might have
  • .
  • L1 time(t1) / start timer /
  • .
  • .
  • L2 time(t2) / stop timer /
  • .
  • elapsed_t difftime(t2, t1) /t2 - t1/
  • printf(Elapsed time 5.2f,elapsed_t)
  • MPI provides MPI_Wtime() for returning time (in
    seconds).

28
Executing MPI programs
  • MPI version 1 standard does not address
    implementation and did not specify how programs
    are to be started and each implementation has its
    own way.

29
  • Several MPI implementations, such as MPICH and
    LAM MPI, use command
  • mpirun -np prog
  • where is the number of processes and prog is
    the program.
  • Additional argument specify computers (see
    later).

30
MPI-2
  • The MPI standard, version 2 does recommend a
    command for starting MPI programs, namely
  • mpiexec -n prog
  • where is the number of processes and prog is
    the program.

31
Sample MPI Programs
32
Hello World
  • include "mpi.h"
  • include ltstdio.hgt
  • int main(int argc,char argv)
  • MPI_Init(argc, argv)
  • printf("Hello World\n")
  • MPI_Finalize()
  • return 0

33
Hello WorldPrinting out rank of process
  • include "mpi.h"
  • include ltstdio.hgt
  • int main(int argc,char argv)
  • int myrank, numprocs
  • MPI_Init(argc, argv)
  • MPI_Comm_rank(MPI_COMM_WORLD,myrank)
  • MPI_Comm_size(MPI_COMM_WORLD,numprocs)
  • printf("Hello World from process d of d\n",
    myrank, numprocs)
  • MPI_Finalize()
  • return 0

34
Question
  • Suppose this program is compiled as helloworld
    and is executed on a single computer with the
    command
  • mpirun -np 4 helloworld
  • What would the output be?

35
Answer
  • Several possible outputs depending upon order
    processes are executed.
  • Example
  • Hello World from process 2 of 4
  • Hello World from process 0 of 4
  • Hello World form process 1 of 4
  • Hello World form process 3 of 4

36
  • Adding communication to get process 0 to print
    all messages
  • include "mpi.h"
  • include ltstdio.hgt
  • int main(int argc,char argv)
  • int myrank, numprocs
  • char greeting80 / message sent from
    slaves to master /
  • MPI_Status status
  • MPI_Init(argc, argv)
  • MPI_Comm_rank(MPI_COMM_WORLD,myrank)
  • MPI_Comm_size(MPI_COMM_WORLD,numprocs)
  • sprintf(greeting,"Hello World from process d of
    d\n",rank,size)
  • if (myrank 0 ) / I am going print
    out everything /
  • printf("s\n",greeting) / print
    greeting from proc 0 /
  • for (i 1 i lt numprocs i) /
    greetings in order /
  • MPI_Recv(geeting,sizeof(greeting),MPI_CHAR,i,1,M
    PI_COMM_WORLD,
  • status)
  • printf(s\n", greeting)
  • else

37
MPI_Get_processor_name()
  • Return name of processor executing code (and
    length of string). Arguments
  • MPI_Get_processor_name(char name,int resultlen)
  • Example
  • int namelen
  • char procnameMPI_MAX_PROCESSOR_NAME
  • MPI_Get_processor_name(procname,namelen)

returned in here
38
  • Easy then to add name in greeting with
  • sprintf(greeting,"Hello World from process d of
    d on s\n", rank, size, procname)

39
Pinging processes and timingMaster-slave
structure
  • include ltmpi.hgt
  • void master(void)
  • void slave(void)
  • int main(int argc, char argv)
  • int myrank
  • printf("This is my ping program\n")
  • MPI_Init(argc, argv)
  • MPI_Comm_rank(MPI_COMM_WORLD, myrank)
  • if (myrank 0)
  • master()
  • else
  • slave()
  • MPI_Finalize()
  • return 0

40
Master routine
  • void master(void)
  • int x 9
  • double starttime, endtime
  • MPI_Status status
  • printf("I am the master - Send me a message when
    you receive this number d\n", x)
  • starttime MPI_Wtime()
  • MPI_Send(x,1,MPI_INT,1,1,MPI_COMM_WORLD)
  • MPI_Recv(x,1,MPI_INT,1,1,MPI_COMM_WORLD,status)
  • endtime MPI_Wtime()
  • printf("I am the master. I got this back d \n",
    x)
  • printf("That took f seconds\n",endtime -
    starttime)

41
Slave routine
  • void slave(void)
  • int x
  • MPI_Status status
  • printf("I am the slave - working\n")
  • MPI_Recv(x,1,MPI_INT,0,1,MPI_COMM_WORLD,status)
  • printf("I am the slave. I got this d \n", x)
  • MPI_Send(x, 1, MPI_INT, 0, 1, MPI_COMM_WORLD)

42
Example using collective routinesMPI_Bcast()MPI
_Reduce()
  • Adding numbers in a file.

43
  • include mpi.h
  • include ltstdio.hgt
  • include ltmath.hgt
  • define MAXSIZE 1000
  • void main(int argc, char argv)
  • int myid, numprocs
  • int dataMAXSIZE, i, x, low, high, myresult,
    result
  • char fn255
  • char fp
  • MPI_Init(argc,argv)
  • MPI_Comm_size(MPI_COMM_WORLD,numprocs)
  • MPI_Comm_rank(MPI_COMM_WORLD,myid)
  • if (myid 0) / Open input file and
    initialize data /
  • strcpy(fn,getenv(HOME))
  • strcat(fn,/MPI/rand_data.txt)
  • if ((fp fopen(fn,r)) NULL)
  • printf(Cant open the input file s\n\n,
    fn)
  • exit(1)

44
Compiling/Executing MPI ProgramsPreliminaries
  • Set up paths
  • Create required directory structure
  • Create a file listing machines to be used
    (hostfile)

45
  • Before starting MPI for the first time, need to
    create a hostfile
  • Sample hostfile
  • terra
  • venus //Currently not used, commented out
  • leo1
  • leo2
  • leo3
  • leo4
  • leo5
  • leo6
  • leo7
  • leo8

46
Compiling/executing (SPMD) MPI program
  • For LAM MPI version 6.5.2. At a command line
  • To start MPI
  • First time lamboot -v hostfile
  • Subsequently lamboot
  • To compile MPI programs
  • mpicc -o file file.c
  • or mpiCC -o file file.cpp
  • To execute MPI program
  • mpirun -v -np no_processors file
  • To remove processes for reboot
  • lamclean -v
  • Terminate LAM
  • lamhalt
  • If fails wipe -v lamhost

47
Compiling/Executing Multiple MPI Programs
  • Create a file, say called appfile, specifying
    programs
  • Example
  • 1 master and 2 slaves, appfile contains
  • n0 master
  • n0-1 slave
  • To execute
  • mpirun -v appfile
  • Sample output
  • 3292 master running on n0 (o)
  • 3296 slave running on n0 (o)
  • 412 slave running on n1

48
Parallel Programming Home Page
  • http//www.cs.uncc.edu/par_prog
  • Gives step-by-step instructions for compiling and
    executing programs, and other information.
Write a Comment
User Comments (0)
About PowerShow.com