MPI: Message Passing Interface

1
MPI Message Passing Interface

• Multiprocessing API for distributed memory
  systems.

2
What is Message Passing?

• Message passing is a computational model in which
  processes have only local memory (they lack a
  common shared memory), but are able to
  communicate with other processes by sending and
  receiving messages.

3
Where can we use Message Passing?
Message passing can be used where-ever it is
possible for processes to exchange messages

• Distributed memory systems
• Networks of Workstations
• Even, shared memory systems.

4
MPI The Message Passing Interface

• MPI is an API for parallel programming on
  distributed memory systems. Specifies a set of
  operations, but says nothing about the
  implementation.
• Popular because it many vendors support it,
  therefore is very portable.
• Most common implementation MPICH.

5
MPI
MPI provides specifications for

• Process creation
• Process termination
• Process ordering
• Interprocess Communication

6
MPI Operations Basic Set.
MPI can be thought of as a small specification,
because any complete implementation need only
provide the following operations

• MPI_INIT
• MPI_COMM_SIZE
• MPI_COMM_RANK
• MPI_SEND
• MPI_RECV
• MPI_FINALIZE

7
MPI Datatypes.
MPI specifies datatypes explicitly in the
message. Why? Because we may have a cluster of
heterogeneous machines. Word sizes and data
formats may differ.

• MPI_INT
• MPI_FLOAT
• MPI_BYTE
• MPI_CHAR
• etc

8
Example Programs.

• Simple Hello world in C.
• Hello World with process ids.
• A simple sum program.

9
MPI Operations Extended Set.

• MPI_BCAST
• MPI_REDUCE

10
Buffered and Non-Blocking Communication.

• MPI_Bsend
• MPI_Brecv
• MPI_Isend
• MPI_Irecv
• MPI_Ssend
• MPI_Srecv

11
Groups and Communicators.

• A group is a collection of processes. The
  processes within a group are ordered by their
  ranks.
• The notion of a group and the context are
  combined in a single object called communicator.

12
Topologies.

• Many problems conceptually fit best on
  particular topologies. Ex the 2D Poisson
  equation is best solved on a 2D array of
  processes.
• MPI provides a routines to create topologies,
  particularly Cartesian topologies
• MPI_CART_CREATE
• MPI_CART_COORDS

13
MPICH tools.

• MPICH provides, apart from an implementation of
  MPI, a set of tools for visualization and
  debugging.
• MPE - Multiprocessing environment set of tools
  which provide routines for Graphing,
  visualization, logging, etc.
• Upshot a viewer for MPE logs.

14
For more information

• Using MPI Portable Parallel Programming with the
  Message Passing Interface, Gropp, et. al.
• www.mcs.anl.gov/mpi

Exercises 1. Could look at the code for
calculating pi in the book and figure it out. 2.
Could write a simple sum program which reads
numbers from a file and adds them.