Figure 7.1 An MPI solution to the Count 3s problem.

1
Figure 7.1 An MPI solution to the Count 3s
problem.
2
Figure 7.1 An MPI solution to the Count 3s
problem. (cont.)
3
Code Spec 7.1 MPI_Init().
4
Code Spec 7.2 MPI_Finalize().
5
Code Spec 7.3 MPI_Comm_Size().
6
Code Spec 7.4 MPI_Comm_Rank().
7
Code Spec 7.5 MPI_Send().
8
Code Spec 7.6 MPI_Recv().
9
Code Spec 7.7 MPI_Reduce().
10
Code Spec 7.8 MPI_Scatter().
11
Code Spec 7.8 MPI_Scatter(). (cont.)
12
Figure 7.2 Replacement code (for lines 1648 of
Figure 7.1) to distribute data using a scatter
operation.
13
Code Spec 7.9 MPI_Gather().
14
Communicators and Groups

• Communicators - Define a set of processes that
  can communicate with each other.
• Group an ordered set of processes for defining
  collective communication operations.
• Each has a unique ID from 0-(P-1)
• A process can belong to multiple groups
• If communicating in a 2-D manner, a process can
  belong to a particular row group as well as
  belonging to a particular column group.

15
Point-to-Point Communication

• Done with send and receive.
• Uses tags to identify logically different
  messages.
• Many flavors
• Blocking waits for local completion
• Non blocking no waiting before starting next
  stmt
• Isend and Irecv
• use wait to make sure complete before using
  buffers
• Synchronous waits for global completion
• Buffered programmer specifies buffer area
• Useful for many big messages
• Ready no system buffering timing issues

16
Figure 7.3 Each message must be copied as it
moves across four address spaces, each
contributing to the overall latency.
17
Scan

• Process i has the result of the operator
  performed on data from processes 0..i.
• A prefix type of operation

18
Code Spec 7.10 MPI_Scan().
19
Code Spec 7.11 MPI_Bcast(). MPI routine to
broadcast data from one root process to all other
processes in the communicator.
20
Code Spec 7.12 MPI_Barrier().
21
Code Spec 7.13 MPI_Wtime().
22
Figure 7.4 Example of collective communication
within a group.
23
Code Spec 7.14 MPI_Comm_group().
24
Code Spec 7.15 MPI_Group_incl().
25
Code Spec 7.16 MPI_Comm_create().
26
Figure 7.5 A 2D relaxation replaceson each
iterationall interior values by the average of
their four nearest neighbors.
27
Figure 7.6 MPI code for the main loop of the 2D
SOR computation.
28
Figure 7.6 MPI code for the main loop of the 2D
SOR computation. (cont.)
29
Figure 7.6 MPI code for the main loop of the 2D
SOR computation. (cont.)
30
Figure 7.8 A 2D SOR MPI program using
non-blocking sends and receives.
31
Figure 7.8 A 2D SOR MPI program using
non-blocking sends and receives. (cont.)
32
Figure 7.8 A 2D SOR MPI program using
non-blocking sends and receives. (cont.)
33
Code Spec 7.17 MPI_Waitall().
34
Figure 7.9 Creating a derived data type.