Collective Communications

1
Collective Communications

• Self Test with solution

2
Self Test

• We want to do a simple broadcast of variable
  abc7 in processor 0 to the same location in all
  other processors of the communicator. What is the
  correct syntax of the call to do this broadcast?
• MPI_Bcast ( abc7, 1, MPI_REAL, 0, comm )
• MPI_Bcast ( abc, 7, MPI_REAL, 0, comm )
• MPI_Broadcast ( abc7, 1, MPI_REAL, 0, comm )

3
Answer

• Correct!
• Sorry, that's not correct.
• the first argument should be the location abc7.
  The second argument should be the number of
  elements to transfer. For this question, we
  transfer 1 data value.
• Sorry, that's not correct.
• the correct function name is MPI_Bcast().
  Otherwise, the arguments are correct.

4
Self Test

• Each processor has a local array a with 50
  elements. Each local array is a slice of a larger
  global array. We wish to compute the average
  (mean) of all elements in the global array. Our
  preferred approach is to add all of the data
  element-by-element onto the root processor, sum
  elements of the resulting array, divide, and
  broadcast the result to all processes. Which
  sequence of calls will accomplish this? Assume
  variables are typed and initialized appropriately.

5
Self Test

• start 0
• final 49
• count final - start 1
• mysum 0
• for ( istart i
• MPI_Reduce ( mysum, sum, 1, MPI_REAL,
• MPI_SUM, root, comm )
• total_countnprocs count
• if ( my_rankroot ) averagesum / total_count
• MPI_Bcast ( average, 1, MPI_REAL, root, comm )

6
Self Test

• start 0
• final 49
• count final - start 1
• MPI_Reduce ( a, sum_array, count,
• MPI_REAL, MPI_SUM, root, comm )
• sum 0
• for ( istart, isum_arrayi
  
• total_countnprocs count
• if ( my_rankroot ) averagesum / total_count
• MPI_Bcast ( average, 1, MPI_REAL, root, comm )

7
Self Test

• start 0
• final 49
• count final - start 1
• mysum 0
• for ( istart i
• my_averagemysum / count
• MPI_Reduce ( my_average, sum, 1, MPI_REAL,
• MPI_SUM, root, comm )
• if ( my_rankroot ) averagesum / nprocs
• MPI_Bcast ( average, 1, MPI_REAL, root, comm )

8
Answer

• Sorry, that's not correct. -- Well, this is not
  the way the instructions specified, but it should
  give the same answer. The instructions were to
  sum all of the data into the root process,
  compute the average, and then broadcast the
  result. This solution computes a partial sum for
  each process, sums these partial sums onto the
  root process, then averages and broadcasts.
• That's correct!
• Sorry, that's not correct. -- This is not what
  the instructions specified. It should produce an
  answer that is close to the correct one, but
  numerically these are different calculations The
  instructions were to sum all of the data into the
  root process, compute the average, and then
  broadcast the result. This solution computes an
  average on each process, sums these averages onto
  the root process, then averages these partial
  averages and broadcasts. This is not likely to be
  the best way to do this calculation.

9
Self Test

• Consider a communicator with 4 processes. How
  many total MPI_Send()'s and MPI_Recv()'s would be
  required to accomplish the following
  
• MPI_Allreduce ( a, x, 1, MPI_REAL, MPI_SUM,
  comm )
  
• 3
• 4
• 12
• 16

10
Answer

• Sorry, that's not correct.
• Sorry, that's not correct.
• That's correct!
• You need to do 3 sends of a from the non-root
  processes and 3 matching receives by the root
  process. You sum these 3 values with the
  corresponding data in the root process and store
  the result in variable x in the root process.
  Then the root process executes 3 sends to the
  non-root processes with 3 matching receives.
• Sorry, that's not correct.

11
Course Problem
12
Course Problem

• Description
• The new problem still implements a parallel
  search of an integer array. The program should
  find all occurrences of a certain integer which
  will be called the target. It should then
  calculate the average of the target value and its
  index. Both the target location and the average
  should be written to an output file. In addition,
  the program should read both the target value and
  all the array elements from an input file.

13
Course Problem

• Exercise
• Modify your code from Chapter 5, to change how
  the master first sends out the target and
  subarray data to the slaves.
  
• Use the MPI broadcast routines to give each slave
  the target.
  
• Use the MPI scatter routine to give all
  processors a section of the array b it will
  search.
  
• When you use the standard MPI scatter routine you
  will see that the global array b is now split up
  into four parts and the master process now has
  the first fourth of the array to search.
• So you should add a search loop (similar to the
  slaves') in the master section of code to search
  for the target and calculate the average and then
  write the result to the output file.
• This is actually an improvement in performance
  since all the processors perform part of the
  search in parallel.