Parallel Computing Using Message-Passing (Our Chapter 6)

1
Parallel Computing Using Message-Passing(Our
Chapter 6)

• REFERENCE Chapter 2 7 of Parallel Progamming
  by Barry Wilkinson and Michael Allen (WA for
  short).
• While oriented towards cluster computing,
  Chapters 3-7 provide a useful collection of
  different types of parallel problems.
• We will look at one or more problems in each
  chapter.
• Provides an good introduction to MIMD computing.
• Different models may have other solutions that
  are better.
• Provides some good problem choices for a testbed
  for proposed models and architectures.
• Other problem types need to be added to testbed.
• E.g., problems that are considered difficult to
  parallelize.
• The authors slides for these chapters will be
  posted in PDF at the course website.
• Currently available at last years course website
• Chapter 2 in WA on MPI and PVM
• MPI, a part of this chapter, was introduced
  earlier.
• The authors slides 76, 77, 79, 83, and 85-91
  will be covered. These cover parts of Sections
  2.2 and 2.3.
• Some of Chapter 2 will be handed out and are part
  of your reading assignment.

2

• Some use of the Big-O, ?, and ? notation occurs
  in Chapters 2-8 and is reviewed in Subsection
  2.3.2.
• These are standard complexity concepts in basic
  algorithms that you should understand.
• You should review any that you are unfamiliar
  with.
• Chapter 3 in Wilkinson /Allen (or WA) covers
  Embarassingly/Pleasingly Parallel Computations
• The main focus in this chapter is computing the
  Mandelbrot Set.
• Some reference material from chptr 3 will be
  distributed.
• Additional reference material on the Mandelbrot
  Sets will be provided from the textbook,
  Computer Graphics by F.S. Hill, 2nd Ed.
• The slides covered are 99-102, 105-113.
• Chapter 4 in WA Partitioning and
  Divide-and-Conquer Strategies.
• Our main focus is section 4.2.1, Sorting Using
  Bucket Sort
• The slides covered are 120, 121, 124-130,
  134-139, possibly 140-145.
• Some reference material from Chapter 4 will be
  provided.
• The textbook material should be used mainly to
  clarify material covered on slides.

3

• Chapter 5 in WA covers pipeline computations.
• Some reference material from Chapter 5 will be
  provided.
• The textbook material can be used mainly as a
  reference to clarify slides covered.
• The slides covered are 90-107. Some slides (e.g.,
  93-97) are only covered lightly.
• Chapter 6 in WA covers Synchronous Computations.
• The slides covered are 120 -132. Most slide are
  only covered lightly (due to time).
• The Heat Distribution Problem (slides 141-150)
  will be discussed briefly especially a short
  comparison of block and strip partitioning.
• Chapter 7 in WA covers Load Balancing and
  Termination Detection.
• Slides 160-163 were covered lightly.
• Slides 163-171, 175-7 may be covered if time
  permits
• The Shortest Path Problem (slides 182-196) is
  omitted due to time.
• May be added in future
• Note increased code is needed due to load
  balancing and termination.