CS 584 Lecture 5

1
CS 584 Lecture 5

• Assignment. Due NOW!!

2
Review

• Partition
• Communication
• Avoid centralized patterns
• Avoid sequential patterns
• Divide and Conquer

3
Agglomeration

• Partition and Communication steps were abstract
• Agglomeration moves to concrete.
• Combine tasks to execute efficiently on some
  parallel computer.
• Consider replication.

4
Agglomeration Goals

• Reduce communication costs by
• increasing computation
• decreasing/increasing granularity
• Retain flexibility for mapping and scaling.
• Reduce software engineering costs.

5
Changing Granularity

• A large number of tasks does not necessarily
  produce an efficient algorithm.
• We must consider the communication costs.
• Reduce communication by
• having fewer tasks
• sending less messages (batching)

6
Surface to Volume Effects

• Communication is proportional to the surface of
  the subdomain.
• Computation is proportional to the volume of the
  subdomain.
• Increasing computation will often decrease
  communication.

7
How many messages total? How much data is sent?
8
How many messages total? How much data is sent?
9
Replicating Computation

• Trade-off replicated computation for reduce
  communication.
• Replication will often reduce execution time as
  well.

10
Summation of N Integers
s sum b broadcast
How many steps?
11
Using Replication (Butterfly)
12
Using Replication
Butterfly to Hypercube
13
Avoid Communication

• Look for tasks that cannot execute concurrently
  because of communication requirements.
• Replication can help accomplish two tasks at the
  same time, like
• Summation
• Broadcast

14
Preserve Flexibility

• Create more tasks than processors.
• Overlap communication and computation.
• Don't incorporate unnecessary limits on the
  number of tasks.

15
Agglomeration Checklist

• Reduce communication costs by increasing
  locality.
• Do benefits of replication outweigh costs?
• Does replication compromise scalability?
• Does the number of tasks still scale with problem
  size?
• Is there still sufficient concurrency?

16
Assignment

• Exercises 2.1, 2.2, 2.4, and 2.12
• When asked to design an algorithm, go through the
  PCAM design stages.