Partitioning and Divide-and-Conquer Strategies

1
Lecture 5
Chapter4 Partitioning and Divide-and-Conquer
Strategies ??????????
2
Partitioning
???????????????,???????,???????????
3
Divide and Conquer??
Characterized by dividing problem into
sub-problems of same form as larger problem.
Further divisions into still smaller
sub-problems, usually done by recursion. ????,????
?????????????,?????????????????????
?????????,?????????????????
4
Partitioning/Divide and Conquer Examples
Many possibilities.
Operations on sequences of number such as
simply adding them together Several sorting
algorithms can often be partitioned or
constructed in a recursive fashion Numerical
integration N-body problem
5
Partitioning a sequence of numbers into parts and
adding the parts
6
??

• Sendrecv (for loop send, for loop recv)
• Bcastrecv (master bcast all to slaves for
  loop recv, slavesreceive all from master send
  result to master)
• Scatterreduce

7
Tree construction
???
8
Dividing a list into parts
send
recv
9
Partial summation
10
??P0 /division phase/ Divide(s1,s1,s2)/divide
s1 into two, s1 and s2/ Send(s2,P4) Divide(s1,s1
,s2) Send(s2,P2) Divide(s1,s1,s2) Send(s2,P1)
/combining phase/ part_sums1 Recv(part_sum1,
P1) part_sumpart_sumpart_sum1 Recv(part_sum1,
P2) part_sumpart_sumpart_sum1 Recv(part_sum1,
P4) part_sumpart_sumpart_sum1
11
??P4 /division phase/ Recv(s1,p0) Divide(s1,s1,
s2) Send(s2,P6) Divide(s1,s1,s2) Send(s2,P5) /
combining phase/ part_sums1 Recv(part_sum1,P
5) part_sumpart_sumpart_sum1 Recv(part_sum1,P
6) part_sumpart_sumpart_sum1 Send(part_sum1,P
0)
12
M???
13
Many Sorting algorithms can be parallelized by
partitioning and by divide and conquer.Example
Bucket sort(???)
14
Bucket sort(???)
One bucket assigned to hold numbers that fall
within each region. Numbers in each bucket
sorted using a sequential sorting algorithm.
Sequential sorting time complexity
O(nlog(n/m). Works well if the original numbers
uniformly distributed across a known interval,
say 0 to a - 1.
15
Parallel version of bucket sort Simple approach
Assign one processor for each bucket.
16
big_array malloc(nsizeof(long
int)) Make_numbers(big_array, n, n/p, p) n_bar
n/p local_array malloc(n_barsizeof(long
int)) MPI_Scatter(big_array, n_bar, MPI_LONG,
local_array, n_bar, MPI_LONG, 0,
MPI_COMM_WORLD) Sequential_sort(local_array,
n_bar) MPI_Gather(local_array, n_bar, MPI_LONG,
big_array, n_bar, MPI_LONG, 0,
MPI_COMM_WORLD)
17
(No Transcript)
18
(No Transcript)
19
Further Parallelization

• Partition sequence into m regions.
• Each processor assigned one region.
• (Hence number of processors, p, equals m.)
• Each processor maintains one big bucket for
  its region.
• Each processor maintains m small buckets, one
  for each
• region.
• Each processor separates numbers in its region
  into its own
• small buckets.
• All small buckets emptied into p big buckets
• Each big bucket sorted by its processor

20
Another parallel version of bucket sort
21
all-to-all broadcast

• An MPI function that can be used to advantage
  here.
• Sends one data element from each process to every
  other process.
• Corresponds to multiple scatter operations, but
  implemented together.
• Should be called all-to-all scatter?

22
From http//www.mhpcc.edu/training/workshop/paral
lel_intro/MAIN.html
23
Applying all-to-all broadcast to emptying small
buckets into big buckets Suppose 4 regions and 4
processors
Big bucket
Small bucket
4.13
24
all-to-all routine actually transfers rows of
an array to columns Transposes a matrix.
25
Numerical Integration

• Computing the area under the curve.
• Parallelized by divided area into smaller areas
  (partitioning).
• Can also apply divide and conquer -- repeatedly
  divide the areas recursively.

26
Numerical integration using rectangles
Each region calculated using an approximation
given by rectangles Aligning the rectangles
27
Numerical integration using trapezoidal method
May not be better!
28
Applying Divide and Conquer
29
Adaptive Quadrature
Solution adapts to shape of curve. Use three
areas, A, B, and C. Computation terminated when
largest of A and B sufficiently close to sum of
remain two areas .
30
Adaptive quadrature with false termination.
Some care might be needed in choosing when to
terminate.
Might cause us to terminate early, as two large
regions are the same (i.e., C 0).
31
Barnes-Hut Algorithm
Start with whole space in which one cube contains
the bodies (or particles).
First, this cube is divided into eight
subcubes. If a subcube contains no
particles, subcube deleted from further
consideration. If a subcube contains one
body, subcube retained. If a subcube
contains more than one body, it is recursively
divided until every subcube contains one body.
32
Creates an octtree - a tree with up to eight
edges from each node. The leaves represent cells
each containing one body. After the tree has
been constructed, the total mass and center of
mass of the subcube is stored at each node.
33
Constructing tree requires a time of O(nlogn),
and so does computing all the forces, so that
overall time complexity of method is O(nlogn).
34
Recursive division of 2-dimensional space
35
Orthogonal Recursive Bisection
(For 2-dimensional area) First, a vertical line
found that divides area into two areas each with
equal number of bodies. For each area, a
horizontal line found that divides it into two
areas each with equal number of bodies. Repeated
as required.