Multilevel Combinatorial Methods in Scientific Computing

1
Multilevel Combinatorial Methods in Scientific
Computing

• Bruce Hendrickson
• Sandia National Laboratories
• Parallel Computing Sciences Dept.

2
An Overdue Acknowledgement

• Parallel Computing Uses Graph Partitioning
• We owe a deep debt to circuit researchers
• KL/FM
• Spectral partitioning
• Hypergraph models
• Terminal propagation

3
In Return

• Weve given you
• Multilevel partitioning
• hMETIS
• Our applications are different from yours
• Underlying geometry
• More regular structure
• Bounded degree
• Partitioning time is more important
• Different algorithmic tradeoffs

4
Multilevel Discrete Algorithm

• Explicitly mimic traditional multigrid
• Construct series of smaller approximations
• Restriction
• Solve on smallest
• Coarse grid solve
• Propagate solution up the levels
• Prolongation
• Periodically perform local improvement
• Smoothing

5
Lots of Possible Variations

• More complex multilevel iterations
• E.g. V-cycle, W-cycle, etc.
• Not much evidence of value for discrete problems
• Key issue properties of coarse problems
• Local refinement multi-scale improvement
• Ill focus on graph algorithms
• Most relevant to VLSI problems

6
Not a New Idea

• Idea is very natural
• Reinvented repeatedly in different settings
• Focus of this workshop is on heuristics for hard
  problems
• Technique also good for poly-time problems
• E.g. Geometric point detection (Kirkpatrick83)

7
Planar Point Detection

• O(n log n) time to preprocess
• O(log n) time to answer query

8
Multilevel Graph Partitioning

• Invented Independently Several Times
• Cong/Smith93
• Bui/Jones93
• H/Leland93
• Related Work
• Garbers/Promel/Steger90, Hagen/Khang91,
  Cheng/Wei91
• Kumar/Karypis95, etc, etc.
• Multigrid Metaphor H/Leland93 (Chaco)
• Popularized by Kumar/Karypis95 (METIS)

9
Multilevel Partitioning

• Construct Sequence of Smaller Graphs
• Partition Smallest
• Project Partition Through Intermediate Levels
• Periodically Refine
• Why does it work so well?
• Refinement on multiple scales (like multigrid)
• Key properties preserved on (weighted) coarse
  graphs
• (Weighted) partition sizes
• (Weighted) edge cuts
• Very fast

10
Coarse Problem Construction

• Find maximal matching
• Contract matching edges
• Sum vertex and edge weights
• Key Properties
• Preserves (weighted) partition sizes
• Preserves (weighted) edge cuts
• Preserves planarity
• Related to min-cut algorithm of Karger/Stein96

11
Extension ITerminal Propagation

• Dunlop/Kernighan85
• Skew partitioning to address constrained vertices
• Also useful for parallel computing
• Move few vertices when repartitioning
• Assign neighboring vertices to near processors
• H/Leland/Van Dreissche96
• Basic idea
• Vertex has gain-like preference to be in
  particular partition

12
MultilevelTerminal Propagation

• How to include in multilevel algorithm?
• Simple idea
• When vertices get merged, sum preferences
• Simple, fast, effective
• Coarse problem precisely mimics original

13
Extension IIFinding Vertex Separators

• Useful for several partitioning applications
• E.g. sparse matrix reorderings
• One idea edge separator first, then min cover
• Problem multilevel power on wrong objective
• Better to reformulate multilevel method
• Find vertex separators directly
• H/Rothberg98

14
MultilevelVertex Separators

• Use same coarse constructor
• Except edge weights dont matter
• Change local refinement coarse solve
• Can mimic KL/FM
• Resulted in improved matrix reordering tool
• Techniques now standard

15
Extension IIIHypergraph Partitioning

• Coarse construction
• Contract pairs of vertices?
• Contract hyperedges?
• Traditional refinement methodology
• See talk tomorrow by George Karypis

16
Envelope Reduction

• Reorder rows/columns of symmetric matrix to keep
  nonzeros near the diagonal

17
Graph Formulation

• Each row/column is a vertex
• Nonzero in (i,j) generates edge eij
• For row i of matrix (vertex i)
• Env(i) max(i-j such that eij in E)
• Envelope ? Env(i)
• Find vertex numbering to minimize envelope
• NP-Hard

18
Status

• Highest Quality algorithm is spectral ordering
• Sort entries of Fiedler vector (Barnard/Pothen/Sim
  on95)
• Eigenvector calculation is expensive
• Fast Sloan (Kumfert/Pothen97) good compromise
• Now multilevel methods are comparable
• (Boman/H96, Hu/Scott01)
• Related ordering problems with VLSI relevance
• Optimal Linear Arrangement
• Minimize ? i-j such that eij in E

19
Challenges for MultilevelEnvelope Minimization

• No precise coarse representation
• Cant express exact objective on coarse problem
• No incremental update for envelope metric
• I.e. no counterpart of Fiduccia/Mattheyses
• Our solution Use approximate metric
• 1-sum / minimum linear arrangement
• Allows for incremental update
• But still not an exact coarse problem
• VLSI applications?

20
Results

• Disappointing for envelope minimization
• We never surpassed best competitor
• Fast Sloan algorithm
• But Hu/Scott01 succeeded with similar ideas
• Better for linear arrangement, but
• Not competitive with Hur/Lillis99
• Multilevel algorithm with expensive refinement

21
Lessons Learned

• Good coarse model is the key
• Need to encode critical properties of full
  problem
• Progress on coarse instance must help real one
• Must allow for efficient refinement methodology
• Different objectives require different coarse
  models
• Quality/Runtime tradeoff varies w/ application
• Must understand needs of your problem domain
• For VLSI, quality is worth waiting for
• All aspects of multilevel algorithm are impacted

22
Conclusions

• Appropriate coarse representation is key
• Lots of existing ideas for construction coarse
  problem
• Matching contraction, independent sets,
  fractional assignment, etc.
• Multigrid metaphor provides important insight
• Were not yet fully exploiting multigrid
  possibilities
• Do we have something to offer algebraic
  multigrid?
• Need for CS recognition of multilevel paradigm
• Rich, general algorithmic framework, but not in
  any textbook
• Not the same as divide-and-conquer

23
Acknowledgements

• Shanghua Teng
• "Coarsening, Sampling and Smoothing Elements of
  the Multilevel Method
• Rob Leland
• Erik Boman
• Ed Rothberg
• Tammy Kolda
• Chuck Alpert
• DOE MICS Office