Interconnection topologies

1
Interconnection topologies

• CS 433
• Laxmikant Kale
• University of Illinois at Urbana-Champaign
• Department of Computer Science

2
Distributed memory m/cs
We discussed the communication interface (Ch 7)
PE0
PE1
PEp
Mem0
Mem0
Memp
Interconnection Network
Now, let us talk about the network topology
3
Topology requirements

• Must connect each processor to all others
• I.e. it should be possible to send messages
  between any pair of processors
• Desirable
• Intermediate routing, if needed, should happen
  without interfering with computation on
  intermediate nodes.
• Example ring (of theoretical interest only).

4
Topology metrics

• What makes one topology better than other?
• What should we measure?
• What can we measure?
• C the number of connections (degree) per node
• How densely is the graph connected?
• Metric Diameter of the graph
• Others?
• How much bandwidth is available?
• Store-and-forward vs. circuit switched
• Number of independent wires
• Bisection bandwidth
• Divide the set of processors in two equal
  partitions
• How much data can flow from one partition to the
  other (per unit time)?
• Take the minimum of this quantity over all
  possible partitionings
• (If all processors were to exchange data with
  random others, ..)

5
Ring just to check our definitions

• Let B be the bandwidth of each link
• C2, D P/2, bisection BW 2B
• If every processor wanted to send b bytes to
  every other.

6
Grids

• 2D and 3D grids

7
Hypercubes

• Definition (recursive)
• To make a hypercube of degree k,
• Take 2 hypercubes of degree (k-1)
• Connect the cooresponding processors from each
  set (with 1 extra link)
• Base case one processor
• Number of processors must be a power of 2
• C log(P)
• Diameter log(P)
• Neighbors?
• Number processors, 0 to P-1, and consider their
  binary representation
• My neighbor in dimension A is obtained by
  flipping my Ath bit.

8
Grids vs Hypercubes

• Is there any disadvantage to hypercubes?
• Seem perfect log P connections, log P diameter
• Wire length
• Why does it matter?
• Signal speed, bandwidth per link (B)
• Grids wires can be very short
• Hypercubes wire length of the same order as the
  size of machine
• Some wires must cross almost the entire physical
  extent
• So, grids are the current choice
• Is there an alternative that combines advantages
  of both?

9
K-ary n-cube

• Disadvantage of hypercube?
• Wire lengths? May span the entire machine
• Leads to reduced bandwidth per link
• Number of connections per node increases with the
  size of the machine
• K-ary ncube allows one to seek intermediate
  points between grids and hypercubes.
• N-dimensional hyper cube, but number of
  processors along each dimension is k (instead of
  2 of binary hypercube)

10
Trees

• Tree as a topology
• branching factor C-1 (except at root C)
• depth log P (to the base C-1)
• diameter 2depth
• Problem?
• Bisection bandwidth
• Bottleneck near root

11
Fat Trees

• If we need more bandwidth as we go near the root
• Add more parallel links as we go up the tree
• Switch complexity?
• Used in the connection machine

12
Dense graphs

• Moore bound
• 3 optimal configurations exist
• P10, C 3
• Other 2?

13
Tree-on-tree

• Two identical trees, with leaves connected to
  each other.
• No one uses it why?

14
Dense graphs

• Moore bound is much tighter than hypercube
• Constant connections, log P (base C-1) diameter
• 1024 processors
• hypercube diameter 10
• Moore bound on diameter with C10 is 4.
• Random graphs are dense
• Specially constructed graphs
• Utility?