Tesseract* A 4D Network Control Plane

1
Tesseract A 4D Network Control Plane

• Hong Yan, David A. Maltz, T. S. Eugene Ng
• Hemant Gogineni, Hui Zhang, Zheng Cai

Tesseract is a 4-dimensional cube
2
Ideally

• Managing network in a simple way
• Directly and explicitly apply policies to network

Internet
Split load between S5 and S6
Shut down S6 for maintenance on May 1
S1
accurate network view
S5
S6
forwarding state
S2
S3
S4
3
Indirect Control - Fact 1Infer network view by
reverse engineering

• Probe routers to fetch configuration
• Monitor control traffic (e.g., LSAs, BGP update)

Internet
S1
?
?
probe routers and guess network view
S5
S6
?
?
?
S2
S3
S4
4
Indirect Control - Fact 2Policies buried in
box-centric configuration

• Many knobs to tune
• Trial and error

Internet
Change OSPF link weights on S2, S3, S4..
Modify routing policies on S2, S3, S4
S1
?
?
probe routers and guess network view
S5
S6
?
?
?
configuration commands
S2
S3
S4
5
Complex configuration is error-prone and is
causing network outages

• interface Ethernet0
• ip address 6.2.5.14 255.255.255.128
• interface Serial1/0.5 point-to-point
• ip address 6.2.2.85 255.255.255.252
• ip access-group 143 in
• frame-relay interface-dlci 28
• router ospf 64
• redistribute connected subnets
• redistribute bgp 64780 metric 1 subnets
• network 66.251.75.128 0.0.0.127 area 0
• router bgp 64780
• redistribute ospf 64 match route-map
  8aTzlvBrbaW
• neighbor 66.253.160.68 remote-as 12762
• neighbor 66.253.160.68 distribute-list 4 in

access-list 143 deny 1.1.0.0/16 access-list 143
permit any route-map 8aTzlvBrbaW deny 10 match
ip address 4 route-map 8aTzlvBrbaW permit 20
match ip address 7 ip route 10.2.2.1/16 10.2.1.7
6
Indirect Control - Fact 3Indirect Control
Creates Subtle Dependencies

• Example
• Policy 1 use C as egress point for traffic from
  AS X
• Policy 2 enable ECMP for A-C flow

D
3
Desired
Unexpected!
1
1
1
AS X
A
3
1
2
4
AS Y
C
B
7
Direct Control A New World

• Express goals explicitly
• Security policies, QoS, egress point selection
• Do not bury goals in box-specific configuration
• Make policy dependencies explicit
• Design network to provide timely and accurate
  view
• Topology, traffic, resource limitations
• Give decision maker the inputs it needs
• Decision maker computes and pushes desired
  network state
• FIB entries, packet filters, queuing parameters
• Simplify router functionality
• Add new functions without modifying/creating
  protocols or upgrading routers

8
How can we get there?
4D
Generating table entries
Decision Computation Service
D
Routing Table Access Control Table NAT
Table Tunnel Table
Dissemination Service
D
Install table entries
Discovery
D
Modeled as a set of tables
Data Plane
D
9
Tesseract A 4D System
10
Bootstrapping Dissemination
DE1
Beac1 DE1
R1
R3
R2
R5
Beac1 DE1 R3 R2 R4
R4
Beac1 DE1 R3 R2 R4 R5
DE2
11
Bootstrapping Dissemination
DE1
R1
R3
R2
R5

• DE beacons establish ctrl topology
• LSAs flow back from routers over ctrl topology
• After link/switch crash, next beacon heals
  topology

R4
DE2
12
Making Decision
R2s Routing Table 10.0.1/24 R3 10.0.2/24
R5 10.0.3/24 eth0 0/0 R5
R2

• DEs input includes TE goals, reachability matrix
• DE creates tables for each router (FIB, filters)
• Tables source-routed to destination via
  dissemination

13
Decision/Dissemination Interface

• Dissemination Plane
• Flood (pkt)
• Send (pkt, dst)
• RegisterUpCall (fun)
• LinkFailure(link)
• PreferredRoute(dst, route)

• Decision Plane

DE1
R1
LSA
LSA
LSA
14
Reusable Decision Algorithms
15
Code Snippet Floyd-Warshall

• for (unsigned k 0 k lt num k)
• for (unsigned i 0 i lt num i)
• for (unsigned j 0 j lt num j)
  
• if (CostMatrixik ! -1
  CostMatrixkj ! -1)
• if (CostMatrixij -1
• CostMatrixij gt CostMatrixik
  CostMatrixkj
• )
• CostMatrixij CostMatrixik
  CostMatrixkj FirstHopMatrixij
  FirstHopMatrixik
• LastHopMatrixij LastHopMatrixkj
  

16
DE Robustness
DE1

• All DEs send beacons
• Routers send state updates to all DEs on network
• DEs can see each others beacons
• DE with lowest ID is only one to write configs to
  routers
• If active DE crashes, its beacons stop
• Next highest ranking DE takes over

R1
DE1 is alive DE1 is boss
DE1 heard too long ago I becoming boss
DE2
17
Evaluation

• Emulab
• Topologies
• Rocketfuel backbone network (114 nodes, 190
  links) with a maximum round trip delay of 250 ms
• Production enterprise network (40 nodes, 60
  links)

18
Routing Convergence Experiments

• On both backbone and enterprise topologies
• Failure scenarios
• Single link failures
• Single node failures
• Regional failures for backbone (failing all nodes
  in one city)
• Link flapping
• Tesseract versus Aggressively Tuned OSPF (Fast
  OSPF)

19
Enterprise Network, Switch Failures
Tesseract
Fast OSPF
20
Backbone Network, Switch Failures
Fast OSPF
Tesseract
21
Backbone Network, Regional Failures
Fast OSPF
Tesseract
22
Microbenchmark Experiments

• A subset of Rocketfuel topologies with varying
  sizes
• Independently fail each link
• Measure
• DE computation time
• Control traffic volume

23
DE Computation Time
24
Control Traffic Volume
25
Tesseract Applications

• Joint Control of Packet Routing and Filtering
• Problem dynamic routing but static packet filter
  placement
• Solution in addition to computing routes, DE
  computes filter placement based on a reachability
  matrix
• Link Cost Driven Ethernet Switching
• Problem Spanning tree switching makes
  inefficient use of available links
• Solution DE computes both spanning tree and
  shortest paths

26
Link Cost Driven Ethernet Switching Multi-Tree
27
RevisitingRandomize Equal-Cost Shortest Path
Selection

• for (unsigned k 0 k lt num k)
• for (unsigned i 0 i lt num i)
• for (unsigned j 0 j lt num j)
  
• if (CostMatrixik ! -1
  CostMatrixkj ! -1)
• if (CostMatrixij -1
• CostMatrixij gt CostMatrixik
  CostMatrixkj
• CostMatrixij CostMatrixik
  CostMatrixkj rand() gt RAND_MAX/2
• )
• CostMatrixij CostMatrixik
  CostMatrixkj FirstHopMatrixij
  FirstHopMatrixik
• LastHopMatrixij LastHopMatrixkj
  

28
Link Cost Driven Ethernet Switching Multi-Tree
29
Throughput Comparison
30
Related Work

• Separation of forwarding elements and control
  elements
• IETF FORCES, GSMP, GMPLS
• SoftRouter Lakshman
• Centralization of decision making logic
• RCP Feamster, SANE Casado
• Alternative frameworks for network control
• Tempest Rooney, FIRE Partridge

31
Summary

• Direct control is desirable
• Make sophisticated control policies easier to
  understand and deploy
• Simplify router software
• Enable easy innovation
• Direct control is implementable
• Tesseract as proof-of-concept
• Sufficiently scalable
• Fast convergence

32
Future Work

• Formulate models that establish bounds of
  Tesseract
• Scale, latency, stability, failure models,
  objectives
• Structuring decision logic
• Arbitrate among multiple, potentially competing
  objectives
• Unify control when some logic takes longer than
  others
• Protocol improvements
• Better dissemination planes
• Tesseract Router
• Deployment in todays networks
• Data center, enterprise, campus, backbone

33
Reality

• Indirect control with primitive configuration
  interface

Convert to Control plane configuration
Reverse-engineer Routing Logic
TE/Security Policy
Config commands
34
Link Cost Driven Ethernet Switching Mesh
35
Effects of Switch Failure on Aggregated
Throughputs