Switching Architectures for Optical Networks

1
Switching Architectures for Optical Networks
2
Internet Reality
Data Center
Access
Long Haul
Access
Metro
Metro
3
Hierarchies of Networks IP / ATM / SONET / WDM
4
Why Optical?

• Enormous bandwidth made available
• DWDM makes 160 channels/? possible in a fiber
• Each wavelength potentially carries about 40
  Gbps
• Hence Tbps speeds become a reality
• Low bit error rates
• 10-9 as compared to 10-5 for copper wires
• Very large distance transmissions with very
  little amplification.

5
Dense Wave Division Multiplexing (DWDM)

• 1

• 2

• 3

Long-haul fiber

• 4

Output fibers

• Multiple wavelength bands on each fiber
• Transmit by combining multiple lasers _at_ different
  frequencies

6
Anatomy of a DWDM System
Terminal B
Terminal A
D E M U X
Transponder Interfaces
M U X
Transponder Interfaces
Post- Amp
Pre- Amp
Line Amplifiers
Direct Connections
Direct Connections

• Basic building blocks
• Optical amplifiers
• Optical multiplexers
• Stable optical sources

7
User Services Core Transport
CORE
EDGE
Frame Relay
Frame Relay
IP Router
IP
ATM Switch
ATM
Sonet ADM
Lease Lines
TDM Switch
Transport Provider Networks
Service Provider Networks
Users Services
8
Core Transport Services

• Provisioned
• SONET circuits.

• Aggregated into
• Lamdbas.

Circuit Origin

• Carried over
• Fiber optic cables.

Circuit Destination
OC-3
OC-3
OC-12
STS-1
STS-1
STS-1
9
WDM Network Wavelength View
10
Relationship of IP and Optical

• Optical brings
• Bandwidth multiplication
• Network simplicity (removal of redundant layers)
• IP brings
• Scalable, mature control plane
• Universal OS and application support
• Global Internet
• Collectively IP and Optical (IPOptical)
  introduces a set of service-enabling technologies

11
Typical Super POP
Interconnection Network
SONET
Core ATM Switch
Voice Switch
Core IP router
Large Multi-service Aggregation Switch
Coupler Opt.amp
OXC
DWDM ADM
DWDM Metro Ring
12
Typical POP
Voice Switch
OXC
D W D M
D W D M
SONET-XC
13
What are the Challenges with Optical Networks?

• Processing Needs to be done with electronics
• Network configuration and management
• Packet processing and scheduling
• Resource allocation, etc.
• Traffic Buffering
• Optics still not mature for this (use Delay Fiber
  Lines)
• 1 pkt 12 kbits _at_ 10 Gbps requires 1.2 ?s of
  delay gt 360 m of fiber)
• Switch configuration
• Relatively slow

14
Optical Hardware

• Optical Add-Drop Multiplexer (OADM)
• Allows transit traffic to bypass node optically

Add and Drop
15
Wavelength Converters

• Improve utilization of available wavelengths on
  links
• All-optical WCs being developed
• Greatly reduce blocking probabilities

16
Late 90s Backbone Nodes
17
Problems

• About 80 traffic through each node is
  pass-through
• No need to electronically process such traffic
• 80-channel DWDM requires 80 ADMs
• Speed upgrade requires replacing all the ADMs in
  the node

18
Today Optical Cross Connect (OXC)
Optical
Crossconnect
Digital
Terabit
ATM
Cross
IP
Backbone
Connect
Router
Switch
DWDM
Multiplexer Demultiplexer
IP
ATM
Router
Switch
Source JPMS
19
Wavelength Cross-Connects (WXCs)

• A WDM network consists of wavelength
  cross-connects (WXCs) (OXC) interconnected by
  fiber links.
• 2 Types of WXCs
• Wavelength selective cross-connect (WSXC)
• Route a message arriving at an incoming fiber on
  some wavelength to an outgoing fiber on the same
  wavelength.
• Wavelength continuity constraint
• Wavelength interchanging cross-connect (WIXC)
• Wavelength conversion employed
• Yield better performance
• Expensive

20
Wavelength Router
Wavelength Router
Control Plane Wavelength Routing Intelligence
Data Plane Optical Cross Connect Matrix
Unidirectional DWDM Links to other Wavelength
Routers
Unidirectional DWDM Links to other Wavelength
Routers
Single Channel Links to IP Routers, SDH Muxes, ...
21
Optical Network Architecture
Mesh Optical Network
UNI
UNI
IP Network
IP Network
IP Router
Control Path
OXC Control unit
Optical Cross Connect (OXC)
Data Path
22
OXC Control Unit

• Each OXC has a control unit
• Responsible for switch configuration
• Communicates with adjacent OXCs or the client
  network through single-hop light paths
• These are Control light paths
• Use standard signaling protocol like GMPLS for
  control functions
• Data light paths carry the data flow
• Originate and terminate at client networks/edge
  routers and transparently traverse the core

23
Optical Cross-connects (No wavelength conversion)
All Optical Cross-connect (OXC) Also known as
PhotonicCross-connect (PXC)
Optical Switch Fabric
24
Optical Cross-Connect with Full Wavelength
Conversion
Wavelength
Converters
l
2
l
1
l
l
l
l
l
l
1,
2,
... ,
n
1,
2,
... ,
n
l
l
2
1
1
1
l
l
n
n
l
l
1
1
l
l
l
l
l
l
1,
2,
... ,
n
1,
2,
... ,
n
l
l
2
2
2
2
l
l
n
n
.
.
.
.
.
.
l
l
1
n
l
l
l
l
l
l
1,
2,
... ,
n
1,
2,
... ,
n
l
l
2
1
M
M
l
l
n
2
Wavelength
Wavelength
Optical CrossBar
Mux
Demux
Switch

• M demultiplexers at incoming side
• M multiplexers at outgoing side
• Mn x Mn optical switch has wavelength converters
  at switch outputs

25
Wavelength Router with O/E and E/O
26
O-E-O Crossconnect Switch (OXC)
Outgoing fibers
Incoming fibers
Individual wavelengths
O
O
Demux
Mux
E/O
E
1
1
E/O
E/O
E/O
2
2
E/O
WDM (many ?s)
E/O
N
E/O
N
E/O
E/O
Switches information signal on a particular
wavelength on an incoming fiber to (another)
wavelength on an outgoing fiber.
27
Optical core network Opaque (O-E-O) and
transparent (O-O) sections
Transparent optical island
E/O
O/E
Client signals
O
O
O
O
E
E
O
to other nodes
from other nodes
O
O
O
O
O
O
E
E
Opaque optical network
28
OEO vs. All-Optical Switches
OEO
All-Optical

• Capable of status monitoring
• Optical signal regenerated improve
  signal-to-noise ratio
• Traffic grooming at various levels
• Less aggregated throughput
• More expensive
• More power consumption

• Unable to monitor the contents of the data stream
• Only optical amplification signal-to-noise
  ratio degraded with distance
• No traffic grooming in sub-wavelength level
• Higher aggregated throughput
• 10X cost saving
• 10X power saving

29
Large customers buy lightpaths
A lightpath is a series of wavelength links from
end to end.
optical fibers
One fiber
Repeater
cross-connect
30
Hierarchical switching Node with switches of
different granularities
O
O
O
A. Entire fibers
Fibers
Fibers
Express trains
O
O
O
B. Wavelength subsets
E
O
O
C. Individual wavelengths
Local trains
31
Wide Area Network (WAN)
WAN Up to 200-500 wavelengths 40-160
Gbit/s/l wavebands (gt 10 l)
OXC Optical Wavelength/Waveband Cross Connect
32
Packet (a) vs. Burst (b) Switching
33
MAN (Country / Region)
IP packets
optical burst formation
34
Optical Switching Technologies

• MEMs MicroElectroMechanical
• Liquid Crystal
• Opto-Mechanical
• Bubble Technology
• Thermo-optic (Silica, Polymer)
• Electro-optic (LiNb03, SOA, InP)
• Acousto-optic
• Others

Maturity of technology, Switching speed,
Scalability, Cost, Relaiability (moving
components or not), etc.
35
MEMS Switches for Optical Cross-Connect
Proven technology, switching time (10 to 25
msec), moving mirrors is a reliability problem.
36
WDM transparent transmission system
(O-O nodes)
Wavelengths aggregator
Wavelengths disaggregator
O
O
O
O
O
O
Fibers
multiple ?s
Optical switching fabric (MEMS devices, etc.)
Tiny mirrors
Incoming fiber
Outgoing fibers
37
Upcoming Optical Technologies

• WDM routing is circuit switched
• Resources are wasted if enough data is not sent
• Wastage more prominent in optical networks
• Techniques for eliminating resource wastage
• Burst Switching
• Packet Switching
• Optical burst switching (OBS) is a new method to
  transmit data
• A burst has an intermediate characteristics
  compared to the basic switching units in circuit
  and packet switching, which are a session and a
  packet, respectively

38
Optical Burst Switching (OBS)

• Group of packets a grouped in to bursts, which
  is the transmission unit
• Before the transmission, a control packet is sent
  out
• The control packet contains the information of
  burst arrival time, burst duration, and
  destination address
• Resources are reserved for this burst along the
  switches along the way
• The burst is then transmitted
• Reservations are torn down after the burst

39
Optical Burst Switching (OBS)
40
Optical Packet Switching

• Fully utilizes the advantages of statistical
  multiplexing
• Optical switching and buffering
• Packet has Header Payload
• Separated at an optical switch
• Header sent to the electronic control unit, which
  configures the switch for packet forwarding
• Payload remains in optical domain, and is
  re-combined with the header at output interface

41
Optical Packet Switch

• Has
• Input interface, Switching fabric, Output
  interface and control unit
• Input interface separates payload and header
• Control unit operates in electronic domain and
  configures the switch fabric
• Output interface regenerates optical signals and
  inserts packet headers
• Issues in optical packet switches
• Synchronization
• Contention resolution

42

• Main operation in a switch
• The header and the payload are separated.
• Header is processed electronically.
• Payload remains as an optical signal throughout
  the switch.
• Payload and header are re-combined at the output
  interface.

hdr
CPU
payload
hdr
payload
hdr
payload
Re-combined Wavelength i output port j
Optical packet
Wavelength i input port j
Optical switch
43
Output port contention

• Assuming a non-blocking switching matrix, more
  than one packet may arrive at the same output
  port at the same time.

Output ports
Optical Switch
Input ports
payload
hdr
. . .
payload
hdr
. . .
. . .
. . .
payload
hdr
44
OPS Architecture Synchronization
Occurs in electronic switches solved by input
buffering
Slotted networks

• Fixed packet size
• Synchronization stages required

Sync.
45
OPS Architecture Synchronization
Slotted networks

• Fixed packet size
• Synchronization stages required

Sync.
46
OPS Architecture Synchronization
Slotted networks

• Fixed packet size
• Synchronization stages required

Sync.
47
OPS Architecture Synchronization
Slotted networks

• Fixed packet size
• Synchronization stages required

Sync.
48
OPS Architecture Synchronization
Slotted networks

• Fixed packet size
• Synchronization stages required

Sync.
49
OPS Architecture Synchronization
Sync.
50
OPS Contention Resolution

• More than one packet trying to go out of the same
  output port at the same time
• Occurs in electronic switches too and is resolved
  by buffering the packets at the output
• Optical buffering ?
• Solutions for contention
• Optical Buffering
• Wavelength multiplexing
• Deflection routing

51
OPS Architecture
Contention Resolutions
1
1
1
2
2
1
3
3
4
4
52
OPS Contention Resolution

• Optical Buffering
• Should hold an optical signal
• How? By delaying it using Optical Delay Lines
  (ODL)
• ODLs are acceptable in prototypes, but not
  commercially viable
• Can convert the signal to electronic domain,
  store, and re-convert the signal back to optical
  domain
• Electronic memories too slow for optical networks

53
OPS Architecture
Contention Resolutions

• Optical buffering

1
1
1
2
2
1
3
3
4
4
54
OPS Architecture
Contention Resolutions

• Optical buffering

1
1
2
2
3
3
4
4
55
OPS Architecture
Contention Resolutions

• Optical buffering

1
1
1
2
2
3
3
4
4
1
56
OPS Contention Resolution

• Wavelength multiplexing
• Resolve contention by transmitting on different
  wavelengths
• Requires wavelength converters -

57
OPS Architecture
Contention Resolutions

• Wavelength conversion

1
1
1
1
2
2
58
OPS Architecture
Contention Resolutions

• Wavelength conversion

1
1
2
2
59
OPS Architecture
Contention Resolutions

• Wavelength conversion

1
1
1
1
2
2
60
OPS Architecture
Contention Resolutions

• Wavelength conversion

1
1
2
2
61
OPS Architecture
Contention Resolutions

• Wavelength conversion

1
1
1
1
2
2
62
Deflection routing

• When there is a conflict between two optical
  packets, one will be routed to the correct output
  port, and the other will be routed to any other
  available output port.
• A deflected optical packet may follow a longer
  path to its destination. In view of this
• The end-to-end delay for an optical packet may be
  unacceptably high.
• Optical packets may have to be re-ordered at the
  destination

63

• Electronic Switches Using Optical Crossbars

64
Scalable Multi-Rack Switch Architecture
Optical links
Line cardrack
Switch Core

• Number of linecards is limited in a single rack
• Limited power supplement, i.e. 10KW
• Physical consideration, i.e. temperature,
  humidity
• Scaling to multiple racks
• Fiber links and central fabrics

65
Logical Architecture of Multi-rack Switches
Scheduler
Line Card
Line Card
Crossbar
Local Buffers
Local Buffers
Fiber I/O
Laser
Laser
Laser
Fiber I/O
Framer
Framer
Laser
Line Card
Line Card
Local Buffers
Local Buffers
Fiber I/O
Laser
Framer
Laser
Laser
Framer
Laser
Fiber I/O
Switch Fabric System

• Optical I/O interfaces connected to WDM fibers
• Electronic packet processing and buffering
• Optical buffering, i.e. fiber delay lines, is
  costly and not mature
• Optical interconnect
• Higher bandwidth, lower latency and extended link
  length than copper twisted lines
• Switch fabric electronic? Optical?

66
Optical Switch Fabric
Scheduler
Line Card
Line Card
Crossbar
Local Buffers
Local Buffers
Fiber I/O
Laser
Laser
Laser
Fiber I/O
Framer
Framer
Laser
Line Card
Line Card
Local Buffers
Local Buffers
Fiber I/O
Laser
Framer
Laser
Laser
Framer
Laser
Fiber I/O
Switch Fabric System

• Less optical-to-electrical conversion inside
  switch
• Cheaper, physically smaller
• Compare to electronic fabric, optical fabric
  brings advantages in
• Low power requirement, Scalability, Port density,
  High capacity
• Technologies that can be used
• 2D/3D MEMS, liquid crystal, bubbles,
  thermo-optic, etc.
• Hybrid architecture takes advantage of the
  strengths of both electronics and optics

67
Electronic Vs. Optical Fabric
Electronic
Trans.Line
Buffer
Inter-connection
Trans.Line
Buffer
Inter-connection
SwitchingFabric
Optical
Electronic
E/O or O/EConversion
favorred
Optical
Trans.Line
Buffer
Inter-connection
Trans.Line
Buffer
Inter-connection
SwitchingFabric
68
Multi-rack Hybrid Packet Switch
69
Features of Optical Fabric

• Less E/O or O/E conversion
• High capacity
• Low power consumption
• Less cost
• However,

• Reconfiguration overhead (50-100ns)
• Tuning of lasers (20-30ns)
• System clock synchronization (10-20ns or higher)

70
Scheduling Under Reconfiguration Overhead

• Traditional slot-by-slot approach

Scheduler
Time Line

• Low bandwidth usage

71
Reduced Rate Scheduling
Fabric setup (reconfigure)
Traffic transfer
Time slot
Slot-by-slot Scheduling, zero fabric setup time
Slot-by-slot Scheduling with reconfigure delay
Reduced rate Scheduling, each schedule is held
for some time

• Challenge fabric reconfiguration delay
• Traditional slot-by-slot scheduling brings lots
  of overhead
• Solution slow down the scheduling frequency to
  compensate
• Each schedule will be held for some time
• Scheduling task
• Find out the matching
• Determine the holding time

72
Scheduling Under Reconfiguration Overhead

• Reduce the scheduling rate
• Bandwidth Usage Transfer/(ReconfigureTransfer)

Constant

• Approaches
• Batch scheduling TSA-based
• Single scheduling Schedule Hold

73
Single Scheduling

• Schedule Hold
• One schedule is generated each time
• Each schedule is held for some time (holding
  time)
• Holding time can be fixed or variable
• Example LQFHold

74

• Routing and Wavelength Assignment

75
Optical Circuit Switching

• An optical path established between two nodes
• Created by allocation of a wavelength throughout
  the path.
• Provides a circuit switched interconnection
  between two nodes.
• Path setup takes at least one RTT
• No optical buffers since path is pre-set
• Desirable to establish light paths between every
  pair of nodes.
• Limitations in WDM routing networks,
• Number of wavelengths is limited.
• Physical constraints
• limited number of optical transceivers limit the
  number of channels.

76
Routing and Wavelength Assignment (RWA)

• Light path establishment involves
• Selecting a physical path between source and
  destination edge nodes
• Assigning a wavelength for the light path
• RWA is more complex than normal routing because
• Wavelength continuity constraint
• A light path must have same wavelength along all
  the links in the path
• Distinct Wavelength Constraint
• Light paths using the same link must have
  different wavelengths

77
No Wavelength Converters
WSXC
Access Fiber
Wavelength 1
POP
POP
Wavelength 2
Wavelength 3
78
With Wavelength Converters
WIXC
Wavelength 1
Access Fiber
POP
POP
Wavelength 2
Wavelength 3
79
Routing and Wavelength Assignment (RWA)

• RWA algorithms based on traffic assumptions
• Static Traffic
• Set of connections for source and destination
  pairs are given
• Dynamic Traffic
• Connection requests arrive to and depart from
  network one by one in a random manner.
• Performance metrics used fall under one of the
  following three categories
• Number of wavelengths required
• Connection blocking probability Ratio between
  number of blocked connections and total number of
  connections arrived

80
Static and Dynamic RWA

• Static RWA
• Light path assignment when traffic is known well
  in advance
• Arises in capacity planning and design of optical
  networks
• Dynamic RWA
• Light path assignment to be done when requests
  arrive in random fashion
• Encountered during real-time network operation

81
Static RWA

• RWA is usually solved as an optimization problem
  with Integer Programming (IP) formulations
• Objective functions
• Minimize average weighted number of hops
• Minimize average packet delay
• Minimize the maximum congestion level
• Minimize number of Wavelenghts

82
Static RWA

• Methodologies for solving Static RWA
• Heuristics for solving the overall ILP
  sub-optimally
• Algorithms that decompose the static RWA problem
  into a set of individual sub-problems, and solve
  a sub-set
• http//www.tct.hut.fi/esa/java/wdm/

• Methodologies for solving Static RWA
• Heuristics for solving the overall ILP
  sub-optimally
• Algorithms that decompose the static RWA problem
  into a set of individual sub-problems, and solve
  a sub-set
• http//www.tct.hut.fi/esa/java/wdm/

• Methodologies for solving Static RWA
• Heuristics for solving the overall ILP
  sub-optimally
• Algorithms that decompose the static RWA problem
  into a set of individual sub-problems, and solve
  a sub-set
• http//www.tct.hut.fi/esa/java/wdm/

83
Solving Dynamic RWA

• During network operation, requests for new
  light-paths come randomly
• These requests will have to be serviced based on
  the network state at that instant
• As the problem is in real-time, dynamic RWA
  algorithms should be simple
• The problem is broken down into two sub-problems
• Routing problem
• Wavelength assignment problem

84
Optical Circuit Switching all the Way End-to-End
!!!

• Why might this be possible
• Huge CS bandwidth (large of wavelength) BW
  efficiency is not very crucial
• Circuit switches have a much higher capacity
  than Packet switches, and QoS is trivial
• Optical Technology is suited for CS

85
How the Internet Looks Like Today
The core of the Internet is already
predominantly CS. Even a large portion of the
access networks use CS (Modem, DSLs)
86
How the Internet Really Looks Like Today
SONET/SDH DWDM
87
How the Internet Really Looks Like Today
Modems, DSL
88
Why Is the Internet Packet Switched in the First
Place?
Gallager Circuit switching is rarely used for
data networks, ... because of very inefficient
use of the links

• PS is bandwidth efficient Statistical
  Multiplexing
• PS networks are robust

Tanenbaum For high reliability, ... the
Internet was to be a datagram subnet, so if some
lines and routers were destroyed, messages
could be ... rerouted
89
Are These Assumptions Valid Today?

• 10-15 average link utilization in the backbone
  today.
• Similar story for access networks

• PS is bandwidth efficient
• PS networks are robust

• Routers/Switches are designed for lt5s
  down-time per year.
• They take gt1min to recover when they do
  (circuit switches must recover in lt50ms).

90
How Can Circuit Switching Help the Internet?

• Simple switches/routers
• No buffering
• No per-packet processing (just per connection
  processing)
• Possible all-optical data path
• Peak allocation of BW
• No delay jitter

Higher capacity switches
Simple but strict QoS
91
Myth Packet switching is simpler

• A typical Internet router contains over 500M
  gates, 32 CPUs and 10Gbytes of memory.
• A circuit switch of the same generation could
  run ten times faster with 1/10th the gates and
  no memory.

92
Packet Switch Capacity
What wed like (more features) QoS, Multicast,
Security,
Instructions per arriving byte
time
93
What Is the Performance of Circuit
Switching?End-to-End
File 10Mbit
100 clients
1 server
1 Gb/s
x 100
94
What Is the Performance of Circuit Switching?
File 10Gbit/10Mbit
100 clients
1 server
1 Gb/s
x 99
95
What Is the Performance of Circuit Switching?
File 10Gbit/10Mbit
100 clients
1 server
1 Gb/s
x 99
1 Mb/s
96
Possible Implementation

• Create a separate circuit for each flow
• IP controls circuits
• Optimize for the most common case
• TCP (85-95 of traffic)
• Data (8-9 out of 10 pkts)

TCP Switching
97
TCP Switching Exposes Circuits to IP
IP routers
TCP Switches
98
TCP Creates a Connection
99
State Management Feasibility

• Amount of state
• Minimum circuit 64 kb/s.
• 156,000 circuits for OC-192.
• Update rate
• About 50,000 new entries per sec for OC-192.
• Readily implemented in hardware or software.

100
Software Implementation Results

• TCP Switching boundary router
• Kernel module in Linux 2.4 1GHz PC
• Forwarding latency
• Forward one packet 21ms.
• Compare to 17ms for IP.
• Compare to 95ms for IP QoS.
• Time to create new circuit 57ms.