Fibre Channel - Topologies

1
Fibre Channel - Topologies Protocols
2
Agenda

• Why Fibre
• Fibre-Speak
• Fibre Pipes
• Topologies
• Protocols

3
Question

• Why is Fibre in Fibre Channel always misspelled
  F-I-B-R-E instead of F-I-B-E-R?

• Answer Its the Britishspelling of the word.

4
Why Fibre
5
Why Fibre Channel?

• Speed
• 100 MB/s now
• 200 MB/s early next year
• 400 MB/s in the future

• Distance
• 30 meters with copper
• 100s of meters with todays multi-mode fibre
• 1000s of meters with tomorrows single mode fibre

• Flexibility
• Point-to-point
• Loops
• Fabrics

• Its the industry standard interconnect
• Clusters and SANs now
• And an important disk interface

6
Whats a Cluster?

• Group of independent systems that
• Function as a single system
• Appear to users as a single system
• And are managed as a single system
• Clusters are virtual servers

Fibre Channel
7
Whats a SAN?

• Dedicated Network for Storage Devices
• RAID Arrays, Tape and Optical libraries
• Storage management software, e.g Back-up,
  archiving,,
• Network Behind the Servers

LAN
SAN
Fibre Channel
8
Whats a Fibre Disk?

• SCSI Disk
• Single SCSI interface
• Fibre Disk
• Dual Fibre Channel interfaces
• If one fibre loop fails, you can still access a
  fibre disk

SCSI Disks
Fibre Disks
9
And Its A Key Mylex Strategy

• SF and FL Controllers
• Fibre to the server, cluster or SAN
• SCSI to the disks
• FF Controller
• Fibre to the server, cluster or SAN
• Fibre to the disks
• Fibre Apple, Lion and Lambersomethingorother
• PCI to the server
• Fibre to the disks
• Fibre to another Fibre Apple or Lion (cluster
  configuration)

10
Fibre-Speak
11
Fibre-Speak

• Fibre Node
• Any device on a fibre loop or fabric with a fibre
  port address
• Servers, Array controllers, Tape Libraries, ...
• Fibre Networking Devices
• Devices used to interconnect nodes in loop or
  fabrics
• Fibre Hubs and Fibre Switches
• Fibre Loop
• Topology where nodes are configured in a loop
  like Token Ring
• Fibre networking device -- Hub(s)
• Fibre Fabric
• Topology where nodes are configured in a star or
  mesh (the cloud)
• Fibre networking device -- Switch(es)
• And it could have hub(s) hanging off switches

12
More Fibre Speak

• GBIC
• Gigabit Interface Converter
• Converts optical signal to copper signal (and
  vice versa)
• Can plug into any fibre port
• DB-9 and HSSDC
• Fibre connectors that provide a fibre port
• External product development kit (SF-DEVKIT) have
  HSSDCs
• LIP
• Loop initialization process
• Similar to a SCSI bus reset
• LRC
• Loop redundancy circuit
• Allows fibre loops to bypass non-operational
  fibre nodes

13
Fibre Pipes
14
Gigabit Transmission

• SCSI
• Ultra 40 MB / s
• Ultra/2 LVD 80 MB / s -- Now
• Ultra/3 LVD 160 MB / s -- Late 99
• Fibre Channel
• 1st Generation Gb Fibre 100 MB/s now
• 2nd Generation 2 Gb Fibre 200 MB/s 2000
• 3rd Generation 4 Gb Fibre 400 MB/s 2001

15
Serial Transmission

• Fibre is a serial technology
• One bit at a time through a pipe

001001110101001

• Serial pipes can be designed to go faster over
  greater distances

16
Fibre Cables

• Fibre cables have two pipes
• One to move bits in each direction
• Also called a link
• Two unidirectional fibres transmitting in
  opposite directions
• fibres -- copper or optical

001001110101001
001001110101001
17
Fibre Channel Pipes

• Fibre Channel defines copper and optical pipes
• Copper Cables
• Electrons represent bits
• Cheaper
• Distance limited -- 30 meters
• Optical Fibre Cables
• Light pulses represent bits
• Pulses are generated by a laser or a LED
• And detected at the other end by a transceiver
• More expensive
• Distances up to thousands of meters
• Depends on type of optical pipe, transmission
  speed, ...

18
Fibre Channel Glass Pipes

• Multi-Mode Optical Fibre -- Now
• Pipe is large enough to simultaneously transmit
  multiple streams of light
• Comes in two diameters
• 50 microns -- de facto standard
• 62.5 microns
• Single Mode Optical Fibre -- Future
• Pipe is only wide enough to transmit a single
  steam of light
• 9 microns

19
Question

• How big is a micron?

20
Speed / Distance of FC Pipes

• Copper-Local Area Wiring
• Multi-Mode Building Wiring
• Single Mode -Campus Wiring

62.5 Micron Multi-Mode 175 m 150 m 90 m
50 Micron Multi-Mode 500 m 300 m 150 m
9 Micron Single Mode 10 km 2 km 2 km
Transmission Speed 100 MB / s 200 MB / s 400
MB / s
Copper 30 m
21
Management Question

• How big is a meter?

• Hint 1,000 Microns Meter

• Answer Enough space to put 4,000
  east coast brains side by side.

22
Topologies
23
Topologies

• Physical Model -- Topologies
• How the servers, arrays, hub and switches are
  physically connected
• Communications Model
• How nodes talk to each other and its always
  point-to-point
• At any point in time, one node talks to one other
  node
• Fibre Ports
• N-Port (node in a fabric topology)
• F-Port (switch port in a fabric topology)
• NL-Port (node in a loop topology
• FL-Port (hub port in a loop topology)FC Node
  server or an arrayFC Ports have world-wide
  addresses like Ethernet portsSF, FL and FF have
  two FC Ports (addresses stored in controllers)

24
Three Fibre Topologies
25
Fibre Hubs

• Networking Device for Connecting Fibre Nodes in a
  Loop
• Simplifies cabling
• Connects fibre nodes with different fibre cable
  types
• Can plug copper and different types of optical
  into the same hub
• Improves cluster or SAN availability
• Allows fibre nodes to be hot plugged and removed
• Bandwidth in a Loop is Constant (100 MB/s Today)
• As servers are added to a loop less bandwidth
  per server
• Cheaper Per Port than Fibre Switches

26
FC Loop w/ Four Port Fibre Hubs
Hub
SF
SF
27
Server Failure - HUB LRC Heals Loop
LRC Isolates Dead Server
X
Hub
SF
SF
28
SF Failure -- HUB LRC Heals Loop
Hub
X
SF
SF
29
Fibre Switches

• Networking Device for Connecting Fibre Nodes in a
  Fabric
• Does everything a hub does
• Same benefits PLUS
• Bandwidth in a Fabric is Scaleable
• Add more servers to a loop
• Bandwidth per server is constant
• But bandwidth for the fabric increases with each
  added node
• Expensive Per Port Compared to Fibre Hubs

30
FC Fabric w/ 4 Port Fibre Switch - 200 MB/s
Pairs of Nodes Communicate in Loops and Fabrics
S0
S1
Switch
But They Communicate Simultaneously in Fabrics
SF 0
SF 1
31
FC Fabric w/ 4 Port Fibre Switch - 200 MB/s
S0
S1
Switch
Any node can talk to any other node
SF 0
SF 1
32
FC Fabric w/ 8 Port Fibre Switch - 400 MB/s
S0
S1
S0
S1
Switch
SF 0
SF 1
SF 0
SF 1
33
Fibre Protocols
34
Fibre Channel Protocol

• Fibre Channel Spec Defines
• Physical aspects
• Nodes, ports, links (cables),
• And the communications model
• Frame -- carries the payload (chunks of data)
• Sequence -- set of related frames (for flow
  control)
• Exchange -- set of related sequences (sets up the
  conversation)
• Fibre Protocol is a Low Level Protocol
• And is used to transport other protocols
• It transports the SCSI protocol to talk to disks
  (SCSI over FC)
• It can transport the IP protocol for array
  management
• Plan to port the GAM server to external
  controllers
• And talk to it from WAM clients (IP over FC)

35
Last Question

• Whats a protocol?

36
Or