Storage Wide-Area Networks (SWANs)

1
Storage Wide-Area Networks(SWANs)

• Randy H. Katz
• Computer Science Division
• Electrical Engineering and Computer Science
  Department
• University of California, Berkeley
• Berkeley, CA 94720-1776

2
Storage Networks

• Primary goal is to share storage among computers
  in complex, heterogeneous environments, with PCs,
  workstations, file servers, and mainframes
• Storage can be
• Direct attached (e.g., host bus adapter/HBA)
• Network attached (via a file server)
• Channel attached (primarily fibre channel, but
  also IBM SSA)

3
Storage Networks
E.g, Hitachi, EMC, IBM Storage Arrays
Fibre Channel Loops Dual Ported Disks And
Controllers
Remote Storage Manager
DI
DI
DI
DI
Cache
Workstation
Cache
Mainframe
Crossbar Interconnect
Cache
Fibre Channel Or ESCON
Cache
HI
HI
HI
HI
Multiple Host Interconnections
4
Storage Networks

• Major development in 1990s storage networks
• Native FC is a point-to-point or
  loop/string-oriented (arbitrated) method of
  interconnection 1 Gbps, extend up to 10 km
• Emergence of FC fabrics FC switches arranged
  hierarchically to enable connectivity between any
  host and any storage device (e.g., Brocade
  Networks dominates this product space)
• Standardize FC protocol stack lives on top of
  such fabrics device naming, transport, CoS, etc.
• SCSI-3 protocol over FC fabrics
• Gigabit ethernet now emerging in SAN environment

5
Basic Attached Storage

• Device attached by SCSI HBA or channel interface
• Host manages the file-to-block mapping

6
Network-Attached Storage (NAS)aka Network File
Service
Host
NetworkInterface (NI)
Network File Server
Host
LAN
File Name, Offset, Length
OS
NetworkInterface (NI)
Host
NetworkInterface (NI)

• Mapping from File to Block done in
  network-attached File Server, not host

7
Network-Attached Secure Devices (NASD)
Gibson_at_CMU Research project on device embedded
protocol stack, authentication
Host
NetworkInterface (NI)
Network File Server
Host
LAN
File Name, Offset, Length
Disk, Cylinder, Track, Sector
OS
NetworkInterface (NI)
Host
NetworkInterface (NI)
Network-attached Secure Device (NASD)
8
Storage PlatformsStorage Virtualization
Main Frame
LUN Logical Unit Logical disk mapping onto
underlying physical disks on logical block to
physical block basis
LUN, Offset, Length
ChannelInterface
Main Frame
Disk Storage Subsystem
Work Station
LUN To PHY
9
NAS
NAS distinguished by an exported Network File
System interface over a standard Local Area
Network-based transport
File Server
Host
NetworkInterface (NI)
Host
LAN
File Server
File Name, Offset, Length
NetworkInterface (NI)
Host
NetworkInterface (NI)
File Server
10
NAS SAN
SAN distinguished by a block-oriented
interfaceUsually implemented across a
channel-oriented fabric
Main Frame
PHY Device, Cyl, Trk, Sector
File Server
Disk Storage Subsystem
Host
NetworkInterface (NI)
Channel Interface
CI
SAN
LUN, Offset, Length
Host
LAN
File Server
Tape Storage Subsystem
LUN, Offset, Length
File Name, Offset, Length
CI
CI
NetworkInterface (NI)
CI
Host
Optical Disk Storage Subsystem
NetworkInterface (NI)
File Server
Main Frame
11
NAS SAN SWAN
Now extend the NAS or the SAN over a
wide-area network transport NOTE wide-area SAN
is new idea
12
Shared StorageReference Model
Application
File/Record Subsystem
Host
Service Subsystem Discovery, Monitoring Resource
Mgmt, Configuration Security, Billing Redundancy
Mgmt, Back-up High Availability,
Fail-over Capacity Planning
Block Aggregation
Storage Domain
SAN
Device
Block Subsystem
13
SAN Reference Model
Block-oriented SAN
Application
FS
File
Host-based
SAN
SAN-based
Block
Device-based
DA
14
SAN Reference Model
Application
NAS Storage
Host
Host
LAN
FS
File
NAS
Host
SAN
Block
Device
15
SAN Reference Model
Application
Heterogeneous Storage Environment
Host
Host
Host
Host s/w raid
LAN
FS
File
NAS
NAS Head
Host
SAN
SAN
Block
Device
DA
16
Seven Layer Stack
IP NFS, CIFS FTP, SNMP, TFTP, Telnet, FCP,
SCSI-3 TCP, UDP IP LAN, MAN, WAN Phy
Sw GigaE NFS, CIFS FTP, SNMP, TFTP, Telnet, FCP,
SCSI-3 TCP, UDP IP Mac Client/Control Phy
FC SCSI-3 VI IP FC-4Protocol I/FFC-3
Encrypt/AuthenticationFC-2 Framing, FC,Class
of Service FC-1 Encoding,Link Control FC-0 Phy
7 6 5 4 3 2 1
Application Presentation Session Transport Network
Data Link Physical
17
Fibre Channel Protocol Stack

• FC-0 gigabit physical layer
• FC-1 data encoding and link layer control
• FC-2 segmentation/reassembly of data frames,
  flow control, class of service
• FC-3 common services, e.g., encryption
• FC-4 upper layer protocol upon which SCSI-3 or
  IP can run
• Contrast with Gigabit Ethernet
• Ethernet framing, VLAN tagging, frame
  prioritization (8 levels), link aggregation, 1.25
  gbps
• IP Gigabit Ethernet emerging for SANs

18
SCSI-3 Terminology
Target
Initiator
Device Server
Request
LUNs
Application Client
SCSI Client-Server Model
Response
Delivery Subsystem (e.g., Fibre Channel orSerial
SCSI over IP)
19
FC over IP (FCIP)
Tape Storage Subsystem
Tape Storage Subsystem
Server
Server
WAN
IP Network
FC Over IP
FC Switch
FC Switch
FC Over IP
Server
Server
JBOD
JBOD
Tunnel Session

• IETF IP Storage (IPS) working group
• Recall FC fabrics developed in context of machine
  room/building-scale interconnect (e.g., no
  congestion control!)
• FC time outs in wide-area? Flow control
  interaction? QoS?
• How does bridging actually work for FC e2e
  management?

20
Internet FC Protocol (iFCP)

• Gateway to gateway protocol, sessions rather than
  tunnels
• TCP for congestion control, error detection,
  recovery
• Plug FC devices directly into iFCP switches
• Session and naming semantics

21
iFCP Services

• 24-bit N-Port Address ltDomain, Area, Portgt
• iSNS Internet Storage Name Servicediscovery and
  management protocol for IP storage networks
  (IPNSP)
• Protocol specification includes address
  translation feature to allow remote storage
  devices to be assigned a local, FC fabric
  compliant address
• Local commands executed locally on the fabric
• Remote commands executed on top of TCP
  connections
• Error Detection/Time Outs
• Security

22
Other Protocols

• Metro Fibre Channel Protocol (mFCP)
• FCP over IP using UDP rather than TCP (link layer
  flow control and pacing)
• Internet SCSI (iSCSI)
• IP to the storage device
• Serial SCSI block data transfer over IP (SCSI
  Access Method Command SetSAM)
• IPSec, command/data ordering, steering to
  application memory

23
iSNS

• Discovery Process
• Device registration
• WWN or iSCSI names
• Zoning/discovery domains
• iSNS objects
• Portals
• Storage Port
• Storage Nodes

24
Storage Applications

• Data Centers
• Server clustering
• Storage centralization, consolidation, management
• LAN-free back-up
• IP Storage for Remote Applications
• Remote back-up
• Remote mirroring
• Disaster recovery
• Content distribution

25
Rhapsody Networks, Inc.

• Storage Application Director
• Alteon box for storage networks
• Peek into storage packets traversing fabric
  (deep frame classification) and invoke codefor
  encryption/decryption, mirroring, LUN mappings,
  etc.
• Per port software processing and cut-through
  fabric routing
• Data copy engine, table lookup engine, in-transit
  I/O mods, data escrow/trap to software for
  complex errors or event processing
• Intelligent queue management