IP-Based Storage Networking

1
IP-Based Storage Networking

• ???
• ???????????

2
Outline

• Introduction
• iSCSI Architecture and Standardization
• Issues and Solutions of  IP Storage
• Performance
• Security
• Cost
• Interoperability
• Storage over WAN
• Conclusion

3
Introduction

• SAN Storage Area Network
• NAS Network Attached Storage
• PCI Bus speed
• Gigabit and 10Gigabit Ethernet
• Storage is no fun until networking comes in.

4
Introduction

• Standardization The Internet Engineering Task
  Force (IETF) has approved the iSCSI standard
  since Feb. 2003.
• A mapping of the SCSI remote procedure invocation
  model on top of the TCP protocol.
• A new SCSI transport as defined by the SCSI
  SAM-2 document.
• Equivalent protocols include SPI-2, FCP-2,
• To take compelling advantages from the
  IP/Ethernet infrastructure.

5
SCSI-3 Architecture Roadmap
6
Layers and Sessions
7
Layer and Session (Cont.)

• Conceptual Layering Model
• SCSI layer builds/receives SCSI Command Data
  Blocks (cf. SCSI Architecture Model - 2)
• iSCSI layer builds/receives iSCSI PUDs
• TCP Connections form an initiator-target
  session
• Session
• A group of TCP connections linking an initiator
  with a target.
• Defined by a session ID

8
What customer problems does iSCSI solve?

• iSCSI provides a cost-effective transport for
  Storage Area Network (SAN) when compared with
  Fibre Channel.
• iSCSI enables affordable storage consolidation
  solutionsparticularly in environments populated
  with mid range servers.
• Together with Storage Management Solution, iSCSI
  also provides affordable disaster recovery,
  backup, and secondary storage solutions.

9
Performance and Cost Hardware versus
software-based solutions

• Software iSCSI initiators provide the lowest cost
  iSCSI solution. A software-only iSCSI initiator
  uses a standard Ethernet NIC or a NIC with TCP
  offload Engine (TOE) to process the iSCSI
  commands and the TCP/IP protocol. For
  workstations/servers with 2 GHz CPUs, iSCSI
  protocol processing does not impose a significant
  overhead for most customer workloads. Example
  Microsoft iSCSI initiator driver.
• Hardware With older CPUs and heavily loaded
  servers, a hardware-assisted iSCSI initiator is
  appropriate as the iSCSI initiator HBA can
  offload the CPU. Example Intel iSCSI HBA (Intel
  Pro 1000T)

10
Performance and CPU overhead (For single Gbps
connection)
iSCSI HBA iSCSI driver SAN/IP
Sequential Read 94MB/s 59MB/s 105MB/s
CPU Utilization(Client) 17 23 35
Note With multiple connections, the performance
can be enhanced even more.
11
Testing Configuration

• Scenarios
• Intel iSCSI HBA (Pro 1000 T)
• Microsoft iSCSI initiator (software)
• SAN/IP Client

GBE Switch
Intel IOMeter running on Win2k Server
FalconStors IPStor Server (iSCSI target)
12
iSCSI Security

• Fibre Channel is perceived to be more secure as
  it is a private network. However, it is a Layer
  2 protocol with no security mechanism built in
  essentially.
• The iSCSI spec, on the other hand, covers
  initiator and target authentication (using CHAP,
  SRP, Kerberos, and SPKM) to prevent unauthorized
  access and permit only trustworthy nodes. In
  addition, IPsec can be used to provide privacy
  and prevents eavesdropping.
• The solutions are readily available today.

13
Security Configuration

• IPsec
• Peers must authenticate each other before data
  transfer
• Data is encrypted on the wire
• Operates at IP layer
• CHAP
• One way authentication mechanism, but may be done
  by both Initiator and Target
• Operates at iSCSI protocol layer
• iSCSI CHAP and IPsec rely upon the peer knowing
  a secret for authentication
• Pre-shared or private key

14
IPsec Configuration

• Each target and initiator pairing has an
  identical Pre-shared Key
• Service provides interface for management app to
  specify pre-shared key and tunnel addresses on
  initiator
• Service caches keys for Microsoft SW and HBA
  initiators
• Service will program Windows IPsec on behalf of
  Microsoft SW initiator

15
iSCSI interoperability

• Operating system and application vendors often
  have a catalog of qualified hardware solutions.
  The Microsoft Windows Catalog lists iSCSI
  hardware devices that have been qualified. In
  late 2003, more than 14 leading storage vendors
  had qualified their iSCSI hardware products under
  Microsoft iSCSI Designed for Windows Logo
  Program.
• Fibre Channel interoperability problems were
  primarily due to two issues. First, the vendors
  implemented the SCSI3 command set differently.
  Secondly, Fibre Channel lacks built-in networking
  capabilities.
• In iSCSIs case, the interoperability issues are
  greatly reduced. In addition, SNIA, SNW, and
  other labs are continuously working on the
  interoperability issues.

16
IDCs prediction

• IDC expects that iSCSI adoption will commence in
  most countries in the Asia Pacific region during
  2003 with progressive deployment expected in
  2004. In many cases, an iSCSI implementation will
  be complementary to existing fibre channel SANs.
• Overall, IDC believes the two most likely places
  where iSCSI will be adopted are
• In smaller organizations that haven't networked
  their storage, yet are familiar with TCP/IP.
• Large organizations that will use iSCSI to link
  FC SANs.
• Graham Penn, Director, Asia Pacific Storage, IDC

17
Microsofts iSCSI initiator

• The Microsoft iSCSI Software Initiator version
  1.0 package was released to the Web June 25,
  2003. The Microsoft iSCSI software initiator
  allows a Windows-based computer to serve as an
  iSCSI initiator to connect to iSCSI targets on an
  Internet Protocol Storage Area Network (IP SAN).
• All iSCSI devices appear in Windows as a local
  disk and can be managed in Disk Administrator as
  any other local disk.
• Download
• http//www.microsoftcom/downloads/details.aspx?Fam
  ilyID12cb3c1a-15d6-4585-b385-befd1319f825Display
  Langen

18
Benefits of using SANs

• Enhance applications performance by freeing up
  enterprise network
• Permits more desktop use of RAID technology
• Consolidated backups and archives
• Disk mirroring, backups to disaster recovery
  sites
• High availability mission critical databases
• Distributed (logical) server clustering
• Disk virtualisation

19
SAN in the WAN

• Enhance applications performance by freeing up
  enterprise network
• Permits more desktop use of RAID technology

• Consolidated backups and archives
• Disk mirroring, backups to disaster
• recovery sites
• High availability mission critical databases
• Distributed (logical) server clustering
• Disk virtualisation

20
Is SAN in the WAN possible ?

• Yes, and SAN traffic loads are typically less
  than many people think
• Very few disks or RAID systems can stream at gt 10
  Mbytes/sec, although peaks of 30 Mbyte/sec are
  common
• Even high performance UNIX servers can rarely
  exceed 20 Mbyte/sec
• NT servers are much worse, typically lt 10 Mbytes
  /sec

21
Is SAN in the WAN possible ?

• Tape Subsystems are quite slow
• 4 to 10 Mbyte/sec streaming is normal. Peak data
  of 20 Mbyte/sec maximum per interface and drive
• Disk mirroring depends upon application
• Transaction or database system often below 1
  Mbyte/sec
• Backups may be faster, but are limited by system
  (controller / drive) performance

22
Is SAN in the WAN possible ?

• Given that high bandwidth network links are
  increasingly affordable
• T3 (45 Mbit/sec) is capable of around 5 MBytes
  /sec easily enough to run a remote DLT drive or
  to handle disk mirroring
• OC3 (155 Mbit/sec) is capable of about 17
  Mbytes/Sec
• OC-12 (622 Mbit/sec) is capable of about 65
  Mbytes/sec

23
Is SAN in the WAN possible?

• Yes, but only with pipelined data transfers
• WAN data takes about 5?Secs to travel 1Km, or
  5mSec for 1000 Km. For a single disk reading or
  writing 64KByte blocks at 10 Mbytes/sec, over
  1000 Km distance, non pipelined operation will
  reduce the performance to about 40 of the
  transfer speed. If 8 such blocks are pipelined,
  performance will be 84 of transfer speed.

24
Is SAN in the WAN possible?
64 Kbyte block
6.4 mSec
5 mSec
5 mSec
Disk Ack
Total time taken to transmit data block
and return ack is 6.4 5 5 mSec 16.4
mSec Lost transmission time due to ack 10
mSec Lost efficiency due to ack 10/16.4 60
25
Is SAN in the WAN possible ?

• Yes, but only with low latency WANs
• Data must not be held in queues within the WAN

1.000
OC-3 (Frame) T3 (Frame)
0.800
0.600
mSec Delay
0.400
0.200
0.000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
26
WAN Delays Vs. Efficiency
100
80
128 Kbyte Block
256 Kbyte Block
60
512 Kbyte Block
Efficiency
1MByte Block
40
20
64 Kbyte Block
0
1
5
10
15
20
One way delay mSec
Efficiency at 10 Mbytes/sec for different block
sizes
27
Machine room technology
Today's Storage Area Networks belong in the
Machine Room

• Very high speed
• Restricted transmission distance
• Unreliable protocols not designed for
  communications use
• Primitive windowing

SAN
28
Storage network speeds
Speed
SCSI Type
Clock Mbytes/sec
SCSI-1
5 Mbytes/sec
5 MHz
SCSI-2 (Narrow)
10 Mbytes/sec
10 MHz
SCSI-2 (Wide)
20 Mbytes/sec
10 MHz
Ultra SCSI (Narrow)
20 Mbytes/sec
20 MHz
Ultra SCSI (Wide)
40 Mbytes/sec
20 MHz
Ultra 2 SCSI
80 Mbytes/sec
40 MHz
Ultra 3 SCSI
40 MHz
160 Mbytes/sec
Ultra320 320Mbytes/sec
29
Storage network speeds

• Fibre Channel
• Up to 100 Mbytes/sec
• Runs at 1 Gbaud using 8B/10B encoding, taken
  directly from FDDI standard
• Frame based technology based on FDDI. Uses FDDI
  checksums
• FC-AL shared between lt 126 devices
• 2 Gbaud and 4 Gbaud Fibrechannel coming

30
Transmission distances

• SCSI
• Low voltage differential 25 metres
• Single ended 3 metres
• FibreChannel (100 Mbytes/sec, 1.06 Gbaud)
• Singlemode, 1300 nM lt 10 Km
• Multimode, 850 nM lt 300 metres
• FibreChannel (25 Mbytes/sec, 266 Mbaud)
• Multimode 850 nM lt 2 Km

31
Storage protocols

• SCSI
• defines a simple bus based transmission scheme
  with limited reliability features
• Fibre Channel
• is conceived as a high speed carrier mechanism
  capable of transporting any bit stream reliably,
  but is really a local protocol
• Sequence retry is very inefficient (subsequent
  sequences are repeated)
• Networking layers are missing FibreChannel is
  really a layer 2 technology

32
Windowing

• Performance at a distance requires efficient
  windowing
• SCSI (and SCSI over FibreChannel) does not allow
  this
• SCSI has no inherent windowing commands are
  acknowledged individually by the target
• Command tag queuing is a solution, but is not
  supported by many devices, and is a higher level
  solution to a lower level problem
• FibreChannel, as a transparent transport
  mechanism, does NOT address this problem

33
SAN in the WAN

• To build Storage Networks that operate over WANs
  we need
• Realistic data speeds
• Adaptation of SCSI or FCP (SCSI over
  FibreChannel) to a networking protocol
• Reliable stream transport
• Disk Profiles operate with FibreChannel Class 3
  service, an unacknowledged datagram service
• The only form of ACK is a sequence abort

34
Applications

• Storage Consolidation through IP
• SAN features such as storage virtualization,
  Capacity-on-Demand mirroring, TimeMark/TimeView
  (Disk Journaling), Replication, Backup and
  Recovery, Storage Vaulting, etc, can be carried
  out in a cost effective manner.
• Diskless Blades
• Storage Infrastructure for On-Demand/Utility
  Computing

35
Conclusion

• Simply put, iSCSI provides network storage
  connectivity at Ethernet prices iSCSI brings
  along a simple and cost-effective solution to
  storage networking
• Easy implementation for diskless servers,
  workstations, blades, and utility/on-demand
  computing.
• Create opportunities to the traditional
  networking and storage companies alike, as the
  networking infrastructure can be leveraged.
• iSCSI and IP Storage have arrived and will change
  the perception of computing forever!
• Storage Management is the key to success!