iSER on IB

1
iSER on IB

• draft-hufferd-iser-ib-00.txt/pdf
• Generalizations to iSER specification
• John Hufferd
• Mike Ko
• Yaron Haviv

2
Abstract

• The iSCSI Extensions for RDMA document currently
  specifies the RDMA data transfer capability for
  iSCSI over iWARP/TCP
• This new document generalizes the iSER document
  to permit it to be used with other RDMA capable
  protocols such as
• InfiniBand

3
Motivation

• Current wordage in iSER is only applicable to
  iSER on iWARP/TCP
• It was felt that the protocol should be made
  generic for RDMA LLPs, and specifically for
• InfiniBand
• Makes an iSCSI based protocol apply across both
  IB and iWARP Networks thus
• Enabling the same basic iSCSI protocol on TCP/IP,
  iWARP, IB
• Eliminating some of the management and discovery
  protocols that would otherwise be needed
• For example, would not have to fix the short
  comings of SRP (SCSI RDMA Protocol) on IB

4
Goals

• To specify changes/adjustments to the wordage in
  the iSER document to make it more General and
  specifically applicable to IB
• These changes should not modify the basic
  operation of iSCSI/iSER when operating on
  iWARP/TCP

5
Examples of changes (1)

• The term iWARP protocol suite is generally
  replaced by RDMA-Capable Protocol (RCP)
• The protocol or protocol suite that provides the
  RDMA functionality, e.g., iWARP, InfiniBand
• Wherever the iWARP term is specific to the TCP
  implementation, it is replaced with iWARP Layer
• The term RNIC is replaced with RDMA-Capable
  Controller (RCC)
• The clause such as an RNIC will be used when an
  iWARP implemented RCC is intended

6
Examples of changes (2)

• The Steering Tag (STag) term has its definition
  extended such that the IB Local Steering Tag
  (L-Key) and the Remote Steering Tag (R-Key)
  are included in the STag definition by way of
  example
• The Definitions for IRD and ORD terms will have
  their definition extended such that the IB
  Responder Resources, and Initiator Depth are
  included in the definitions by way of example
• The Term RDMA-capable protocol (RCP) is used when
  ever any RDMA wire protocol or RDMA protocol
  stack is applicable
• RDMAP is used only when it explicitly refers the
  iWARP protocol

7
Examples of changes (3)

• The Term RDMAP Stream is replaced by the term
  RCP Stream and defined as
• RCP Stream - A single bidirectional association
  between the peer RDMA-capable protocol layers on
  two Nodes over a single transport-level stream.
• For TCP an RCP Stream is also known as an RDMAP
  Stream
• For iSER/TCP, the association is created when the
  connection transitions to iSER-assisted mode
  following a successful Login Phase during which
  iSER support is negotiated
• Needed since IB start its RCP stream mode at
  connection time

8
Examples of changes (4)

• The term RCP Message is defined and used as a
  replacement for the term RDMAP Message the
  definition is
• RCP Message The sequence of packets of the
  RDMA-capable protocol which represent a single
  RDMA operation or a part of RDMA Read Operation.
  For TCP, an RCP Message is also known as an RDMAP
  Message
• When discussing the iSER Hello and HelloReply
  Messages the term "iSER Message" is used
• Instead of RDMAP Message
• This distinction is needed in order to
  accommodate LLPs that have native message
  delivery capability, such as or IB

9
Example of Changes (5)

• The following is placed/updated in the Acronym
  Section
• HCA Host Channel Adapter
• IB InfiniBand
• IPoIB IP over InfiniBand
• LLP Lower Layer Protocol
• TCA Target Channel Adapter

10
Example of Changes (6)

• We permit the iSCSI layer (if appropriate) to use
  the RCP message mode capability immediately after
  connection establishment before enabling
  iSER-assisted mode
• Appropriate for IB
• In this case the iSER Hello and HelloReply
  Messages are not the first RCP Messages, but they
  are the first iSER Messages
• Added a discussion of connection establishment
  along with the use of the RCP messaging
  protocol, for exchanging Login Request and Login
  Response Messages
• Examples are discussed that are appropriate for
  IB, along with the transitioning of the
  connection to iSER mode

11
Examples of changes (7)

• The discussion of Security specifies that all non
  IP protocols will define their own requirements
  for IPsec
• However the iSCSI requirements for IPsec are
  still required
• Wherever an iSER Message enters an IP environment
  from a non IP one (such as IB)
• There is now a SHOULD implement (optional to use)
  requirement on non IP networks for a packet by
  packet security facility that is at least as
  strong as that required by iSCSI
• Note Should is used instead of MUST since
  currently IB does not have such a capability.
  Also, since some implementations use IB like an
  internal PCI-e Bus, some felt it inappropriate to
  have a MUST

12
Generic example of iSCSI/iSER layering in Full
Feature Mode
13
IB Informative Appendix (B)

• Explains how an InfiniBand RC connection can be
  used to carry the iSER protocol
• Various IB Network topologies are shown
• Host Side IB Network, including Gateways
• Storage Side that includes iSER/IB
• IB iSER discovery process is described
• Use of IP over IB (IPoIB)
• SendTargets
• SLP
• iSNS
• Conversion of IP address into IB GID via ARP
  processes
• Discussion of what the IBTA are expected to define

14
Recommendations

• Accept this Draft as an IPS Draft

15
Backup Charts
16
Host Side iSER/IB Topology
17
Storage Side with iSER/IB
18
Info What is Expected that IBTA will define

• The iSER ServiceID
• Means for permitting a Host to establish an
  iSCSI/iSER connection with a peer InfiniBand
  end-node
• Indicating when that end node does not support
  iSER
• So the Host would be able to fall back to
  iSCSI/TCP over IPoIB
• Means for permitting the Host to establish
  connections with
• IB iSER connections on Storage Controllers in
  preference to a Gateway path
• or
• IB iSER connected Gateways in preference to IPoIB
  connected Gateways/Bridges
• or
• Connections to Target Storage Controllers that
  accept iSCSI via IPoIB
• How the ServiceID can be added to the IP port
  number during the connection process