Internet DRAFT - draft-carrier-rddp-rnic-interop

draft-carrier-rddp-rnic-interop





INTERNET-DRAFT                                     J. Carrier 
Category: Informational                              Adaptec 
draft-carrier-rddp-rnic-interop-00.txt                J. Pinkerton 
Expires: May 2005                                    Microsoft 
                                                    
                                                   November 2004 
                                      

                           RNIC Interoperability 

1  Status of this Memo 

   By submitting this Internet-Draft, I certify that any applicable 
   patent or other IPR claims of which I am aware of have been 
   disclosed, and any of which I become aware will be disclosed, in 
   accordance with RFC 3668. 

   By submitting this Internet-Draft, I accept the provisions of 
   Section 4 of RFC 3667. 

   This document is an Internet-Draft and is subject to all provisions 
   of Section 10 of RFC2026. 

   Internet-Drafts are working documents of the Internet Engineering 
   Task Force (IETF), its areas, and its working groups.  Note that 
   other groups may also distribute working documents as Internet-
   Drafts. 

   Internet-Drafts are draft documents valid for a maximum of six 
   months and may be updated, replaced, or obsoleted by other documents 
   at any time.  It is inappropriate to use Internet-Drafts as 
   reference material or to cite them other than as "work in progress." 

   The list of current Internet-Drafts can be accessed at 
   http://www.ietf.org/1id-abstracts.html  

   The list of Internet-Draft Shadow Directories can be accessed at 
   http://www.ietf.org/shadow.html.  

2  Abstract 

   This document describes a mechanism for enabling RDMA Network 
   Interface Cards (RNICs) that implement both the IETF and RDMAC 
   versions of the DDP and RDMAP protocols to interoperate with legacy 
   RNICs that are based on the RDMAC version of the protocols.  The 
   proposed scheme uses MPA Request/Reply Frames to negotiate the 
   protocol version as well as MPA marker and MPA CRC.  A ULP or other 
   supporting entity must perform this negotiation on behalf of the 
   RDMAC RNIC because RDMAC MPA does not define an MPA connection 
   scheme using MPA Request/Reply frames. This draft also makes 




Carrier & Pinkerton         Expires May 2005                          1 
Internet Draft           RNIC Interoperability            November 2004 
 

   specific recommendations for minor changes to the IETF MPA, DDP, and 
   RDMAP internet drafts to aid in interoperability. 

   Table of Contents 

   1    Status of this Memo                                          1 
   2    Abstract                                                     1 
   3    Introduction                                                 3 
   4    Proposed Modification to MPA Request/Reply Frame             5 
   5    Proposed Appendix B: IETF RNIC Interoperability with 
             RDMA Consortium Protocols                               7 
   5.1  Negotiated Parameters                                        7 
   5.2  RDMAC RNIC and Non-permissive IETF RNIC                      8 
   5.2.1     RDMAC RNIC Initiator                                    9 
   5.2.2     Non-Permissive IETF RNIC Initiator                      9 
   5.3  RDMAC RNIC and Permissive IETF RNIC                         10 
   5.3.1     RDMAC RNIC Initiator                                   10 
   5.3.2     Permissive IETF RNIC Initiator                         11 
   5.4  Non-Permissive IETF RNIC and Permissive IETF RNIC           11 
   6    Other Changes                                               12 
   6.1  Additional Changes to the MPA/DDP/RDMAP Drafts              12 
   6.2  Verbs Extensions                                            12 
   6.2.1     Changes to QueryRNIC Output:                           12 
   6.2.2     Changes to CreateQP input:                             12 
   6.2.3     Changes to CreateQP output:                            13 
   6.2.4     Changes to QueryQP Output:                             13 
   6.2.5     Changes to ModifyQP input:                             13 
   6.2.6     Changes to ModifyQP output:                            13 
   7    References                                                  14 
   7.1  Normative References                                        14 
   7.2  Informative References                                      14 
   8    Author's Addresses                                          15 
   9    Acknowledgments                                             16 
   10   Full Copyright Statement                                    18 
    

   Table of Figures 

   Figure 1. Connection Parameters for the RNIC Types.               8 
   Figure 2: MPA negotiation between an RDMAC RNIC and a Non-
             permissive IETF RNIC.                                   9 
   Figure 3: MPA negotiation between an RDMAC RNIC and a 
             Permissive IETF RNIC.                                  10 
   Figure 4: MPA negotiation between a Non-permissive IETF 
             RNIC and a Permissive IETF RNIC.                       11 
    







Carrier & Pinkerton         Expires May 2005                          2 
Internet Draft           RNIC Interoperability            November 2004 
 

3  Introduction 

   As products based on the IETF MPA, DDP and RDMAP protocols come to 
   market, they will encounter early RNIC implementations based on the 
   RDMAC versions of these protocols.  Although the IETF does not 
   specify how to interoperate with non-IETF protocols, this draft 
   describes a suggested framework for ensuring the widest range of 
   interoperability with earlier RNIC implementations, especially when 
   an RNIC can select between the RDMAC and IETF versions of the 
   protocols on a per connection basis.  

   Of the three iWARP protocols, DDP and RDMAP have changed the least 
   since the RDMAC author teams first submitted the wire protocols to 
   the RDDP WG.   The IETF and RDMAC protocols have the same semantics 
   and wire protocol format, but have different version numbers.  The 
   RDMAC uses version of zero [RDMACDDP, RDMACRDMAP] and the IETF uses 
   version of one [IETFDDP, IETFRDMAP]. Additionally, the IETF versions 
   have a more thorough analysis of security issues [RDMASEC], which 
   drove normative statements into the [DDP] and [RDMAP] drafts. 
   However, the only semantic changes of the [DDP] and [RDMAP] security 
   sections was to make IPSec support normative. Thus the IETF and 
   RDMAC versions should be able to interoperate, except for the minor 
   issues outlined in this specification. 

   The DDP/RDMAP specifications describe that the version number should 
   be used to validate the received DDP segment.  They do not, however, 
   state what version numbers are valid.  Some RNIC implementations may 
   recognize the similarity between the IETF and RDMAC protocols and 
   permissively allow either version one or zero; others may strictly 
   require that the version number received match the version number 
   sent.   Since  the specifications do not describe the version check, 
   both implementations are valid and there is no way to predict how an 
   IETF RNIC will interoperate with an RDMAC peer.   

   MPA has had the most significant changes since it was first 
   submitted to the IETF.  The IETF protocol [IETFMPA] makes markers 
   and CRC negotiable, whereas they are mandatory in the RDMAC version 
   [RDMACMPA].  The IETF MPA also includes an exchange of start-up 
   messages to ensure that both ends of the connection agree on the use 
   of MPA markers and CRC before the connection is converted to RDMA 
   mode. 

   One approach that has been suggested is to specify interoperation 
   between RDMAC and IETF RNICs by exchanging MPA Request/Reply frames 
   on behalf of RDMAC RNICs.  Even if the applications can negotiate 
   the correct parameters for the RDMAC RNIC (markers and CRC enabled), 
   there is still the issue of the DDP/RDMAP version numbers that must 
   be resolved.  Without this resolution the interoperability of the 
   end nodes cannot be determined until after the connection converts 




Carrier & Pinkerton         Expires May 2005                          3 
Internet Draft           RNIC Interoperability            November 2004 
 

   to RDMA mode and the end nodes begin interpreting each others DDP 
   segments. 

   The purpose of this draft is to describe an information exchange at 
   the level of MPA to enable RNIC peers to negotiate a common protocol 
   implementation using the MPA Request/Reply message exchange.   The 
   draft also proposes that DDP and RDMAP report their capability to 
   support RDMAC as well as IETF wire protocols to ULPs so that the 
   ULPs can configure the connection appropriately. 












































Carrier & Pinkerton         Expires May 2005                          4 
Internet Draft           RNIC Interoperability            November 2004 
 

4  Proposed Modification to MPA Request/Reply Frame 

   It is expected that RNICs will be available in three flavors:  

      RDMAC RNICs -- implementations that require MPA markers and CRCs 
          and use version zero (0) for DDP and RDMAP. 

      IETF RNICs -- implementations that negotiate MPA markers and CRCs 
          and use version one (1) for DDP and RDMAP. 

      RDMAC/IETF RNICs -- implementations that negotiate MPA markers 
          and CRCs, but can use either version zero (0) or version one 
          (1) for DDP and RDMAP depending on the ULP request. 

   As mentioned above, using the current specifications, 
   interoperability between strict IETF RNICs and RDMAC RNICs is 
   undefined.   Even if a ULP or other supporting entity exchanges the 
   MPA Request/Reply messages on behalf of the RDMAC RNIC, the success 
   or the failure of the RDMA connection will depend on the specific 
   RNIC implementation and will not be known until after both endpoints 
   convert from streaming to RDMA mode. 

   If an IETF RDMAP/DDP RNIC can be downgraded dynamically to an RDMAC 
   mode (i.e. use version 0 DDP/RDMAP), then the success or failure of 
   the connection could be determined during the MPA exchange with a 
   simple modification of the REV field in the MPA Request/Reply 
   Frames.   

   The IETF MPA protocol defines the format of the MPA Request/Reply 
   Frames in section 6.1.1 of [IETFMPA].  The following is the current 
   definition of the Rev field: 

      "Rev:     This field contains the Revision of MPA.  For this 
          version of the specification senders MUST set this field to 
          zero.  MPA receivers compliant with this version of the 
          specification MUST check this field for zero, and close the 
          connection and report an error locally if any other value is 
          detected." 

   There are two changes to this definition required to enable protocol 
   version negotiation: 

      1.  "The sender must set this field to one."  

      Since the IETF DDP and RDMAP protocols have version of one, then 
      the IETF MPA protocol should also have the version of one, 
      instead of zero.  Changing the MPA version number would allow the 
      ULPs or other supporting entities sending MPA Request/Reply 





Carrier & Pinkerton         Expires May 2005                          5 
Internet Draft           RNIC Interoperability            November 2004 
 

      Frames on behalf of RDMAC RNICs to set this value to zero, which 
      is also the version of the RDMAC DDP and RDMAP protocols. 

      2.  "If the receiver cannot interoperate with the received 
          version, it MUST gracefully close the connection and report 
          an error locally.  If the receiver can interoperate with the 
          received version, then MPA should report the negotiated 
          version to the ULP."  

      The description of the version validation must change to allow 
      receivers capable of altering their DDP and RDMAP version number 
      to accommodate both the RDMAC and IETF versions of the protocols.  
      In other words, whether or not MPA closes the connection depends 
      on whether an RNIC implements a strict interpretation of the IETF 
      protocol or a more permissive interpretation that can downgrade 
      the protocol version number on a per connection basis. 

   The following is the new Rev definition for the MPA Request/Reply 
   Frame that includes the two proposed changes above: 

      "Rev:     This field contains the Revision of MPA.  For this 
          version of the specification senders MUST set this field to 
          one.  MPA receivers compliant with this version of the 
          specification MUST check this field.  If the MPA receiver 
          cannot interoperate with the received version, then it MUST 
          close the connection and report an error locally.  Otherwise, 
          the MPA receiver should report the received version to the 
          ULP." 

   If the above definition is acceptable, then the following section 
   should be included as an appendix to the MPA specification. 






















Carrier & Pinkerton         Expires May 2005                          6 
Internet Draft           RNIC Interoperability            November 2004 
 

5  Proposed Appendix B: IETF RNIC Interoperability with RDMA Consortium 
   Protocols 

   This appendix is Informative. 

   Without the exchange of MPA Request/Reply Frames, there is no 
   standard mechanism for enabling RDMAC RNICs to interoperate with 
   IETF RNICs.  Even if a ULP uses a well-known port to start an IETF 
   RNIC immediately in RDMA mode (i.e., without exchanging the MPA 
   Request/Reply messages), there is no reason to believe an IETF RNIC 
   will interoperate with an RDMAC RNIC because of the differences in 
   the version number in the DDP and RDMAP headers on the wire. 

   Therefore, the ULP or other supporting entity at the RDMAC RNIC must 
   implement MPA Request/Reply Frames on behalf of the RNIC in order to 
   negotiate the connection parameters.  The following section 
   describes the results following the exchange of the MPA 
   Request/Reply Frames before the conversion from streaming to RDMA 
   mode.   

5.1 Negotiated Parameters 

   Three types of RNICs are considered: 

   Upgraded RDMAC RNIC - an RNIC implementing the RDMAC protocols which 
       has a ULP or other supporting entity that exchanges the MPA 
       Request/Reply Frames in streaming mode before the conversion to 
       RDMA mode.  

   Non-permissive IETF RNIC - an RNIC implementing the IETF protocols 
       which is not capable of implementing the RDMAC protocols.  Such 
       an RNIC can only interoperate with other IETF RNICs. 

   Permissive IETF RNIC - an RNIC implementing the IETF protocols which 
       is capable of implementing the RDMAC protocols on a per 
       connection basis. 

   The values used by these three RNIC types for the MPA, DDP, and 
   RDMAP versions as well as MPA markers and CRC are summarized in 
   Figure 1.  













Carrier & Pinkerton         Expires May 2005                          7 
Internet Draft           RNIC Interoperability            November 2004 
 

  +----------------++-----------+-----------+-----------+-----------+ 
  | RNIC TYPE      || DDP/RDMAP |    MPA    |    MPA    |    MPA    | 
  |                ||  Version  | Revision  |  Markers  |    CRC    | 
  +----------------++-----------+-----------+-----------+-----------+ 
  +----------------++-----------+-----------+-----------+-----------+ 
  | RDMAC          ||     0     |     0     |     1     |     1     | 
  |                ||           |           |           |           | 
  +----------------++-----------+-----------+-----------+-----------+ 
  | IETF           ||     1     |     1     |  0 or 1   |  0 or 1   | 
  | Non-permissive ||           |           |           |           | 
  +----------------++-----------+-----------+-----------+-----------+ 
  | IETF           ||  1 or 0   |  1 or 0   |  0 or 1   |  0 or 1   | 
  | permissive     ||           |           |           |           | 
  +----------------++-----------+-----------+-----------+-----------+ 
          Figure 1. Connection Parameters for the RNIC Types. 
            For MPA markers and MPA CRC, enabled=1, disabled=0. 

   It is assumed there is no mixing of versions allowed between DDP and 
   RDMAP.  The RNIC either generates the RDMAC protocols on the wire 
   (version is zero) or the IETF protocols (version is one). 

   During the exchange of the MPA Request/Reply Frames, each peer 
   provides its MPA Revision, Marker preference (M: 0=disabled, 
   1=enabled), and CRC preference.  The MPA Revision provided in the 
   MPA Request Frame and the MPA Reply Frame may differ. 

   From the information in the MPA Request/Reply Frames, each side sets 
   the Version field (V: 0=RDMAC, 1=IETF) of the DDP/RDMAP protocols as 
   well as the state of the Markers for each half connection.  Between 
   DDP and RDMAP, no mixing of versions is allowed. Moreover, the DDP 
   and RDMAP version MUST be identical in the two directions.  The RNIC 
   either generates the RDMAC protocols on the wire (version is zero) 
   or the IETF protocols (version is one).    

   In the following sections, the figures do not discuss CRC 
   negotiation because there is no interoperability issue for CRCs.  
   Since the RDMAC RNIC will always request CRC use, then, according to 
   the IETF MPA specification, both peers MUST generate and check CRCs. 

5.2 RDMAC RNIC and Non-permissive IETF RNIC 

   Figure 2 shows that a Non-permissive IETF RNIC cannot interoperate 
   with an RDMAC RNIC, despite the fact that both peers exchange MPA 
   Request/Reply Frames.  For a Non-permissive IETF RNIC, the MPA 
   negotiation has no effect on the DDP/RDMAP version and it is unable 
   to interoperate with the RDMAC RNIC.  The Non-permissive IETF RNIC 
   should gracefully close the connection when it receives an MPA 
   Request/Reply Frame with the Rev field equal to zero. 





Carrier & Pinkerton         Expires May 2005                          8 
Internet Draft           RNIC Interoperability            November 2004 
 

   The rows in the figure show the state of the Marker field in the MPA 
   Request Frame sent by the MPA Initiator.  The columns show the state 
   of the Marker field in the MPA Reply Frame sent by the MPA 
   Responder.  Each type of RNIC is shown as an initiator and a 
   responder.  The connection results are shown in the lower right 
   corner, at the intersection of the different RNIC types, where V=0 
   is the RDMAC DDP/RDMAP version, V=1 is the IETF DDP/RDMAC version, 
   M=0 means MPA markers are disabled and M=1 means MPA markers are 
   enabled. The negotiated marker state is shown as X/Y, for the 
   receive direction of the initiator/responder. 

         +---------------------------++-----------------------+ 
         |   MPA                     ||          MPA          | 
         | CONNECT                   ||       Responder       | 
         |   MODE  +-----------------++-------+---------------+ 
         |         |   RNIC          || RDMAC |     IETF      | 
         |         |   TYPE          ||       | Non-permissive| 
         |         |          +------++-------+-------+-------+ 
         |         |          |MARKER|| M=1   | M=0   |  M=1  | 
         +---------+----------+------++-------+-------+-------+ 
         +---------+----------+------++-------+-------+-------+ 
         |         |   RDMAC  | M=1  || V=0   | close | close | 
         |         |          |      || M=1/1 |       |       | 
         |         +----------+------++-------+-------+-------+ 
         |   MPA   |          | M=0  || close | V=1   | V=1   | 
         |Initiator|   IETF   |      ||       | M=0/0 | M=0/1 | 
         |         |Non-perms.+------++-------+-------+-------+ 
         |         |          | M=1  || close | V=1   | V=1   | 
         |         |          |      ||       | M=1/0 | M=1/1 | 
         +---------+----------+------++-------+-------+-------+ 
  Figure 2: MPA negotiation between an RDMAC RNIC and a Non-permissive 
                               IETF RNIC. 

5.2.1   RDMAC RNIC Initiator 

   If the RDMAC RNIC is the MPA Initiator, its ULP sends an MPA Request 
   Frame with Rev field set to zero and the M and C bits set to one.  
   Because the Non-permissive IETF RNIC cannot dynamically downgrade 
   the version number it uses for DDP and RDMAP, it would send an MPA 
   Reply Frame with the Rev field equal to one and then gracefully 
   close the connection.   

5.2.2   Non-Permissive IETF RNIC Initiator 

   If the Non-permissive IETF RNIC is the MPA Initiator, it sends an 
   MPA Request Frame with Rev field equal to one.  The ULP or 
   supporting entity for the RDMAC RNIC responds with an MPA Reply 
   Frame that has the Rev field equal to zero and the M bit set to one.  





Carrier & Pinkerton         Expires May 2005                          9 
Internet Draft           RNIC Interoperability            November 2004 
 

   The Non-permissive IETF RNIC will gracefully close the connection 
   after it reads the incompatible Rev field in the MPA Reply Frame. 

5.3 RDMAC RNIC and Permissive IETF RNIC 

   Figure 3 shows that a Permissive IETF RNIC can interoperate with an 
   RDMAC RNIC regardless of its Marker preference.  The figure uses the 
   same format as shown with the Non-permissive IETF RNIC.   

         +---------------------------++-----------------------+ 
         |   MPA                     ||          MPA          | 
         | CONNECT                   ||       Responder       | 
         |   MODE  +-----------------++-------+---------------+ 
         |         |   RNIC          || RDMAC |     IETF      | 
         |         |   TYPE          ||       |  Permissive   | 
         |         |          +------++-------+-------+-------+ 
         |         |          |MARKER|| M=1   | M=0   | M=1   | 
         +---------+----------+------++-------+-------+-------+ 
         +---------+----------+------++-------+-------+-------+ 
         |         |   RDMAC  | M=1  || V=0   | N/A   | V=0   | 
         |         |          |      || M=1/1 |       | M=1/1 | 
         |         +----------+------++-------+-------+-------+ 
         |   MPA   |          | M=0  || V=0   | V=1   | V=1   | 
         |Initiator|   IETF   |      || M=1/1 | M=0/0 | M=0/1 | 
         |         |Permissive+------++-------+-------+-------+ 
         |         |          | M=1  || V=0   | V=1   | V=1   | 
         |         |          |      || M=1/1 | M=1/0 | M=1/1 | 
         +---------+----------+------++-------+-------+-------+ 
 Figure 3: MPA negotiation between an RDMAC RNIC and a Permissive IETF 
                                 RNIC. 

   A truly Permissive IETF RNIC will recognize an RDMAC RNIC from the 
   Rev field of the MPA Request/Reply Frames and then adjust its 
   receive Marker state and DDP/RDMAP version to accommodate the RDMAC 
   RNIC.  As a result, as an MPA Responder, the Permissive IETF RNIC 
   will never return an MPA Reply Frame with the M bit set to zero.  
   This case is shown as a not applicable (N/A) in Figure 3. 

5.3.1   RDMAC RNIC Initiator 

   When the RDMAC RNIC is the MPA Initiator, its ULP or other 
   supporting entity prepares an MPA Request message and sets the 
   revision to zero and the M bit and C bit to one.   

   The Permissive IETF Responder receives the MPA Request message and 
   checks the revision field.  Since it is capable of generating RDMAC 
   DDP/RDMAP headers, it sends an MPA Reply message with revision set 
   to zero and the M and C bits set to one.  The Responder must inform 
   its ULP that it is generating version zero DDP/RDMAP messages.  




Carrier & Pinkerton         Expires May 2005                         10 
Internet Draft           RNIC Interoperability            November 2004 
 

5.3.2   Permissive IETF RNIC Initiator 

   If the Permissive IETF RNIC is the MPA Initiator, it prepares the 
   MPA Request Frame setting the Rev field to one.  Regardless of the 
   value of the M bit in the MPA Request Frame, the ULP or other 
   supporting entity for the RDMAC RNIC will create an MPA Reply Frame 
   with Rev equal to zero and the M bit set to one.  

   When the Initiator reads the Rev field of the MPA Reply Frame and 
   finds that its peer is an RDMAC RNIC, it must inform its ULP that it 
   should generate version zero DDP/RDMAP messages and enable MPA 
   markers and CRC. 

5.4 Non-Permissive IETF RNIC and Permissive IETF RNIC 

   For completeness, Figure 4 shows the results of MPA negotiation 
   between a Non-permissive IETF RNIC and a Permissive IETF RNIC.  The 
   important point from this figure is that an IETF RNIC cannot detect 
   whether its peer is a Permissive or Non-permissive RNIC.  

     +---------------------------++-------------------------------+ 
     |   MPA                     ||              MPA              | 
     | CONNECT                   ||            Responder          | 
     |   MODE  +-----------------++---------------+---------------+ 
     |         |   RNIC          ||     IETF      |     IETF      | 
     |         |   TYPE          || Non-permissive|  Permissive   | 
     |         |          +------++-------+-------+-------+-------+ 
     |         |          |MARKER|| M=0   | M=1   | M=0   | M=1   | 
     +---------+----------+------++-------+-------+-------+-------+ 
     +---------+----------+------++-------+-------+-------+-------+ 
     |         |          | M=0  || V=1   | V=1   | V=1   | V=1   | 
     |         |   IETF   |      || M=0/0 | M=0/1 | M=0/0 | M=0/1 | 
     |         |Non-perms.+------++-------+-------+-------+-------+ 
     |         |          | M=1  || V=1   | V=1   | V=1   | V=1   | 
     |         |          |      || M=1/0 | M=1/1 | M=1/0 | M=1/1 | 
     |   MPA   +----------+------++-------+-------+-------+-------+ 
     |Initiator|          | M=0  || V=1   | V=1   | V=1   | V=1   | 
     |         |   IETF   |      || M=0/0 | M=0/1 | M=0/0 | M=0/1 | 
     |         |Permissive+------++-------+-------+-------+-------+ 
     |         |          | M=1  || V=1   | V=1   | V=1   | V=1   | 
     |         |          |      || M=1/0 | M=1/1 | M=1/0 | M=1/1 | 
     +---------+----------+------++-------+-------+-------+-------+ 
   Figure 4: MPA negotiation between a Non-permissive IETF RNIC and a 
                         Permissive IETF RNIC. 

    







Carrier & Pinkerton         Expires May 2005                         11 
Internet Draft           RNIC Interoperability            November 2004 
 

6  Other Changes 

6.1 Additional Changes to the MPA/DDP/RDMAP Drafts 

   The RDMAP protocol needs to inform the consumer of the RDMA data 
   stream of the results of the version negotiation.  This is not 
   required per the current IETF MPA/DDP/RDMAP drafts. Thus the drafts 
   would need to be changed to state that  MPA must pass the negotiated 
   version to DDP, DDP must pass the version to RDMAP and RDMAP must 
   make the version available to the ULP.   

   For example, the following text is a suggested addition to [IETFMPA] 
   section 3.2, "MPA's Interaction with DDP" : 

      "MPA MUST provide the protocol version negotiated with its peer 
      to DDP.  DDP will use this version to set the version in its 
      header and to report the version to RDMAP." 

6.2 Verbs Extensions 

   Most RNIC implementations include software to provide a Verbs 
   interface [VERBS] to the host operating environment, even though 
   Verbs are outside the scope of IETF protocols.  The following 
   extensions to Verbs are necessary to allow Verbs consumers to be 
   active participants in RNIC interoperability. 

   Note that some implementations will reserve RNIC resources before 
   exchange of the MPA Request/Reply Messages. Other implementations 
   will create the QP after the MPA Request/Reply Messages. Thus both 
   CreateQP and ModifyQP need to be changed to enable both approaches. 

6.2.1   Changes to QueryRNIC Output: 

      - MPA Marker Receive preference 

      - DDP/RDMAP version number 

      - DDP/RDMAP version selectable on a per QP basis 

6.2.2   Changes to CreateQP input: 

      - Set MPA Marker for receive/transmit 

      - Set DDP and RDMAP version number {0, 1} 









Carrier & Pinkerton         Expires May 2005                         12 
Internet Draft           RNIC Interoperability            November 2004 
 

6.2.3   Changes to CreateQP output: 

      - New error code stating version number not supported. 

      - New error code stating disabling markers not supported. 

6.2.4   Changes to QueryQP Output: 

      - MPA Marker selection for receive and/or transmit 

      - DDP/RDMAP version number 

6.2.5   Changes to ModifyQP input: 

      Only on transition from Idle to Idle, or Idle to RTS. 

      - Set MPA Marker for receive/transmit 

      - Set DDP and RDMAP version number {0, 1} 

6.2.6   Changes to ModifyQP output: 

      - New error code stating version number not supported. 

      - New error code stating disabling markers not supported. 

       


























Carrier & Pinkerton         Expires May 2005                         13 
Internet Draft           RNIC Interoperability            November 2004 
 

7  References 

7.1 Normative References  

   [RFC2026] Bradner, S., "The Internet Standards Process -- Revision 
       3", BCP 9, RFC 2026, October 1996. 

   [RFC3667] Bradner, S., "IETF Rights in Contributions", BCP 78, RFC 
       3667, February 2004. 

   [RFC3668] Bradner, S., Ed., "Intellectual Property Rights in IETF 
       Technology", BCP 79, RFC 3668, February 2004. 

   [IETFMPA] Culley, P., Elzur, U., Recio, R., Bailey, S., Carrier, J., 
       "Marker PDU Aligned Framing for TCP Specification", Internet 
       Draft draft-ietf-rddp-mpa-01.txt, July 2004. 

   [IETFDDP] Shah, H., J. Pinkerton, R.Recio, and P. Culley, "Direct 
       Data Placement over Reliable Transports", Internet-Draft draft-
       ietf-rddp-ddp-02.txt, February 2004.  

   [IETFRDMAP] Recio, R., P. Culley, D. Garcia, J. Hilland, "An RDMA 
       Protocol Specification", Internet-Draft draft-ietf-rddp-rdmap-
       01.txt, October 2003.  

   [RDMASEC]  Pinkerton J., Deleganes E., Romanow A., Bitan S., 
       "DDP/RDMAP Security", draft-ietf-rddp-security-01.txt (work in 
       progress), February 2004. 

   [IPSEC]  Atkinson, R., Kent, S., "Security Architecture for the 
       Internet Protocol", RFC 2401, November 1998. 

7.2 Informative References  

   [RDMACMPA] P. Culley et al., "Marker PDU Aligned Framing for TCP 
       Specification", RDMA Consortium Draft Specification draft-
       culley-iwarp-mpa-v1.0, October 2002 

   [RDMACDDP] H. Shah et al., "Direct Data Placement over Reliable 
       Transports", RDMA Consortium Draft Specification draft-
       shahiwarp-ddp-v1.0, October 2002 

   [RDMACRDMAP] R. Recio et al., "An RDMA Protocol Specification", RDMA 
       Consortium Draft Specification draft-recio-iwarp-rdmap-v1.0, 
       October 2002 

   [VERBS] J. Hilland et al., "RDMA Protocol Verbs Specification", 
       RDMAC Consortium Draft Specification draft-hilland-iwarp-verbs-
       v1.0-RDMAC, April 2003 




Carrier & Pinkerton         Expires May 2005                         14 
Internet Draft           RNIC Interoperability            November 2004 
 

8  Author's Addresses 

   John Carrier 
       Adaptec Inc. 
       691 South Milpitas Blvd. 
       Milpitas, CA 95035 USA 
       Phone: (360) 378-8526 
       Email: john_carrier@adaptec.com 

   Jim Pinkerton 
       Microsoft, Inc. 
       One Microsoft Way 
       Redmond, WA USA 98052 USA 
       Phone:  
       Email: jpink@microsoft.com 






































Carrier & Pinkerton         Expires May 2005                         15 
Internet Draft           RNIC Interoperability            November 2004 
 

9  Acknowledgments 

   Caitlin Bestler 
       Siliquent Technologies 
       1300 Crittenden, Suite 201 
       Mountain View, CA 94043 USA 
       Phone: +1 (650) 962-1632 ext 100 
       Email: caitlinb@siliquent.com 

   Paul R. Culley 
       Hewlett-Packard Company 
       20555 SH 249 
       Houston, TX 77070 USA 
       Phone: +1 (281) 514-5543 
       Email: paul.culley@hp.com 

   Uri Elzur 
       Broadcom 
       16215 Alton Parkway 
       CA 92618 USA 
       Phone: +1 (949) 585-6432 
       Email: uri@broadcom.com 

   Fredy Neeser  
       IBM Zurich Research Laboratory 
       Saeumerstrasse 4 
       CH-8803 Rueschlikon, Switzerland 
       Phone: +41 (0)1 724 8487 
       Email: nfd@zurich.ibm.com 

   Renato J Recio 
       IBM 
       Internal Zip 9043 
       11400 Burnett Road 
       Austin, TX  78759 USA 
       Phone: +1 (512) 838-3685 
       Email: recio@us.ibm.com 

   Barry Reinhold 
       Lamprey Networks 
       PO Box 539 
       Durham, NH  03824 USA 
       Phone: +1 (603) 868-8411 
       Email: bbr@lampreynetworks.com 









Carrier & Pinkerton         Expires May 2005                         16 
Internet Draft           RNIC Interoperability            November 2004 
 

   Tom Talpey 
       Network Appliance 
       375 Totten Pond Road 
       Waltham, MA 02451 USA 
       Phone: +1 (781) 768-5329 
       EMail: thomas.talpey@netapp.com 

   Patricia Thaler 
       Agilent Technologies, Inc. 
       1101 Creekside Ridge Drive, #100  
       M/S-RG10 
       Roseville, CA 95678 USA 
       Phone: +1 (916) 788-5662 
       email: pat_thaler@agilent.com 

    





































Carrier & Pinkerton         Expires May 2005                         17 
Internet Draft           RNIC Interoperability            November 2004 
 

10 Full Copyright Statement 

   Copyright (C) The Internet Society (2004).  This document is subject 
   to the rights, licenses and restrictions contained in BCP78, and 
   except as set forth therein, the authors retain all their rights. 

   This document and translations of it may be copied and furnished to 
   others, and derivative works that comment on or otherwise explain it 
   or assist in its implementation may be prepared, copied, published 
   and distributed, in whole or in part, without restriction of any 
   kind, provided that the above copyright notice and this paragraph 
   are included on all such copies and derivative works. However, this 
   document itself may not be modified in any way, such as by removing 
   the copyright notice or references to the Internet Society or other 
   Internet organizations, except as needed for the purpose of 
   developing Internet standards in which case the procedures for 
   copyrights defined in the Internet Standards process must be 
   followed, or as required to translate it into languages other than 
   English.  

   The limited permissions granted above are perpetual and will not be 
   revoked by the Internet Society or its successors or assigns. This 
   document and the information contained herein is provided on an "AS 
   IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK 
   FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT 
   NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN 
   WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF 
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.  

   Funding for the RFC Editor function is currently provided by the 
   Internet Society. 






















Carrier & Pinkerton         Expires May 2005                         18