Internet DRAFT - draft-elzur-iwarp-mpa-tcp-analysis
draft-elzur-iwarp-mpa-tcp-analysis
INTERNET DRAFT Uri Elzur
draft-elzur-iwarp-mpa-tcp-analysis-00.txt Broadcom
Expires: July, 2003 Bob Teisberg
Dwight Barron
Paul Culley
Hewlett-Packard
Jim Pinkerton
Microsoft
John Carrier
Adaptec
February 2003
Analysis of MPA over TCP Operations
1 Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
2 Abstract
Further explanation and analysis of architectural recommendations
contained in the recent Internet-Draft, "Marker PDU Aligned Framing
for TCP Specification" [MPA], is provided. The impact of the
following three attributes of MPA over TCP is examined:
* packing of multiple DDP ULPDUs into one TCP segment;
* simplifying the receiver due to transmitter alignment of DDP
headers with TCP headers; and
* mistakenly attempting to interoperate an MPA-enabled endpoint
with a non-MPA-enabled endpoint.
Elzur, et al. Expires - August 2003 [Page 1]
Analysis of MPA over TCP February 2003
Table of Contents
1 Status of this Memo.........................................1
2 Abstract....................................................1
3 Introduction................................................3
4 Definitions.................................................4
5 Assumptions.................................................6
5.1 MPA is layered beneath DDP [DDP]............................6
5.2 MPA preserves DDP message framing...........................6
5.3 The size of the ULPDU passed to MPA is less than EMSS under
normal conditions...........................................6
5.4 Out-of-order placement but NO out-of-order delivery.........6
6 No Packing..................................................7
7 The Value of Header Alignment..............................11
7.1 Impact of lack of Header Alignment on the receiver
computational load and complexity..........................12
7.2 Header Alignment effects on TCP wire protocol..............16
8 Interoperating between MPA applications and non-MPA
applications...............................................19
8.1 Negotiation of MPA-enabled mode............................20
8.2 Analysis of existing TCP services..........................23
8.2.1 The "Little" TCP Services................................23
8.2.2 ULPs Using Only Text Messages............................24
8.2.3 ULPs with Fixed Initial Message..........................25
8.2.4 Protocols with framed command headers....................25
9 Security Considerations....................................30
10 IANA Considerations........................................31
11 References.................................................32
11.1 Primary References.......................................32
11.2 "Little" TCP Services....................................32
11.3 ULPs using only Text Messages............................33
11.4 ULPs with Fixed Initial Message..........................34
11.5 ULPs with Framed Command Headers.........................34
12 Author's Addresses.........................................36
13 Acknowledgments............................................37
14 Full Copyright Statement...................................38
Table of Figures
Figure 1: Non-aligned FPDU freely placed in TCP octet stream....13
Figure 2: Aligned FPDU placed immediately after TCP header......15
Figure 3: MPA Enablement Error..................................19
Figure 4: MPA Transition via Well Known Port or Service Location
Protocol..............................................21
Figure 5: MPA Transition via Octet Stream Negotiation...........22
Figure 6: Effect of Improperly MPA-Enabled DNS Resolver.........27
Elzur, et al. Expires - August 2003 [Page 2]
Analysis of MPA over TCP February 2003
3 Introduction
This paper analyzes the impact of MPA (Marker PDU Aligned Framing
for TCP [MPA]) on the TCP sender, receiver, and wire protocol.
One of MPA's high level goals is to provide enough information, when
combined with the Direct Data Placement Protocol [DDP], to enable
out-of-order placement of DDP payload into the final Upper Layer
Protocol (ULP) buffer. Note that DDP separates the act of placing
data into a ULP buffer from that of notifying the ULP that the ULP
buffer is available for use. In DDP terminology, the former is
defined as "Placement", and the later is defined as "Delivery". MPA
supports in-order delivery of the data to the ULP, including support
for direct data placementin the final ULP buffer location when TCP
segments arrive out-of-order. Effectively, the goal is to use the
pre-posted ULP buffers as the TCP receive buffer, where the
reassembly of the ULP Protocol Data Unit (PDU) by TCP (with MPA and
DDP) is done in place, in the ULP buffer, with no data copies.
The paper walks through the advantages and disadvantages of the two
main TCP sender modifications proposed by MPA:
1) that MPA require the TCP sender to do "Header Alignment", where a
TCP segment is required to begin with an MPA Framing Protocol Data
Unit (FPDU) (if there is payload present) and that there be an
integral number of FPDUs in a TCP segment (under conditions where
the Path MTU is not changing).
2) that MPA require "no packing" of FPDUs -- i.e. exactly zero or
one FPDUs are present in a single TCP segment.
The paper concludes that the worst case analysis for "no packing" is
bad enough that it out-weighs the advantages and should be removed
from the MPA specification.
The paper also concludes that the scaling advantages of Header
Alignment are strong, based primarily on fairly drastic TCP receive
buffer reduction requirements and simplified receive handling. The
analysis also shows that there is little effect to TCP wire
behavior.
Finally, the paper examines interoperability issues between an
unmodified TCP stack and a modified TCP stack, for a wide variety of
applications and a wide variety of combinations.
Elzur, et al. Expires - August 2003 [Page 3]
Analysis of MPA over TCP February 2003
4 Definitions
DDP - Direct Data Placement Protocol [DDP]
Data Delivery (Delivery, Delivered, Delivers) - Delivery is defined
as the process of informing the ULP or consumer that a
particular Message is available for use. This is specifically
different from "Placement", which may generally occur in any
order, while the order of "Delivery" is strictly defined. See
"Data Placement".
Data Placement (Placement, Placed, Places) - For DDP, this term is
specifically used to indicate the process of writing to a data
buffer by a DDP implementation. DDP Segments carry Placement
information, which may be used by the receiving DDP
implementation to perform Data Placement of the DDP Segment ULP
Payload. See "Data Delivery".
EMSS - Effective Maximum Segment Size. EMSS is the smaller of the
TCP maximum segment size (MSS) [RFC0793], and the current Path
Maximum Transfer Unit (PMTU) [RFC1191].
FPDU - Framing Protocol Data Unit. The unit of data created by a
ULP utilizing the MPA framing protocol. A complete MPA FPDU
includes the MPA length, MPA payload, MPA CRC and potentially
Markers as appropriate.
Header Alignment - the property that a TCP segment begins with an
FPDU and the TCP segment includes an integer number of FPDUs.
MPA - the protocol defined by the "Marker PDU Aligned Framing for
TCP Specification" [MPA].
MPA-aware TCP - a TCP implementation that is aware of the receiver
efficiencies of MPA Header Alignment and is capable of sending
TCP segments that begin with an FPDU.
MPA-enabled - MPA is enabled if the MPA protocol is visible on the
wire. When the sender is MPA-enabled, it is inserting framing
and markers. When the receiver is MPA-enabled, it is
interpreting framing and markers.
MULPDU - Maximum ULPDU. The current maximum size of the record that
is acceptable for DDP to pass to MPA for transmission.
PDU - protocol data unit
Elzur, et al. Expires - August 2003 [Page 4]
Analysis of MPA over TCP February 2003
ULP - Upper Layer Protocol. The protocol layer above the protocol
layer currently being referenced. The ULP for MPA is DDP [DDP].
ULPs may be classified as Passive if they are awaiting a
connection request or Active if they are initiating a connection
request.
ULPDU - Upper Layer Protocol Data Unit. The data record defined by
the layer above MPA (DDP). ULPDU corresponds to DDP's "DDP
Segment".
Elzur, et al. Expires - August 2003 [Page 5]
Analysis of MPA over TCP February 2003
5 Assumptions
5.1 MPA is layered beneath DDP [DDP]
MPA is an adaptation layer between DDP and TCP. DDP requires
preservation of DDP segment boundaries and a CRC32C digest covering
the DDP header and data. MPA adds these features to the TCP stream
so that DDP over TCP has the same basic properties as DDP over SCTP.
5.2 MPA preserves DDP message framing
MPA was designed as a framing layer specifically for DDP and was not
intended as a general-purpose framing layer for any other ULP using
TCP.
A framing layer allows ULPs using it to receive indications from the
transport layer only when complete ULPDUs are present. As a framing
layer, MPA is not aware of the content of the DDP PDU, only that it
has received and, if necessary, reassembled a complete PDU for
delivery to the DDP.
5.3 The size of the ULPDU passed to MPA is less than EMSS under normal
conditions
To make reception of a complete DDP PDU on every received segment
possible, DDP passes to MPA a PDU that is no larger than the EMSS of
the underlying fabric. Each FPDU that MPA creates contains
sufficient information for the receiver to directly place the ULP
payload in the correct location in the correct receive buffer.
Edge cases when this condition does not occur are dealt with, but do
not need to be on the fast path
5.4 Out-of-order placement but NO out-of-order delivery
DDP receives complete DDP PDUs from MPA. Each DDP PDU contains the
information necessary to place its ULP payload directly in the
correct location in host memory.
Because each DDP segment is self-describing, it is possible for DDP
segments received out of order to have their ULP payload placed
immediately in the ULP receive buffer.
Data delivery to the ULP is guaranteed to be in the order the data
was sent. DDP only indicates data delivery to the ULP after TCP has
acknowledged the complete byte stream.
Elzur, et al. Expires - August 2003 [Page 6]
Analysis of MPA over TCP February 2003
6 No Packing
MPA as originally proposed requires an MPA-aware TCP sender to
segment the datastream in such a way that each TCP segment contains
a single FPDU. This requirement is referred to in this document as
the "no packing" rule. Let us examine the costs and benefits of the
"no packing" rule.
The Header Alignment rule means that Placement information is
guaranteed to immediately follow the TCP header in the typical case
(See Section 7, The Value of Header Alignment, page 11, for the
analysis of the value of Header Alignment). If the "no packing" rule
is in effect, it further guarantees (in the typical case) that
because no additional FPDUs will be in the TCP payload, the receiver
does not have to look for additional placement information within
the TCP payload. This allows the receiver logic to be simplified.
For instance, the receiver logic needs to support one context lookup
per frame and one data movement operation per frame. Only in the
instance of a PMTU change is it necessary to examine the remainder
of the TCP payload for DDP headers. Because PMTU changes are
presumed to be rare, the latter case can be delegated to a "slow
path" processing mode, while the "fast path" for the common case can
be extremely simple.
The original argument for "no packing" also examined typical ULP
behavior for applications expected to see strong advantages from
Direct Data Placement -- specifically transaction based applications
or throughput oriented applications. Request/response protocols
typically send one FPDU per TCP segment. A response may be short or
quite long, but in any case would fill all TCP segments up to the
last one, providing TCP segmentation behavior similar to an
unmodified TCP stack. A similar argument applies to ULPs optimized
for throughput, which send long, uninterrupted sequences of PMTU-
sized FPDUs.
Thus for many applications the rule has no effect on TCP
segmentation, and it enabled simplified receive logic because the
receiver did not have to peak into the TCP segment at some arbitrary
offset to find the MPA/DDP headers.
On the other hand, a ULP which sends long sequences of small FPDUs
is strongly affected by the "no packing" rule.
Several specific consequences of the "no packing" rule deserve
detailed discussion.
The "no packing" rule tends to increase the total number of TCP
segments transmitted. In the best case, as noted earlier, the
number of segments is unchanged. In the worst case, where all
Elzur, et al. Expires - August 2003 [Page 7]
Analysis of MPA over TCP February 2003
ULPDUs are 1 octet long, the number of segments increases by dozens
of times. We will show the penalty for the "no packing" rule for
networks using the full Ethernet frames or the smaller default IP
frame size.
For both calculations, the Minimum FPDU Size (MinFPDUSize) for the
worst case ULPDU is the sum of the MPA Header Size (MPAHdrSize), the
DDP Header Size (DDPHdrSize), the worst case DDP Payload Size
(DDPPayLd), the number of MPA Pad octets (MPAPad) needed to make the
FPDU size a multiple of 4, and the MPA CRC size (MPACRC):
MPAHdrSize = 2 octets
DDPHdrSize = 14 octets
DDPPayld = 1 octet
MPAPad = 3 octets
MPACRC = 4 octets
MinFPDUSize = MPAHdrSize + DDPHdrSize + DDPPayld + MPAPad
+ MPACRC
= 2 + 14 + 1 + 3 + 4
= 24
1. Ethernet frames
a. The expected Number of Markers in an Ethernet frame
(EthNMarkers) is calculated by dividing the MPA Marker
Interval (MPAMrkIntvl) into the Ethernet Frame Size
(EthFrmSize):
MPAMrkIntvl = 512 octets
EthFrmSize = 1460 octets of Ethernet payload
EthNMarkers =~ EthFrmSize / MPAMrkIntvl
=~ 2.9
Elzur, et al. Expires - August 2003 [Page 8]
Analysis of MPA over TCP February 2003
b. The expansion of FPDUs into an Ethernet frame (EthExpansion)
is the number of times MinFPDUSize octets can be put into an
Ethernet Frame (EthFrmSize) after removing the number of
octets consumed by markers (MPAMrkSize * EthNMarkers) in the
frame:
MPAMrkSize = 4 octets
EthExpansion = (EthFrmSize - MPAMrkSize * EthNMarkers) /
MinFPDUSize
=~ (1460 - 4 * 2.9) / 24
=~ 60
2. Default IP packets
a. The expected Number of Markers in the Default IP packet
(DefNMarkers) is calculated by dividing the MPA Marker
Interval (MPAMrkIntvl) into the Default IP Packet Size
(DefPktSize):
DefPktSize = 536 octets
DefNMarkers = DefPktSize / MPAMrkIntvl
=~ 1.05
b. The expansion of FPDUs into the Default IP Packet
(DefExpansion) is the number of times MinFPDUSize octets can
be put into an IP Packet (DefPktSize) after removing the
number of octets consumed by markers (MPAMrkSize *
DefNMarkers) in the packet:
DefExpansion = (DefPktSize - MPAMrkSize * DefNMarkers) /
MinFPDUSize
=~ (536 - 4 * 1.05) / 24
=~ 22
In the worst case where all ULPDU's are one octet long, the "no
packing" rule forces the transmission of roughly 60 times as many
packets on Ethernet as MPA with packing allowed. Even with smaller
IP packets, the "no packing" rule would force using more than 20
times as many packets.
Elzur, et al. Expires - August 2003 [Page 9]
Analysis of MPA over TCP February 2003
Clearly the effect of the "no packing" rule on the number of extra
packets depends on the nature of the ULP and workload. The worst
case involves protocols that send long sequences of small ULPDUs.
The existence of protocols such as telnet [RFC0854] shows that at
least in some cases real-world applications may tend to approach the
worst case.
As a direct consequence of the increased number of data segments
transmitted, the number of TCP ACKs increases proportionally. A
series of minimum-sized ULPDUs which could have been packed into two
TCP segments on an Ethernet network, prompting a single ACK in
response, would consume 120 segments with the "no packing" rule in
effect, resulting in as many as 60 ACKs.
Dividing a datastream into a large number of small segments impairs
the efficiency of the slow start algorithm. While the number of
packets necessary to reach an efficient line utilization is the same
as in a conventional TCP implementation, the total payload
transmitted during slow start is reduced substantially. In other
words, a sender obeying the "no packing" rule could pay a
substantial performance penalty during the slow start phase if long
sequences of small ULPDUs are used.
It is clear from this analysis that the drawbacks of the "no
packing" rule are substantial, but the benefits are small. The "no
packing" requirement should be changed to state that MPA-aware TCP MAY
support packing at the transmitter and MUST support packing at the receiver.
However, as noted above, certain applications (e.g., transaction-
based applications) will prioritize minimal latency over maximum
wire efficiency. In such scenarios it is anticipated there will be
minimal opportunity for packing at the transmitter, and receivers
may choose to optimize their performance for this anticipated
behavior.
Elzur, et al. Expires - August 2003 [Page 10]
Analysis of MPA over TCP February 2003
7 The Value of Header Alignment
Significant receiver optimizations can be achieved when Header
Alignment and complete FPDUs are the common case. The optimizations
allow utilizing significantly fewer buffers on the receiver and less
computation per FPDU. The net effect is the ability to build a
"Flow-Through" receiver that enables TCP-based solutions to scale to
10G and beyond in an economical way. The optimizations are
especially relevant to hardware implementations of receivers that
process multiple protocol layers - Data Link Layer (e.g., Ethernet),
Network and Transport Layer (e.g., TCP/IP), and even some ULP on top
of TCP (e.g., MPA/DDP). As network speed increases, there is an
increasing desire to use a hardware based receiver in order to
achieve an efficient high performance solution.
A TCP receiver, under worst case conditions, has to allocate buffers
(BufferSizeTCP) whose capacities are a function of the bandwidth-
delay product. Thus:
BufferSizeTCP = K * bandwidth [octets/S] * Delay [S].
Where bandwidth is the end-to-end bandwidth of the connection, delay
is the round trip delay of the connection, and K is an
implementation dependent constant.
Thus BufferSizeTCP scales with the end-to-end bandwidth (10x more
buffers for a 10x increase in end-to-end bandwidth). As this
buffering approach may scale poorly for hardware or software
implementations alike, several approaches allow reduction in the
amount of buffering required for high-speed TCP communication.
The MPA/DDP approach is to enable the ULP's buffer to be used as the
TCP receive buffer. If the application pre-posts a sufficient amount
of buffering, and each TCP segment has sufficient information to
place the payload into the right application buffer, when an out-of-
order TCP segment arrives it could potentially be placed directly in
the ULP buffer. However, placement can only be done when a complete
FPDU with the placement information is available to the receiver,
and the FPDU contents contain enough information to place the data
into the correct ULP buffer (e.g., there is a DDP header available).
For the case when the FPDU is not aligned with the TCP segment, it
may take, on average, 2 TCP segments to assemble one FPDU.
Therefore, the receiver has to allocate BufferSizeNAF (Buffer Size,
Non-Aligned FPDU) octets:
BufferSizeNAF = K1* EMSS * number_of_connections + K2 * EMSS
Elzur, et al. Expires - August 2003 [Page 11]
Analysis of MPA over TCP February 2003
Where K1 and K2 are implementation dependent constants and EMSS is
the effective maximum segment size.
For example, a 1 Gbps link with 10,000 connections and an EMSS of
1500B would require 15 MB of memory. Often the number of connections
used scales with the network speed, aggravating the situation for
higher speeds.
A Header Aligned FPDU would allow the receiver to allocate
BufferSizeAF (Buffer Size, Aligned FPDU) octets:
BufferSizeAF = K2 * EMSS
for the same conditions. A Header Aligned receiver may require
memory in the range of ~100s of KB - which is feasible for an on-
chip memory and enables a "Flow-Through" design, in which the data
flows through the NIC and is placed directly in the destination
buffer. Assuming most of the connections support Header Alignment,
the receiver buffers no longer scale with number of connections.
Additional optimizations can be achieved in a balanced I/O sub-
system -- where the system interface of the network controller
provides ample bandwidth as compared with the network bandwidth. For
almost twenty years this has been the case and the trend is expected
to continue - while Ethernet speeds have scaled by 1000 (from 10
megabit/sec to 10 gigabit/sec), I/O bus bandwidth of volume CPU
architectures has scaled from ~2 MB/sec to ~2 GB/sec (PC-XT bus to
PCI-X DDR). Under these conditions, the Header Aligned FPDU approach
allows BufferSizeAF to be indifferent to network speed. It is
primarily a function of the local processing time for a given frame.
Thus when the Header Aligned FPDU approach is used, receive
buffering is expected to scale gracefully (i.e. less than linear
scaling) as network speed is increased.
7.1 Impact of lack of Header Alignment on the receiver computational
load and complexity
The receiver must perform IP and TCP processing, and then perform
FPDU CRC checks, before it can trust the FPDU header placement
information. For simplicity of the description, the assumption is
that a FPDU is carried in no more than 2 TCP segments. In reality,
with no Header Alignment, an FPDU can be carried by more than 2 TCP
segments (e.g., if the PMTU was reduced).
Elzur, et al. Expires - August 2003 [Page 12]
Analysis of MPA over TCP February 2003
----++-----------------------------++-----------------------++-----
+---||---------------+ +--------||--------+ +----------||----+
| TCP Seg X-1 | | TCP Seg X | | TCP Seg X+1 |
+---||---------------+ +--------||--------+ +----------||----+
----++-----------------------------++-----------------------++-----
FPDU #N-1 FPDU #N
Figure 1: Non-aligned FPDU freely placed in TCP octet stream
The receiver algorithm for processing TCP segments (e.g., TCP
segment #X in Figure 1: Non-aligned FPDU freely placed in TCP octet
stream) carrying non-aligned FPDUs (in-order or out-of-order)
includes:
1. Data Link Layer processing (whole frame) - typically including a
CRC calculation.
2. Network Layer processing (assuming not an IP fragment, the whole
Data Link Layer frame contains one IP datagram. IP fragments
should be reassembled in a local buffer. This is not a
performance optimization goal)
3. Transport Layer processing -- TCP protocol processing, header
and checksum checks.
a. Classify incoming TCP segment using the 5 tuple (IP SRC, IP
DST, TCP SRC Port, TCP DST Port, protocol)
4. Find FPDU message boundaries.
a. Get MPA state information for the connection
i. If the TCP segment is in-order, use the receiver managed
MPA state information to calculate where the previous
FPDU message (#N-1) ends in the current TCP segment X.
(previously, when the MPA receiver processed the first
part of FPDU #N-1, it calculated the number of bytes
remaining to complete FPDU #N-1 by using the MPA Length
field).
.1. Get the stored partial CRC for FPDU #N-1
.2. Complete CRC calculation for FPDU #N-1 data (first
portion of TCP segment #X)
.3. Check CRC calculation for FPDU #N-1
Elzur, et al. Expires - August 2003 [Page 13]
Analysis of MPA over TCP February 2003
.4. If no FPDU CRC errors, placement is allowed
.5. Locate the local buffer for the first portion of
FPDU#N-1, CopyData(local buffer of first portion of
FPDU #N-1, host buffer address, length)
.6. Compute host buffer address for second portion of
FPDU #N-1
.7. CopyData (local buffer of second portion of FPDU #N-
1, host buffer address for second portion, length)
.8. Calculate the octet offset into the TCP segment for
the next FPDU #N.
.9. Start Calculation of CRC for available data for FPDU
#N
.10. Store partial CRC results for FPDU #N
.11. Store local buffer address of first portion of FPDU
#N
.12. No further action is possible on FPDU #N, before it
is completely received
ii. If TCP out-of-order, receiver must buffer the data until
at least one complete FPDU is received. Typically
buffering for more than one TCP segment per connection
is required. Use the MPA based Markers to calculate
where FPDU boundaries are.
.1. When a complete FPDU is available, a similar
procedure to the in-order algorithm above is used.
There is additional complexity, though, because when
the missing segment arrives, this TCP segment must
be run through the CRC engine after the CRC is
calculated for the missing segment.
If we assume Header Alignment, the following diagram and the
algorithm below apply. Note that when using MPA, the receiver is
assumed to actively detect presence or loss of Header Alignment for
every TCP segment received.
Elzur, et al. Expires - August 2003 [Page 14]
Analysis of MPA over TCP February 2003
+--------------------------+ +--------------------------+
+--|--------------------------+ +--|--------------------------+
| | TCP Seg X | | | TCP Seg X+1 |
+--|--------------------------+ +--|--------------------------+
+--------------------------+ +--------------------------+
FPDU #N FPDU #N+1
Figure 2: Aligned FPDU placed immediately after TCP header
The receiver algorithm for Header Aligned frames (in-order or out-
of-order) includes:
1. Data Link Layer processing (whole frame) - typically including a
CRC calculation.
2. Network Layer processing (assuming not an IP fragment, the whole
Data Link Layer frame contains one IP datagram. IP fragments
should be reassembled in a local buffer. This is not a
performance optimization goal)
3. Transport Layer processing -- TCP protocol processing, header
and checksum checks.
a. Classify incoming TCP segment using the 5 tuple (IP SRC, IP
DST, TCP SRC Port, TCP DST Port, protocol)
4. Check for Header Alignment. (Described in detail in [MPA]
section 7.4). Assuming Header Alignment for the rest of the
algorithm below.
a. If the header is not aligned, see the algorithm defined in
the prior section.
5. If TCP is in-order or out-of-order the MPA header is at the
beginning of the current TCP payload. Get the FPDU length from
the FPDU header.
6. Calculate CRC over FPDU
7. Check CRC calculation for FPDU #N
8. If no FPDU CRC errors, placement is allowed
9. CopyData(TCP segment #X, host buffer address, length)
10. Loop to #5 until all the FPDUs in the TCP segment are consumed
in order to handle FPDU packing (see section 6).
Elzur, et al. Expires - August 2003 [Page 15]
Analysis of MPA over TCP February 2003
Implementation note: In both cases the receiver has to classify the
incoming TCP segment and associate it with one of the flows it
maintains. In the case of no Header Alignment, the receiver is
forced to classify incoming traffic before it can calculate the FPDU
CRC. In the case of Header Alignment the operations order is left to
the implementor.
The Header Aligned receiver algorithm is significantly simpler.
There is no need to locally buffer portions of FPDUs. Accessing
state information is also substantially simplified - the normal
case does not require retrieving information to find out where a
FPDU starts and ends or retrieval of a partial CRC before the CRC
calculation can commence. This avoids adding internal latencies,
having multiple data passes through the CRC machine, or scheduling
multiple commands for moving the data to the host buffer.
The aligned FPDU approach is useful for in-order and out-of-order
reception. The receiver can use the same mechanisms for data storage
in both cases, and only needs to account for when all the TCP segments have
arrived to enable delivery. . The Header Alignment, along with the high
probability that at least one complete FPDU is found with every TCP
segment, allows the receiver to perform data placement for out-of-
order TCP segments with no need for intermediate buffering.
Essentially the TCP receive buffer has been eliminated and TCP
reassembly is done in place within the ULP buffer.
In case Header Alignment is not found, the receiver should follow
the algorithm for non aligned FPDU reception which may be slower and
less efficient.
7.2 Header Alignment effects on TCP wire protocol
An MPA-aware TCP exposes its EMSS to MPA. MPA uses the EMSS to
calculate its MULPDU, which it then exposes to DDP, its ULP. DDP
uses the MULPDU to segment its payload so that each FPDU sent by MPA
fits completely into one TCP segment. This has no impact on wire
protocol and exposing this information is already supported on many
TCP implementations, including all modern flavors of BSD networking,
through the TCP_MAXSEG socket option.
In the common case, the ULP (i.e. DDP over MPA) messages provided to
the TCP layer are segmented to MULPDU size. It is assumed that the
ULP message size is bounded by MULPDU, such that a single ULP
message can be encapsulated in a single TCP segment. Therefore, in
the common case, there is no increase in the number of TCP segments
emitted. For smaller ULP messages, the sender can also apply
packing, i.e. the sender packs as many complete FPDUs as possible
into one TCP segment (See Section 6, No Packing, on page 7). The
requirement to always have a complete FPDU may increase the number
Elzur, et al. Expires - August 2003 [Page 16]
Analysis of MPA over TCP February 2003
of TCP segments emitted. Typically, a ULP message size varies from
few bytes to multiple EMSS (e.g., 64 Kbytes). In some cases the ULP
may post more than one message at the time for transmission, giving
the sender an opportunity for packing. In the case where more than
one FPDU is available for transmission and the FPDUs are
encapsulated into a TCP segment and there is no room in the TCP
segment to include the next complete FPDU, another TCP segment is
sent. In this corner case some of the TCP segments are not full
size. In the worst case scenario, the ULP may choose a FPDU size
that is EMSS/2 +1 and has multiple messages available for
transmission. For this poor choice of FPDU size, the average TCP
segment size is therefore about 1/2 of the EMSS and the number of
TCP segments emitted is approaching 2x of what is possible without
the requirement to encapsulate an integer number of complete FPDUs
in every TCP segment. This is a dynamic situation that only lasts
for the duration where the sender ULP has multiple non-optimal
messages for transmission and this causes a minor impact on the wire
utilization.
However, it is not expected that requiring Header Alignment will
have a measurable impact on wire behavior of most applications.
Throughput applications with large I/Os are expected to take full
advantage of the EMSS. Another class of applications with many
small outstanding buffers (as compared to EMSS) is expected to use
packing when applicable. Transaction oriented applications are also
optimal.
TCP retransmission is another area that can affect sender behavior.
TCP supports retransmission of the exact, originally transmitted
segment (see [RFC0793] section 2.6, [RFC0793] section 3.7 "managing
the window" and [RFC1122] section 4.2.2.15 ). In the unlikely event
that part of the original segment has been received and acknowledged
by the remote peer (e.g., a resegmenting middlebox, as documented in
[MPA]), a better available bandwidth utilization may be possible by
re-transmitting only the missing octets. If an MPA-aware TCP
retransmits complete FPDUs, there may be some marginal bandwidth
loss.
Another area where a change in the TCP segment number may have
impact is that of Slow Start and Congestion Avoidance. Slow-start
exponential increase is measured in segments per second, as the
algorithm focuses on the overhead per segment at the source for
congestion that eventually results in dropped segments. Slow-start
exponential bandwidth growth for MPA-aware TCP is similar to any TCP
implementation. Congestion Avoidance allows for a linear growth in
available bandwidth when recovering after a packet drop. Similar to
the analysis for slow-start, MPA-aware TCP doesn't change the
behavior of the algorithm. Therefore the average size of the segment
versus EMSS is not a major factor in the assessment of the bandwidth
Elzur, et al. Expires - August 2003 [Page 17]
Analysis of MPA over TCP February 2003
growth for a sender. Both Slow Start and Congestion Avoidance for an
MPA-aware TCP will behave similarly to any TCP sender and allow an
MPA-aware TCP to enjoy the theoretical performance limits of the
algorithms.
In summary, the ULP messages generated at the sender (e.g., the
amount of messages grouped for every transmission request) and
message size distribution has the most significant impact over the
number of TCP segments emitted. The worst case effect for certain
ULPs (with average message size of EMSS/2+1 to EMSS), is bounded by
an increase of up to 2x in the number of TCP segments and
acknowledges. In reality the effect is expected to be marginal.
See the MPA specification for additional documentation on corner
cases which are expected to lose Header Alignment and cause the
previously documented algorithm to be executed.
Elzur, et al. Expires - August 2003 [Page 18]
Analysis of MPA over TCP February 2003
8 Interoperating between MPA applications and non-MPA applications
ULPs that use MPA are required to enable MPA at an agreed-upon point
in the TCP datastream. If they fail to do so, the condition
illustrated in Figure 3: MPA Enablement Error arises. This
condition is referred to as an MPA enablement error. With the
understanding that MPA enablement errors should not occur, some
concerns have been raised about their effects if they do. The
remainder of this section addresses those concerns for a variety of
ULPs.
MPA is enabled if the MPA protocol is visible on the wire. When the
sender is MPA-enabled, it is inserting framing and markers. When
the receiver is MPA-enabled, it is interpreting framing and markers.
MPA enablement is orthogonal to MPA awareness. MPA can be enabled
on a strictly layered MPA implementation running over a non-MPA-
aware TCP. It can be disabled on an MPA-aware TCP implementation.
When first enabled, MPA always sends a marker preceding the first
FPDU. Because the marker is located on the boundary between FPDUs,
its initial value is always 0. Consequently, any ULP which never
starts its datastream with four zero octets is easily proved safe
with respect to MPA enablement errors.
Node A Node B
(MPA active) (MPA not active)
+---------------+ +---------------+
| ULP | | ULP |
+---------------+ | |
| MPA | | |
+---------------+ +---------------+
| TCP | | TCP |
+---------------+ +---------------+
| IP | | IP |
+---------------+ +---------------+
| |
+--------------------------+
Figure 3: MPA Enablement Error
Elzur, et al. Expires - August 2003 [Page 19]
Analysis of MPA over TCP February 2003
8.1 Negotiation of MPA-enabled mode
Transition to MPA can be accomplished by three possible methods
examined herein. The first two methods involve the use of specific
TCP ports for ULPs using MPA, either by use of a well known IANA
port or use of a service locator protocol. In these usage models it
is anticipated that both the Active and Passive ULPs will enable MPA
mode operation for their respective transmitters and receivers prior
to any data exchange. The third model is by negotiation of MPA
transition by the ULP using octet stream messages to accomplish a
ULP specific <MPA Hello>, <MPA Hello ACK>, <MPA ACK> three way
exchange. Transition to MPA framing will cause the transmitter to
always insert a 4 octet marker modulo the marker interval, and the
receiver to check the MPA CRC. The first MPA marker is defined to
have a value of 0, and it will always follow the last expected octet
to be transferred in octet stream mode. This leads to two distinct
error detection scenarios. Receivers that are expecting MPA framing
will quickly detect a CRC error in addition to any ULP header errors
(e.g., DDP) if given octet stream data from a non MPA-enabled
transmitter. This will cause MPA to drop the connection. Receivers
that are not expecting MPA framing will see four octets of zeros
immediately in the octet stream at the point the transition to MPA
was expected to occur.
Anticipated error cases are examined in the following figures.
Figure 4: MPA Transition via Well Known Port or Service Location
Protocol deals with the cases where the transition to MPA is
expected prior to data exchange and examines the four possible
combinations of mismatching MPA-enabled endpoints with non-MPA-
enabled endpoints. These error scenarios imply a configuration error
between active and passive nodes and are likely to simultaneously
represent a configuration error of the ULPs as well. Figure 5: MPA
Transition via Octet Stream Negotiation examines cases where
transition to MPA mode is by exchange of ULP specific MPA Hello/ACK
messages.
Elzur, et al. Expires - August 2003 [Page 20]
Analysis of MPA over TCP February 2003
+-------------+---------------------------+---------------------------+
| | MPA-Enabled | Non-MPA-Enabled |
| | Passive | Passive |
| | | |
+-------------+---------------------------+---------------------------+
| MPA-Enabled | First octets transmitted | Passive receiver does not |
| Active | are MPA mode. | understand MPA header, |
| | | which starts with 0. |
| | Receivers check for MPA | See section 8.2 for |
| | framing. | anticipated behavior from |
| | | existing protocols that |
| | Successful Transition to | run over TCP. |
| | MPA mode. | |
| | | Active receiver expects |
| | | MPA framing and CRC, |
| | | which will not be |
| | | present. Expect quick |
| | | detection and closing |
| | | the connection in error. |
| | | |
+-------------+---------------------------+---------------------------+
| Non- | First octets transmitted | Normal octet stream mode |
| MPA-Enabled | in octet streaming mode. | operation could occur, |
| Active | | but this indicates a con- |
| | Passive receiver expects | figuration error that |
| | MPA framing and CRC, | should be detected by the |
| | which will not be present.| ULP at either node. |
| | Expect quick detection | |
| | and closing the connec- | |
| | tion in error. | |
| | | |
+-------------+---------------------------+---------------------------+
Figure 4: MPA Transition via Well Known Port or Service Location
Protocol
Elzur, et al. Expires - August 2003 [Page 21]
Analysis of MPA over TCP February 2003
+-------------+----------------------------+--------------------------+
| | MPA-Enabled | Non-MPA-Enabled |
| | Passive | Passive |
| | | |
+-------------+----------------------------+--------------------------+
| MPA-Enabled | Completes Hello, Hello | Passive does not under- |
| Active | Ack exchange. | stand MPA Hello sequence |
| | | in octet-stream mode, |
| | Successful Transition | either |
| | to MPA mode. | |
| | | * Passive side ULP dis- |
| | | connects due to ULP |
| | | protocol error |
| | | |
| | | * Active side ULP times |
| | | out waiting for MPA |
| | | Hello ACK. |
| | | |
+-------------+----------------------------+--------------------------+
| Non- | Active side sends an octet | Normal octet-stream mode |
| MPA-Enabled | stream sequence that looks | operation |
| Active | like MPA Hello without | |
| | being aware of MPA. | |
| | Passive side ULP sees this | |
| | message and mistakenly | |
| | tries to transition to MPA | |
| | mode. This situation would | |
| | most likely be the result | |
| | of a ULP protocol error. | |
| | Passive side will trans- | |
| | ition to MPA mode on the | |
| | receiver and transmit MPA | |
| | Hello Ack, resulting in | |
| | one of two error cases: | |
| | | |
| | * MPA Hello Ack should | |
| | cause a ULP protocol | |
| | error at the Active | |
| | side receiver | |
| | | |
| | * Passive side receiver | |
| | will expect MPA framing | |
| | and CRC, which will | |
| | fail very quickly | |
| | | |
+-------------+----------------------------+--------------------------+
Figure 5: MPA Transition via Octet Stream Negotiation
Elzur, et al. Expires - August 2003 [Page 22]
Analysis of MPA over TCP February 2003
8.2 Analysis of existing TCP services
The following sections examine the initial octet stream exchanges
for many of the ULPs used over a TCP transport to determine behavior
if they were inadvertently subjected to MPA framed messages. It
should be kept in mind that if the procedures described in the
preceding section are followed, none of the conditions analyzed here
are possible. If a ULP is determined to be unsafe with respect to
MPA enablement errors, it means only that the ULP does not protect
itself against connection attempts by clients using some other ULP.
Any such protection is the responsibility of the ULP, not MPA.
8.2.1 The "Little" TCP Services
There is a group of TCP services, collectively referred to as the
"little" TCP services, which are occasionally useful for debugging
networks. Normally the "client" for these services is telnet, which
simply sends all typed character and displays all received
characters verbatim. Most of the "little" services are easily
proved safe.
The echo [RFC0862] server copies all received data back to the
client. Its behavior is identical even if the echo client or server
is accidentally MPA-enabled.
The discard [RFC0863] server never sends anything and discards
everything it receives. Neither the client nor the server can
detect the presence or absence of MPA on either side.
The chargen [RFC0864] server ignores all data sent to it and sends a
continuous stream of data. The format of the data is unspecified,
although printable, ASCII text is recommended. In any case, a non-
MPA-enabled chargen client receiving MPA frames will simply treat
them as data. An MPA-enabled chargen client receiving non-MPA
frames will detect a bad initial marker or a CRC error in the first
"frame".
The quote [RFC0865] server ignores all data sent to it and sends a
short octet string before closing the connection. The format of the
data is unspecified, although printable, ASCII text is recommended.
In any case, a non-MPA-enabled quote client receiving MPA frames
will simply treat them as data. An MPA-enabled quote client
receiving non-MPA frames will almost certainly detect a CRC error in
the first "frame". In the unlikely event that the quote of the day
is a properly formed MPA frame, an MPA-enabled client will treat the
"payload" of the bogus frame as data.
The daytime [RFC0867] server ignores all data sent to it and sends a
text timestamp. A non-MPA-enabled daytime client receiving MPA
Elzur, et al. Expires - August 2003 [Page 23]
Analysis of MPA over TCP February 2003
frames will simply treat them as data. An MPA-enabled daytime
client receiving non-MPA frames will detect a CRC error in the first
"frame".
The time [RFC0868] server ignores all data sent to it and sends a
32-bit binary timestamp representing seconds since 00:00 GMT 01
January, 1900. The timestamp will wrap sometime in 2036. An MPA-
enabled time client receiving non-MPA frames will attempt interpret
the time as an initial marker. If the timestamp is 0, it will be
considered valid, but the server will close the connection before
sending an MPA header. If the timestamp is not 0, it will be
rejected. A non MPA-enabled time client receiving MPA frames will
interpret the initial marker as a 0 timestamp and display the time
as 00:00 GMT 01 January, 1900.
8.2.2 ULPs Using Only Text Messages
Many ULPs format all TCP payload as lines of printable, ASCII text.
Such ULPs may all be analyzed together.
If a non-MPA-enabled endpoint A somehow becomes connected to an MPA-
enabled endpoint B, the precise sequence of events depends on
whether A or B is the first to send. If A sends first, B expects
the first four octets to be an initial marker, which must be all
zeros. Since A sends printable, ASCII text, and 0h is not printable
ASCII, then B detects a protocol violation and should terminate the
connection. If B sends first, A expects printable, ASCII text, but
receives four octets of zeros, so A detects a protocol violation and
should terminate the connection.
ULPs of this type are:
NNTP [RFC0977]
finger [RFC1288]
gopher [RFC1436] the first two octets are always CR LF
POP3 [RFC1939]
IMAP4 [RFC2060]
IRC client [RFC2812]
IRC server [RFC2813]
BEEP [RFC3081]
SIP [RFC3261]
Elzur, et al. Expires - August 2003 [Page 24]
Analysis of MPA over TCP February 2003
whois [RFC0954]
TACACS [RFC1492]
ident [RFC1413]
rwhois [RFC2167]
ACAP [RFC2244]
TIP [RFC2371]
8.2.3 ULPs with Fixed Initial Message
Several ULPs begin every connection with a fixed octet-string.
These ULPs are safe with respect to MPA enablement errors if the
first four octets of the initial string cannot be mistaken for valid
initial marker (four zero octets). These ULPs are:
HTTP/1.0 [RFC1945] Initial string begins with "HTTP"
RTSP [RFC2326] Initial string begins with "RTSP"
HTTP/1.1 [RFC2616] Initial string begins with "HTTP"
SMTP [RFC2821] Initial string begins with "HELO" or "EHLO"
CIFS [CIFS] Initial string begins with "\xffSMB"
gnutella [GNUT] Initial string begins with "GNUTELLA CONNECT"
8.2.4 Protocols with framed command headers
Several protocols always exchange command header. This analysis
looks at the impact of the first four octets of an MPA datastream
being interpreted as a command header and vice versa.
8.2.4.1 iSCSI
The first messages on any iSCSI [iSCSI] connection are a login
exchange. Since the first octet of a login request or response is a
login command code (value 3), iSCSI is safe with respect to MPA
enablement errors.
8.2.4.2 iSNS
The first two octets of every iSNS [iSNS] transmission are the
protocol version, which is currently 1. Consequently iSNS is safe
with respect to MPA enablement errors.
Elzur, et al. Expires - August 2003 [Page 25]
Analysis of MPA over TCP February 2003
8.2.4.3 DNS
In many cases, DNS [RFC1035] uses UDP, rendering MPA irrelevant.
This section analyzes the implications of MPA enablement errors in
the case where DNS runs over TCP.
Every DNS message sent over TCP is preceded by a two-octet length
field. Consequently, the initial marker of an MPA datastream would
be interpreted by a non-MPA-enabled receiver as a length field of 0.
The precise consequences of the error depend on whether the receiver
is the server or the resolver (i.e. client).
If a non-MPA-enabled resolver connects to an (improperly) MPA-
enabled DNS server, the server will interpret the length field of
the first request as an invalid initial marker and terminate the
connection. This configuration is safe with respect to MPA
enablement errors.
If an (improperly) MPA-enabled client connects to a non-MPA-enabled
DNS server, the server will interpret the upper 16 bits of the
initial marker as a length field of 0, indicating a null message.
The server might be expected to respond to a null message with a
format error, which is a safe outcome. However, [RFC1035] does not
mandate this behavior. The receiver could instead simply discard
the "null message" and continue. This latter case requires more
analysis.
Elzur, et al. Expires - August 2003 [Page 26]
Analysis of MPA over TCP February 2003
intended by resolver interpreted by server
0 1 0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-------------------------------+ +-------------------------------+
|marker[0..15] = 0x0000 | |length = 0x0000 (null message) |
+-------------------------------+ +-------------------------------+
|marker[16..31] = 0x0000 | |length = 0x0000 (null message) |
+-------------------------------+ +-------------------------------+
|MULPDU length | |message length |
+-------------------------------+ +-------------------------------+
|ID | |ID |
+-+-------+-+-+-+-+-----+-------+ +-+-------+-+-+-+-+-----+-------+
|Q| Opcode|A|T|R|R| 0 | RCODE | |Q| Opcode|A|T|R|R| 0 | RCODE |
|R| |A|C|D|A| | | |R| |A|C|D|A| | |
+-+-------+-+-+-+-+-----+-------+ +-+-------+-+-+-+-+-----+-------+
|QDCOUNT | |QDCOUNT |
+-------------------------------+ +-------------------------------+
|ANCOUNT | |ANCOUNT |
+-------------------------------+ +-------------------------------+
|NSCOUNT | |NSCOUNT |
+-------------------------------+ +-------------------------------+
|ARCOUNT | |ARCOUNT |
+-------------------------------+ +-------------------------------+
|data | |data |
| ... | | ... |
+-------------------------------+ +-------------------------------+
|pad // | |second message |
+-------------------------------+ | |
|CRC[0-15] | | |
+-------------------------------+ | |
|CRC[16-31] | | |
+-------------------------------+ +-------------------------------+
Figure 6: Effect of Improperly MPA-Enabled DNS Resolver
Figure 6: Effect of Improperly MPA-Enabled DNS Resolver illustrates
the correspondence between protocol fields as intended by the sender
and interpreted by the receiver. The initial marker would be
interpreted as a pair of null messages. The server would then
interpret the MULPDU length field as a DNS request length. As it
happens, the length field is correct, and the subsequent payload
fields line up properly, so the server will correctly interpret the
query and respond to it. The response will have a non-zero length
field, which MPA at the resolver will interpret as the upper half of
an erroneous initial marker, terminating the connection.
Elzur, et al. Expires - August 2003 [Page 27]
Analysis of MPA over TCP February 2003
Meanwhile the server will continue processing the incoming
datastream, interpreting the first 16 bits of the concatenated pad
and CRC as the length field of a second query. The second "query"
will be null, incomplete or too short to be valid. If the "query"
is null, the server will interpret the next 16 bits as a length, and
so forth. Eventually it will either safely run out of data or
detect a "query" that is incomplete or too short. If the "query" is
incomplete, the server will wait for additional payload, doing
nothing until the resolver disconnects; if it is too short, the
server will respond with a format error. In either case, no harm
results.
The interaction between MPA and DNS is complex, but as the analysis
shows, even in the worst case DNS is safe with respect to MPA
enablement errors.
8.2.4.4 LPR
The first octet of every request made to the LPR [RFC1179] daemon is
a printable ASCII character or a command code from the set
{1,2,3,4,5}. Consequently LPR is safe with respect to MPA
enablement errors.
8.2.4.5 Kerberos
The first octet of every Kerberos [RFC1510] request is the version
number (currently 5). Consequently Kerberos is safe with respect to
MPA enablement errors.
8.2.4.6 BGP-4
The first PDU in the BGP-4 protocol [RFC1771] is an OPEN with a
marker (covering the first four octets) of all one bits.
Consequently BGP-4 is safe with respect to MPA enablement errors.
8.2.4.7 LDAP v2 and LDAP v3
All LDAP [RFC1777, RFC2251] messages are encapsulated in an
LDAPMessage, which is defined as a SEQUENCE under ASN.1 Basic
Encoding Rules. Since the leading octet of a SEQUENCE is an ASN.1
BER type code of 0x30, no LDAP datastream can begin with 4 zero
octets. Consequently LDAP is safe with respect to MPA enablement
errors.
8.2.4.8 RTP
The most significant two bits of the first octet of an RTP [RFC1889]
payload contain the protocol version number (currently 2).
Consequently RTP is safe with respect to MPA enablement errors.
Elzur, et al. Expires - August 2003 [Page 28]
Analysis of MPA over TCP February 2003
8.2.4.9 Socks
The first octet of a SOCKS [RFC1928] datastream contain the protocol
version number (currently 5). Consequently SOCKS is safe with
respect to MPA enablement errors.
8.2.4.10 TLS
The first octet of a TLS [RFC2246] datastream is a command code from
the set {20,21,22,23,255}. Consequently TLS is safe with respect to
MPA enablement errors. This implies further than any ULP using TLS
is safe with respect to MPA enablement errors.
8.2.4.11 SLP V2
The first octet of an SLPv2 [RFC2608] datastream is the protocol
version number (currently 2). Consequently SLPv2 is safe with
respect to MPA enablement errors.
Elzur, et al. Expires - August 2003 [Page 29]
Analysis of MPA over TCP February 2003
9 Security Considerations
This document does not define protocols; hence it does not create
any new security considerations.
Elzur, et al. Expires - August 2003 [Page 30]
Analysis of MPA over TCP February 2003
10 IANA Considerations
This Internet Draft does not define any new protocols, thus there
are no IANA considerations.
Elzur, et al. Expires - August 2003 [Page 31]
Analysis of MPA over TCP February 2003
11 References
11.1 Primary References
[RFC0793] J. Postel, "Transmission Control Protocol", RFC 793,
September 1981.
[RFC0854] J. Postel & J.K. Reynolds, "Telnet Protocol
Specification", RFC 854, May 1983.
[RFC1122] R. Braden, Ed., "Requirements for Internet Hosts -
Communication Layers", RFC 1122, October 1989.
[RFC1191] J.C. Mogul & S.E. Deering, "Path MTU discovery", RFC 1191,
November 1990.
[RFC2026] S. Bradner, "The Internet Standards Process -- Revision
3", BCP 9, RFC 2026, October 1996.
[RFC2119] S. Bradner, "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[MPA] P. Culley et al., "Marker PDU Aligned Framing for TCP
Specification", draft-cully-iwarp-mpa-01.txt (work in progress),
October 2002
[RDMAP] R. Recio et al., "RDMA Protocol Specification", draft-recio-
iwarp-01.txt (work in progress), October 2002
[DDP] H. Shah et al., "Direct Data Placement over Reliable
Transports", draft-shah-iwarp-ddp-01.txt (work in progress),
October 2002
11.2 "Little" TCP Services
[RFC0862] J. Postel, "Echo Protocol", RFC 862, May 1983.
[RFC0863] J. Postel, "Discard Protocol", RFC 863, May 1983.
[RFC0864] J. Postel, "Character Generator Protocol", RFC 864, May
1983.
[RFC0865] J. Postel, "Quote of the Day Protocol", RFC 865, May 1983.
[RFC0867] J. Postel, "Daytime Protocol", RFC 867, May 1983.
[RFC0868] J. Postel & K. Harrenstien, "Time Protocol", RFC 868, May
1983.
Elzur, et al. Expires - August 2003 [Page 32]
Analysis of MPA over TCP February 2003
11.3 ULPs using only Text Messages
[RFC0977] B. Kantor & P. Lapsley, "Network News Transfer Protocol",
RFC 977, February 1986.
[RFC1288] D. Zimmerman, "The Finger User Information Protocol", RFC
1288, December 1991.
[RFC1436] F. Anklesaria, et al., "The Internet Gopher Protocol (a
distributed document search and retrieval protocol)", RFC 1436,
March 1993.
[RFC1939] J. Myers & M. Rose, "Post Office Protocol - Version 3",
RFC 1939, May 1996.
[RFC2060] M. Crispin, "Internet Message Access Protocol - Version
4rev1", RFC 2060, December 1996.
[RFC2812] C. Kalt, "Internet Relay Chat: Client Protocol", RFC 2812,
April 2000.
[RFC2813] C. Kalt, "Internet Relay Chat: Server Protocol", RFC 2813,
April 2000.
[RFC3081] M. Rose, "Mapping the BEEP Core onto TCP", RFC 3081, March
2001.
[RFC3261] J. Rosenberg, et al., "SIP: Session Initiation Protocol",
RFC 3261, June 2002.
[RFC0954] K. Harrenstien, et al., "NICNAME/WHOIS", RFC 954, October
1985.
[RFC1492] C. Finseth, "An Access Control Protocol, Sometimes Called
TACACS", RFC 1492, July 1993.
[RFC1413] M. St. Johns, "Identification Protocol", RFC 1413,
February 1993.
[RFC2167] S. Williamson, et al., "Referral Whois (RWhois) Protocol
V1.5", RFC 2167, June 1997.
[RFC2244] C. Newman & J. G. Myers, "ACAP -- Application
Configuration Access Protocol", RFC 2244, November 1997.
[RFC2371] J. Lyon, et al., "Transaction Internet Protocol Version
3.0", RFC 2371, July 1998.
Elzur, et al. Expires - August 2003 [Page 33]
Analysis of MPA over TCP February 2003
11.4 ULPs with Fixed Initial Message
[RFC1945] T. Berners-Lee, et al., "Hypertext Transfer Protocol --
HTTP/1.0", RFC 1945, May 1996.
[RFC2326] H. Schulzrinne, et al., "Real Time Streaming Protocol
(RTSP)", RFC 2326, April 1998.
[RFC2616] R. Fielding, et al., "Hypertext Transfer Protocol --
HTTP/1.1", RFC 2616, June 1999.
[RFC2821] J. Klensin, ed., "Simple Mail Transfer Protocol", RFC
2821, April 2001.
[CIFS] Storage Networking Industry Association, "Common Internet
File System (CIFS) Technical Reference",
http://www.snia.org/tech_activities/CIFS/CIFS-TR-1p00_FINAL.pdf,
March 2002.
[GNUT] Anonymous, "The Gnutella Protocol Specification v0.4",
http://www9.limewire.com/developer/gnutella_protocol_0.4.pdf.
11.5 ULPs with Framed Command Headers
[iSCSI] J. Satran et al., "iSCSI", draft-ietf-iscsi-19.txt (work in
progress), January 2003
[iSNS] J. Tseng et al., "Internet Storage Name Service (iSNS)",
draft-ietf-ips-isns-16.txt (work in progress), January 2003
[RFC1035] P.V. Mockapetris, "Domain names - implementation and
specification", RFC 1035, November 1987.
[RFC1179] L. McLaughlin, "Line printer daemon protocol", RFC 1179,
August 1990.
[RFC1510] J. Kohl & C. Neuman, "The Kerberos Network Authentication
Service (V5)", RFC 1510, September 1993.
[RFC1771] Y. Rekhter & T. Li, "A Border Gateway Protocol 4 (BGP-4)",
RFC 1771, March 1995
[RFC1777] W. Yeong, et al., "Lightweight Directory Access Protocol",
RFC 1777, March 1995.
[RFC2251] M. Wahl, et al., "Lightweight Directory Access Protocol
(v3)", RFC 2251, December 1997.
Elzur, et al. Expires - August 2003 [Page 34]
Analysis of MPA over TCP February 2003
[RFC1889] H. Schulzrinne, et al., "RTP: A Transport Protocol for
Real-Time Applications", RFC 1889, January 1996.
[RFC1928] M. Leech, et al., "SOCKS Protocol Version 5", RFC 1928,.
March 1996.
[RFC2246] T. Dierks & C. Allen, "The TLS Protocol Version 1.0", RFC
2246, January 1999.
[RFC2608] E. Guttman, et al., "Service Location Protocol, Version
2", RFC 2608, June 1999.
Elzur, et al. Expires - August 2003 [Page 35]
Analysis of MPA over TCP February 2003
12 Author's Addresses
Uri Elzur
Broadcom Corporation
16215 Alton Parkway
Irvine, CA 92619-7013 USA
Phone: +1 (949) 585-6432
Email: Uri@Broadcom.com
James Pinkerton
Microsoft Corporation
One Microsoft Way
Redmond, WA 98052 USA
Phone: +1 (425) 705-5442
Email: jpink@microsoft.com
Robert Teisberg
Hewlett-Packard Company
14231 Tandem Blvd.
Austin, TX 78728
Phone: +1 (512) 432-8119
Email: Robert.Teisberg@hp.com
Dwight Barron
Hewlett-Packard Company
20555 SH 249
Houston, TX 77070-2698 USA
Phone: +1 (281) 514-2769
Email: Dwight.Barron@Hp.com
John Carrier
Adaptec, Inc.
691 S. Milpitas Blvd.
Milpitas, CA 95035 USA
Phone: +1 (360) 378-8526
Email: john_carrier@adaptec.com
Paul R. Culley
Hewlett-Packard Company
20555 SH 249
Houston, TX 77070-2698 USA
Phone: +1 (281) 514-5543
Email: paul.culley@hp.com
Elzur, et al. Expires - August 2003 [Page 36]
Analysis of MPA over TCP February 2003
13 Acknowledgments
Vadim Makhervaks
IBM Corp., Haifa Development Lab
Haifa, Israel
Phone: +972-4-829-6537
Email: VADIK@il.ibm.com
Renato Recio
IBM Corp.
11501 Burnett Road
Austin, Tx. USA 78758
Phone: 512-838-3685
Email: recio@us.ibm.com
Tom Talpey
Network Appliance
375 Totten Pond Road
Waltham, MA 02451 USA
Phone: +1 (781) 768-5329
EMail: thomas.talpey@netapp.com
Patricia Thaler
Agilent Technologies, Inc.
1101 Creekside Ridge Drive, #100
M/S-RG10
Roseville, CA 95678
Phone: +1-916-788-5662
Email: pat_thaler@agilent.com
Elzur, et al. Expires - August 2003 [Page 37]
Analysis of MPA over TCP February 2003
14 Full Copyright Statement
Copyright (C) The Internet Society (2003). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph
are included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Funding for the RFC Editor function is currently provided by the
Internet Society.
Elzur, et al. Expires - August 2003 [Page 38]
