Internet DRAFT - draft-dekok-forces-metadata
draft-dekok-forces-metadata
Internet Draft A. DeKok
Expiration: March 2003 IDT, Inc.
Working Group: Forces October, 2003
A Discussion of Metadata in ForCES
draft-dekok-forces-metadata-00.txt
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as ``work in progress.''
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Abstract
This document describes a model for discussing metadata in ForCES.
It defines multiple kinds of metadata, and shows which ones are in
scope for ForCES, and which ones are out of scope for ForCES. It
further defines a vocabulary for discussing metadata, which should
help to avoid much of the historical confusion and disagreement
surrounding discussions of metadata.
Table of Contents
Abstract...........................................................1
1. Introduction ...................................................2
2. An exposition of metadata ......................................2
2.1 Internal versus External Metadata ...........................2
2.2 Implicit versus Explicit Metadata ...........................3
3. Further Exposition .............................................3
3.1 More detailed examples ......................................3
3.2 The concept of "scale" as it impacts metadata ...............4
4. Implication for ForCES .........................................4
Internet Draft A Discussion of Metadata in ForCES [Page 1]
Internet draft March 2003
5. Security Considerations ........................................5
6. Intellectual Property Right ....................................5
7. Normative References ...........................................5
8. Informative References .........................................5
9. Acknowledgments ................................................5
10. Authors' Addresses ............................................6
1. Introduction
Metadata has historically been understood to mean "data about data".
While this definition is better than nothing, it has contributed
significantly to misunderstanding and miscommunication, when disparate
parties attempt to discuss the topic. Our goal in this document is to
give a more detailed definition of metadata, to show where different
kinds of metadata occur in network devices, and to explain those
differences via specific examples.
2. An Exposition of Metadata
Our presentation of metadata divides it by two orthogonal axes: Internal
versus External, and Implicit versus Explicit.
2.1 Internal versus External Metadata
We define Internal metadata as data which is produced and consumed
entirely within a particular context (usually network device or chip).
For example, an Ethernet MAC may have an internal timer associated with
a packet it is trying to transmit. If a collision is detected during
transmission, the MAC performs an exponential back-off, and attempts to
re-transmit the packet. The timer associated with that exponential
back-off is internal to the MAC, and usually cannot be seen by any
driver software. Nevertheless, that timing data is in the MAC, and is
associated with the current packet. We can therefore describe that
timing data as metadata about the current packet.
We define External metadata as data which is visible outside of a
particular context (usually network device or chip).
For example, a chip implementing an 8-port switch fabric may need to be
told the output port to which an input packet will be redirected, if it
does not make that decision itself. That information may be supplied
via in-band data, such as a series of bits which precede the packet, or
it may be supplied via other methods. In either case, the information
which tells the switch fabric chip where to redirect the packet may be
described as metadata, which is external to the switch fabric chip.
Internet Draft A Discussion of Metadata in ForCES [Page 2]
Internet draft March 2003
2.2 Implicit versus Explicit Metadata
We define Implicit metadata as data which may be determined implicitely
from context.
For example, an Ethernet MAC is never told that the packets it receives
are in Ethernet format. That information is determined implicitly by
the fact that the physical form factor of Ethernet cabling ensures
(generally) that only transmitters of Ethernet packets are connected to
receivers of Ethernet packets. The receivers, therefore, are never told
explicitly that the packets are in Ethernet format. Instead, they
proceed under the assumption that the packets are Ethernet.
We define Explicit metadata as data which is given as information in
addition to the packet data.
For example, a chip implementing an 8-port switch fabric may be given
explicit information as to the output port to which an input packet will
be redirected. That information often cannot be determined implicitly
from context, but must be explicitly supplied by something external to
the switch fabric chip.
3. Further exposition.
The exposition of the axes used to describe metadata in Section 2 is
admittedly somewhat naive. This section gives more detailed
examples, and shows how the concept of "scale" impacts metadata.
3.1 More detailed examples
TCP packets are carried within IP packets, and the IP packet contains
a "protocol" field, with an explicit value of "6", for TCP. That
protocol field can be viewed from the perspective of the TCP packet
as Explicit, External metadata. In contrast, the format of the TCP
packet is implicitely defined, and is never sent over the wire.
Therefore, the TCP packet format can be viewed as Implicit, External
metadata.
IP MPLS can be viewed as associating 32 bits of Explicit, External
metadata with an IP packet. That metadata informs MPLS-aware devices
as to how the IP packet should be forwarded. The use of that
metadata allows MPLS-aware devices to perform that forwarding without
examining or modifying the IP packet.
More examples should be inserted here.
Internet Draft A Discussion of Metadata in ForCES [Page 3]
Internet draft March 2003
3.2 The concept of "scale" as it impacts metadata
The ForCES model document[2] describes how a network device may be
modelled via multiple Function Elements (FE's), each of which may be
modelled via multiple Logical Function Blocks (LFB's.) We now
discuss how the view of metadata changes at each level of that model.
The metadata exchanged between devices at one level of the model may
be viewed as External, Explicit metadata at that level. For example,
the metadata distributed between LFB's may include information such
as classification result, which may be represented as a sequence of
bits prefixing a packet.
At a higher level, however, the lower-level metadata may not be
visible. For example, an IPv4 forwarder may be represented as a set
of LFB's which exchange packets and metadata. At the FE level, the
metadata inside of the IPv4 forwarder may not be visible. The other
FE's will simply see a packet entering the device, and some time
later, a modified packet exiting the device.
The discussion of scale may be repeated until it encompasses the
Internet. Sets of LFB's form FE's, sets of FE's form network
devices, sets of network devices form networks, and sets of networks
form the Internet. At each level of scale, we can characterize
metadata as in Section 2, even though that same metadata may have a
different characterization at another scale.
This multiple interpretation of metadata becomes even more difficult
when we realize that different vendors have implementations which may
exist at multiple scales, or at disparate scales. For example, one
vendor may choose to implement an IPv4 forwarder in a monolithic
chip, while another implements an IPv4 forwarder through a set of
distinct chips. Both implementations must be controllable and
addressable through ForCES, which is a difficult proposition to
satisfy in general.
4. Implications for ForCES.
The discussion of metadata within the context of ForCES has
historically been contentious. We hope that this exposition of
metadata helps to decrease the confusion surrounding metadata, by
supplying a clear vocabulary which may be used to discuss the issue.
Further, we can use the preceding characterization of metadata to
narrow the scope of using metadata within the context of ForCES.
Internet Draft A Discussion of Metadata in ForCES [Page 4]
Internet draft March 2003
Specifically, Internal metadata should be described as outside of the
scope of ForCES. Vendors of network devices should be free to choose
any manner of implementing or representing metadata within their
devices. Any attempt by ForCES to standardize the representation or
use of Internal metadata would be an unwarranted burden on vendors.
In contrast, as one of the goals of ForCES is to facilitate inter-
vendor communication, External metadata is within the scope of
ForCES. The group should create definitions of, and standards for,
External metadata which permit different vendor implementations of
devices to communicate metadata.
That metadata may be Implicit (inputs to LFB 'X' are assumed to be
outputs from LFB 'Y'), and Explicit (this packet is to go out port
"4").
We hope that this discussion of metadata terminology, scope, and
scale, helps in facilitating communication among members of ForCES.
5. Security Considerations
There are no security considerations associated with this document.
6. Normative References
[1] Khosravi, H. et al., "Requirements for Separation of IP Control
and Forwarding", work in progress, draft-ietf-forces-
requirements-10.txt, Intel Labs, July 2003.
7. Informative References
[2] Yang, Lily et. al., "ForCES Forwarding Element Model", work in
progress, draft-ietf-forces-model-01.txt, Intel Labs, October 2003.
8. Intellectual Property Right
The authors are not aware of any intellectual property right issues
pertaining to this document.
9. Acknowledgements
The authors would also like to thank the following individuals for
their valuable technical input: David Maxwell, Michael Richardson,
Internet Draft A Discussion of Metadata in ForCES [Page 5]
Internet draft March 2003
and Wojtek Fraczak.
10. Authors' Address
Alan DeKok
IDT Inc.
1575 Carling Ave.
Ottawa, ON
Canada
K1Z 7M3
Email: alan.dekok@idt.com
Internet Draft A Discussion of Metadata in ForCES [Page 6]
