Internet DRAFT - draft-bclaise-ipfix-reliability
draft-bclaise-ipfix-reliability
B. Claise
P. Aitken
Internet Draft R. Stewart
draft-bclaise-ipfix-reliability-01.txt P. Lei
Expires: September 07 2006 Cisco Systems
March 2006
IPFIX Reliability Extensions
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress".
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
This Internet-Draft will expire on September 07, 2006.
Copyright Notice
Copyright (C) The Internet Society (2006).
Abstract
This memo defines an extension to the IP Flow Information eXport
(IPFIX) protocol in order to accommodate the specific requirements of
billing.
Conventions used in this document
Claise B. [Page 1]
IPFIX Protocol Specification for billing March 2006
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119.
Table of Contents
1. Open Issues..................................................2
2. Introduction.................................................2
2.1 IPFIX Documents Overview...................................3
3. Terminology..................................................3
4. Reliability..................................................4
4.1 Requirements...............................................4
4.2 Choice of the IPFIX Transport Protocol.....................4
4.3 Backup and Failover........................................5
4.4 Reliable Server Pooling Support............................5
4.5 Application Level Acknowledgments..........................6
5. Uniqueness...................................................6
5.1 Requirements...............................................6
5.2 Data Records De-duplication and Completeness...............7
6. Security.....................................................7
6.1 Requirements...............................................7
6.2 IPsec or TLS...............................................8
7. Security Considerations......................................8
8. The Collecting Process' side.................................8
9. References...................................................8
9.1 Normative References.......................................8
9.2 Informative References.....................................9
1. Open Issues
This section covers the open issues, still to be resolved/updated in
this draft.
1. Investigate whether the application level acknowledgments are
required.
2. The round-robin RserPool policy is the only that makes for IPFIX.
Should we have a new policy that goes send back to the primary
when he is back online?
2. Introduction
The IP Flow Information eXport (IPFIX) protocol serves for
transmitting information related to measured IP traffic over the
Internet. The protocol specification in [IPFIX-PROTO] defines how
Information Elements are transmitted. For Information Elements, it
specifies the encoding of a set of basic data types.
Claise B. [Page 2]
IPFIX Protocol Specification for billing March 2006
The list of Information Elements that can be transmitted by the
protocol, such as Flow attributes (source IP address, number of
packets, etc.) and information about the Metering and Exporting
Processes (packet Observation Point, sampling rate, Flow timeout
interval, etc.), is specified in [IPFIX-INFO].
The IPFIX Working Group went through the process of specifying the
requirements for the export of measured IP flow information out of
routers, traffic measurement probes, and middleboxes [RFC3917]. While
the generic requirements for all application types were specified,
the appendix explains the derivation of the requirements from the
applications: it expresses the level of requirements (may, should,
must), per application, for every single requirement. The following
applications were looked at: QoS monitoring, attack/intrusion
detection, traffic engineering, traffic profiling, and usage based
billing. However, as expressed in the IPFIX Applicability Statement
draft [IPFIX-AS], it must be noted that the reliability requirements
defined in [RFC3917] are not sufficient to guarantee the level of
reliability that is needed for many usage-based accounting systems.
Particular reliability requirements for accounting systems are
discussed in [RFC2975].
This document specifies how the IPFIX requirements and improvements
to be suitable for billing applications that require a higher level
of reliable.
2.1 IPFIX Documents Overview
The IPFIX protocol provides network administrators with access to IP
flow information. The architecture for the export of measured IP
flow information out of an IPFIX exporting process to a collecting
process is defined in [IPFIX-ARCH], per the requirements defined in
[RFC3917]. The IPFIX protocol document [IPFIX-PROTO] specifies how
IPFIX data record and templates are carried via a congestion-aware
transport protocol from IPFIX exporting processes to IPFIX collecting
process. IPFIX has a formal description of IPFIX information
elements, their name, type and additional semantic information, as
specified in [IPFIX-INFO]. Finally [IPFIX-AS] describes what type of
applications can use the IPFIX protocol and how they can use the
information provided. It furthermore shows how the IPFIX framework
relates to other architectures and frameworks. This document
specifies how IPFIX can be made suitable for billing applications
that require a higher level of reliable.
3. Terminology
The IPFIX-specific terminology used in this document is defined in
section 3 of [IPFIX-PROTO]. As in [IPFIX-PROTO], these IPFIX-
Claise B. [Page 3]
IPFIX Protocol Specification for billing March 2006
specific terms have the first letter of a word capitalized when used
in this document. As a minimum, the terms defined in the terminology
summary table (figure A) are used throughout this document.
+------------------+---------------------------------------------+
| | Contents |
| +--------------------+------------------------+
| Set | Template | Record |
+------------------+--------------------+------------------------+
| Data Set | / | Data Record(s) |
+------------------+--------------------+------------------------+
| Template Set | Template Record(s) | / |
+------------------+--------------------+------------------------+
| Options Template | Options Template | / |
| Set | Record(s) | |
+------------------+--------------------+------------------------+
Figure A: Terminology Summary Table
4. Reliability
4.1 Requirements
Billing applications require reliability to ensure that exported Data
Records are received by a collector.
A dedicated mechanism or dedicated mechanisms at the protocol level
should allow an extra level of reliability for the Data Records.
4.2 Choice of the IPFIX Transport Protocol
The IPFIX Protocol Specification [IPFIX-PROTO] has been designed to
be transport protocol independent. The IPFIX reliability extension
required the use of SCTP [RFC2960] using the PR-SCTP [RFC3758]
extension. Refer to [IPFIX-PROTO] for the detailed specification of
SCTP in IPFIX. The UDP and TCP transport protocols, which may be
used in IPFIX under certain conditions [IPFIX-PROTO], MUST NOT be
used for the IPFIX reliability extension.
[IPFIX-PROTO] specifies that the IPFIX Template Set and Options
Template Set MUST be sent over the reliable stream zero. The IPFIX
reliability extensions impose that the Data Records MUST also be sent
over a reliable stream. The reliable stream on which the Data
Records are sent MAY be the stream zero. The reliable stream on
which the Data Records are sent MAY be another reliable stream than
stream zero.
Claise B. [Page 4]
IPFIX Protocol Specification for billing March 2006
4.3 Backup and Failover
The Collecting Process failover MUST be supported by the Exporting
Process, for which a second SCTP association MUST be opened in
advance. All Templates and Option Templates MUST be sent ahead of
time to the second SCTP association. The SCTP association parameters
SHOULD be tuned in order to allow a minimum detection time in case of
connection failure.
SCTP provides the ability of a sender to retrieve the unacknowledged
data when an association fails. Each SCTP API uses various
definitions to ask the SCTP interface for this retrieval. An example
of this can be found in the SOCKET's API (draft-ietf-tsvg-sctpsocket-
11.txt) it is called a "SCTP_SEND_FAILED" notification. To receive it
the sender subscribes to the "SCTP_send_failure_event" using the
socket option SCTP_EVENTS. Each Exporting process should use such a
mechanism to receive these send failed notifications.
After subscribing to the SCTP's API for failure data notifications,
an SCTP sender, at the failing of an association to a collector, will
be able to retrieve all of the pending data that has been queued to
the collector but NOT acknowledged. Each notification comes with the
data, and an indication as to if the data was SENT and not
acknowledged or if the data was never sent (due to congestion or
receiver window limitations). The Exporting process MUST retransmit
this information to its backup collector. Information that was never
sent can be safely sent to the backup collector just like other new
data. Data that as been sent to the previous association but not
acknowledged should be processed with care by the backup collector
since it is possible that the data was read and processed already by
the failed collector.
4.4 Reliable Server Pooling Support
The RFC 3237 "Requirements for Reliable Server Pooling" [RFC3237]
that clearly express the requirements for Reliable Server Pooling
(RserPool), could easily be applied to IPFIX when an extra set of
reliability is required. If the Exporting Process and Collecting
Process require a more capable fault tolerance and higher
availability, the (RSerPool) architecture [RSERPOOL-ARCH] SHOULD be
used. RSerPool uses the features of SCTP.
The RSerPool architecture provides a framework for building fault
tolerant and highly available client/server applications between Pool
Users (clients/users) and Pool Elements (servers/services). Pools are
identified by a Pool Handle (name). In the context of RSerPool for
IPFIX, the Exporting Processes are Pool Users and the Collecting
Claise B. [Page 5]
IPFIX Protocol Specification for billing March 2006
Processes are the Pool Elements, which provide the collection
services to the Exporting Processes.
The Collecting Processes must be configured into the desired pool(s)
and registered under the desired Pool Handle. Collecting Processes
may be part of multiple pools and thus provide collection services to
pools. The number of Collecting Processes may be increased
dynamically by simply having additional Collecting Processes register
into the pool. Similarly, the number may be decreased by having some
Collecting Processes de-register from the pool. The Collecting
Process should also provide a state context value to RSerPool to
allow any Collecting Process that may take over for a failed
Collecting Process to quickly lookup this state context to resume
collecting services.
The Exporting Processes use services of the desired pool by sending
the export information to the desired Pool Handle. If this is the
first message sent to the pool, RSerPool will select an available
Collecting Process to be used for this Exporting Process according to
the pool's selection policy [RSERPOOL-POLICIES]. The association
between the Exporting Process and selected Collecting Process will be
maintained unless the Exporting Process restarts, or a failover has
occurred for the Collecting Process. That is, additional sends to
the same Pool Handle will result in messages being sent to the same
Collecting Process.
When a Collecting Process fails, RSerPool will automatically select a
new Collecting Process from the pool and will associate the new
process to the Exporting Process. The data retrieval procedures
described in 4.3.1 above will be performed on behalf of the Exporting
and new Collecting Processes.
4.5 Application Level Acknowledgments
EDITOR'S NOTE: evaluate and potentially write some text
5. Uniqueness
5.1 Requirements
Billing applications require a de-duplication mechanism in order to
eliminate redundant duplication of Data Records. They also require a
mechanism to uniquely identify Data Records on different Collectors.
A typical example is an export transport connection failure to the
primary Collector, which triggers export to the backup Collector.
In order to process all the billing Data Records, the primary
Collector must identify whether duplicated Data Records have been
Claise B. [Page 6]
IPFIX Protocol Specification for billing March 2006
received during the transport connection failure, must transfer all
the Data Records from the backup Collector, and must eliminate the
duplicate ones.
5.2 Data Records De-duplication and Completeness
The Collecting Process MUST create an unique packet ID out of the
IPFIX Message Export Time, Sequence Number, Source ID, and Exporter.
The Collector MUST associate every Data Record with this unique
packet ID before one of the two following tasks is executed:
- Data Records de-duplication.
- Data Records accumulation for other Collector(s) in case of
collector failover or partial export to different Collectors.
The Collector, which is considered as the primary Collector, SHOULD
check the Data Records de-duplication and Data Records accumulation
with other Collectors before executing any record aggregation or
filtering.
Once the de-duplication and accumulation tasks are executed, the
unique packet ID associated with the Data Records MAY be discarded.
Note that the unique packet ID could also be useful for Data Records
accumulation in case of duplicate export to two Collectors on the top
of UDP, which doesn't guarantee the reliable delivery of IPFIX
Messages.
EDITOR'S NOTE:
- this should be a little bit expanded to explain how the primary
collector gets the data records from the secondary collector,
discard them if already available, and store them if not
available.
- Should also explain what is a primary collector? Do we need a
communication between the 2 collectors? We don't want a situation
where primary collector is down, and the backup doesn't retrieve
the information from the "primary". To solve this, can we have a
kind of HSRP priority, set by the router, which is the only one to
know where one collector is down (or at least, when the connection
between the router and the collector is down, which is what we
care about)
6. Security
6.1 Requirements
Claise B. [Page 7]
IPFIX Protocol Specification for billing March 2006
Billing applications require security to prevent tampering by
ensuring the validity of the received Data Records, while preventing
unauthorized access to those Data Records to ensure privacy.
Proper security mechanisms are needed in order to avoid tampering and
eavesdropping.
6.2 IPsec or TLS
The IPFIX protocol MUST run on the top of IPsec or TLS [TLS], as
specified in [IPFIX-PROTO].
7. Security Considerations
This draft is an extension the IPFIX protocol specifications. As
such, it does not address new security considerations that were not
covered in [IPFIX-PROTO].
8. The Collecting Process' side
After the detection of the SCTP association failure, the primary
Collecting Process SHOULD query all the Data Records from the
secondary Collecting Process on regular basis, in order to de-
duplicate the Data Records and to complete the billing records.
The Collecting Process MUST either process all the Data Records
contained into an IPFIX Messages, or MUST not process any of the Data
Records contained in an IPFIX Messages. By processing, the authors
mean the aggregating or filtering functions.
9. References
9.1 Normative References
[IPFIX-INFO] Quittek, J., Bryant S., Claise, B., Meyer, J.
"Information Model for IP Flow Information Export" draft-ietf-ipfix-
info-09, July 2005
[IPFIX-PROTO] Claise, B., "Information Model for IP Flow Information
Export" draft-ietf-ipfix-protocol-17, July 2005
[IPFIX-ARCH] Sadasivan, G., Brownlee, N., Claise, B., Quittek, J.,
"Architecture Model for IP Flow Information Export" draft-ietf-ipfix-
arch-08.txt", May 2005
[RFC2960] Stewart, R. (ed.) "Stream Control Transmission Protocol",
RFC 2960, October 2000
Claise B. [Page 8]
IPFIX Protocol Specification for billing March 2006
[RFC3237] Xie, Q., Stewart, R., Shore, M., Ong, L., Loughney, J.,
Stillman, M., " Requirements for Reliable Server Pooling ", RFC
3237, January 2002
[RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., Conrad, P.
"Stream Control Transmission Protocol (SCTP) Partial Reliability
Extension", RFC 3758, May 2004
[RSERPOOL-ARCH] Tuexen, M., Xie, Q., Stewart, F., Shore, M.,
Loughney, J., Silverton, A., " Architecture for Reliable Server
Pooling "draft-ietf-rserpool-arch-10.txt", July 2005
[RSERPOOL-POLICIES] Tuexen, M., Dreibholz, T., "Reliable Server
Pooling Policies" draft-ietf-rserpool-policies-02.txt, February 2006
9.2 Informative References
[RFC3917] Quittek, J., Zseby, T., Claise, B., Zander, S.,
"Requirements for IP Flow Information Export" RFC 3917, October 2004
[IPFIX-AS] Zseby, T., Boschi, E., Brownlee, N., Claise, B., "IPFIX
Applicability", draft-ietf-ipfix-as-05.txt, May 2005
[RFC2975] Aboba, B., Arkko, J., Harrington, D. "Introduction to
Accounting Management", RFC 2975, October 2000
[TLS] Dierks, T. and C. Allen, "The TLS Protocol Version
1.0", RFC 2246, January 1999.
Authors' Addresses
Benoit Claise
Cisco Systems
De Kleetlaan 6a b1
1831 Diegem
Belgium
Phone: +32 2 704 5622
E-mail: bclaise@cisco.com
Paul Aitken
Cisco Systems
3rd Floor, 96 Commercial Quay, Commercial Street
EH6 6LX Edinburgh
Scotland
Phone: +44 (0)131 561-3616
E-mail: paitken@cisco.com
Claise B. [Page 9]
IPFIX Protocol Specification for billing March 2006
Randall R. Stewart
Cisco Systems
4875 Forest Drive
Suite 200
Columbia, SC 29206
United States
E-mail: rrs@cisco.com
Peter Lei
Cisco Systems
8735 W Higgins Rd, Suite 300
Chicago, IL 60631
United States
Phone: +1 773 695 8201
E-mail: peterlei@cisco.com
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed
to pertain to the implementation or use of the technology described
in this document or the extent to which any license under such
rights might or might not be available; nor does it represent that
it has made any independent effort to identify any such rights.
Information on the procedures with respect to rights in RFC
documents can be found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use
of such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository
at http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at ietf-
ipr@ietf.org.
The IETF has been notified of intellectual property rights claimed
in regard to some or all of the specification contained in this
document. For more information consult the online list of claimed
rights.
Claise B. [Page 10]
IPFIX Protocol Specification for billing March 2006
Disclaimer of Validity
This document and the information contained herein are provided on
an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE
INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement
Copyright (C) The Internet Society (2006). This document is subject
to the rights, licenses and restrictions contained in BCP 78, and
except as set forth therein, the authors retain all their rights.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
Claise B. [Page 11]
