] > Hitachi Europe

c/o ETH Zurich Gloriastrasse 35 8092 Zurich Switzerland +41 44 632 70 57 elisa.boschi@hitachi-eu.com

Hitachi Europe

c/o ETH Zurich Gloriastrasse 35 8092 Zurich Switzerland +41 44 632 70 13 brian.trammell@hitachi-eu.com

Fraunhofer IFAM

Wiener Str. 12 28359 Bremen Germany +49 421 2246206 lutz.mark@ifam.fraunhofer.de

Fraunhofer Institute for Open Communication Systems

Kaiserin-Augusta-Allee 31 10589 Berlin Germany +49 30 3463 7153 tanja.zseby@fokus.fraunhofer.de

Operations IPFIX Working Group This document describes an extension to the IP Flow Information Export (IPFIX) protocol, which is used to represent and transmit data from IP flow measurement devices for collection, storage and analysis, to allow the encoding of IPFIX Information Model properties within an IPFIX Message stream. This enables the export of extended type information for enterprise-specific Information Elements, and the storage of such information within IPFIX Files, facilitating interoperability and reusability among a wide variety of applications and tools.

IP Flow Information Export (IPFIX) provides a template mechanism for the flexible description of Data Records, by defining a record as a collection of Information Elements defined in an IANA registry, However, these Templates provide limited information about the type of described data; indeed, they encode only the size of the fields defined by these Information Elements. There presently exists no mechanism to provide full type information for these Information Elements, as is defined for the Information Elements in the IPFIX Information Model. This especially limits the interoperability of enterprise-specific Information Elements. It is not possible to use analysis tools on IPFIX records containing these partially defined Information Elements that have not been developed with a priori knowledge of their types, since such tools will not be able to decode them; these tools can only treat and store them as opaque octet arrays. However, if richer information is available, additional operations such as efficient storage, display, and limited analysis of records containing enterprise-specific Information Elements become possible, even for Collecting Processes that had not been specifically developed to understand them. This document proposes a general mechanism to encode the full set of properties available for the definition of Information Elements within the IPFIX Information Model inline within an IPFIX Message stream using IPFIX Options. This mechanism may be used to fully define type information for Information Elements used within a message stream, without resort to an external reference or reliance on out-of-band configuration, thereby improving the interoperability of enterprise-specific Information Elements. Note that the solution described in this draft is not intended as a replacement for registration with IANA of generally useful Information Elements. It introduces overhead and does not lead to real interoperability as provided by standardization. Therefore we highly recommend to standardize all new generally useful Information Elements by registering them with IANA. Standardization is straightforward, and the type information that needs to be specified in order to support the proposed solution provides a perfect basis for the description required for standardizing the Information Element. It might happen that an Information Element previously described by the mechanism in this document later becomes an IANA-registered, standard Information Element. In such environments old and new versions of the Information Element can coexist. A translation between Information Elements expressed by the described solution and standardized Information Elements is therefore not necessary, and is out of scope for this document.

"Specification of the IPFIX Protocol for the Exchange of IP Traffic Flow Information" (informally, the IPFIX Protocol document) and its associated documents define the IPFIX Protocol, which provides network engineers and administrators with access to IP traffic flow information. "Architecture for IP Flow Information Export" (the IPFIX Architecture document) defines the architecture for the export of measured IP flow information out of an IPFIX Exporting Process to an IPFIX Collecting Process, and the basic terminology used to describe the elements of this architecture, per the requirements defined in "Requirements for IP Flow Information Export". The IPFIX Protocol document then covers the details of the method for transporting IPFIX Data Records and Templates via a congestion-aware transport protocol from an IPFIX Exporting Process to an IPFIX Collecting Process. "Information Model for IP Flow Information Export" (informally, the IPFIX Information Model document) describes the Information Elements used by IPFIX, including details on Information Element naming, numbering, and data type encoding. This document references the Protocol and Architecture documents for terminology and extends the IPFIX Information Model to provide new Information Elements for the representation of Information Element properties. It draws data type definitions and data type semantics definitions from the Information Model; the encodings of these data types

Terms used in this document that are defined in the Terminology section of the IPFIX Protocol document are to be interpreted as defined there. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

This section describes the mechanism used to encode Information Element type information within an IPFIX Message stream. This mechanism consists of an Options Template Record used to define Information Element type records, and a set of Information Elements required by these type records. We first specify the necessary Information Elements, followed by the structure of the Options Template describing the type records. Note that Information Element type records require one Information Element, informationElementId, that is defined in the PSAMP Information Model. This Information Element supports references only to IANA-defined Information Elements; the privateEnterprseNumber Information Element

A description of the abstract data type of an IPFIX information element. These are taken from the abstract data types defined in section 3.1 of the IPFIX Information Model; see that section for more information on the types described below. This field may take the following values: Value Description 0x00octetArray 0x01unsigned8 0x02unsigned16 0x03unsigned32 0x04unsigned64 0x05signed8 0x06signed16 0x07signed32 0x08signed64 0x09float32 0x0Afloat64 0x0Bboolean 0x0CmacAddress 0x0Dstring 0x0EdateTimeSeconds 0x0FdateTimeMilliseconds 0x10dateTimeMicroseconds 0x11dateTimeNanoseconds 0x12ipv4Address 0x13ipv6Address These types are registered in the IANA IPFIX Information Element Data Type subregistry. This subregistry is intended to assign numbers for type names, not to provide a mechanism for adding data types to the IPFIX Protocol, and as such requires a Standards Action to modify. unsigned8 TBD1 Proposed Section 3.1 of the IPFIX Information Model

A string containing a human-readable description of an Information Element. See the Security Considerations section for notes on string handling for Information Element type records. string TBD2 Proposed

A string containing the name of an Information Element. See the Security Considerations section for notes on string handling for Information Element type records. string TBD3 Proposed

Contains the inclusive low end of the range of acceptable values for an Information Element. unsigned64 quantity TBD4 Proposed

Contains the inclusive high end of the range of acceptable values for an Information Element. unsigned64 quantity TBD5 Proposed

A description of the semantics of an IPFIX information element. These are taken from the data type semantics defined in section 3.2 of the IPFIX Information Model; see that section for more information on the types described below. This field may take the following values; the special value 0x00 (default) is used to note that no semantics apply to the field; it cannot be manipulated by a Collecting Process or File Reader that does not understand it a priori. Value Description 0x00default 0x01quantity 0x02totalCounter 0x03deltaCounter 0x04identifier 0x05flags These semantics are registered in the IANA IPFIX Information Element Semantics subregistry. This subregistry is intended to assign numbers for semantics names, not to provide a mechanism for adding semantics to the IPFIX Protocol, and as such requires a Standards Action to modify. unsigned8 TBD6 Proposed Section 3.2 of the IPFIX Information Model

A description of the units of an IPFIX Information Element. These correspond to the units implicitly defined in the Information Element definitions in section 5 of the IPFIX Information Model; see that section for more information on the types described below. This field may take the following values; the special value 0x00 (none) is used to note that the field is unitless. Value Name Notes 0x0000none 0x0001bits 0x0002octets 0x0003packets 0x0004flows 0x0005seconds 0x0006milliseconds 0x0007microseconds 0x0008nanoseconds 0x00094-octet wordsfor IPv4 header length 0x000Amessagesfor reliability reporting 0x000Bhopsfor TTL 0x000Centriesfor MPLS label stack These types are registered in the IANA IPFIX Information Element Units subregistry; new types may be added on a First Come First Served basis. unsigned16 TBD7 Proposed Section 5 of the IPFIX Information Model

A private enterprise number, as assigned by IANA. Within the context of an Information Element Type record, this element can be used along with the informationElementId element to scope properties to a specific Information Element. To export type information about an IANA-assigned Information Element, set the privateEnterpriseNumber to 0, or do not export the privateEnterpriseNumber in the type record. To export type information about an enterprise-specific Information Element, export the enterprise number in privateEnterpriseNumber, and export the Information Element number with the Enterprise bit cleared in informationElementId. The Enterprise bit in the associated informationElementId Information Element MUST be ignored by the Collecting Process. unsigned32 identifier TBD8 Proposed Sections 3.2 and 3.4.1 of the IPFIX Protocol; section 8.2.3 of the PSAMP Information Model.

The Information Element Type Options Template attaches type information to Information Elements used within Template Records, as scoped to an Observation Domain within a Transport Session. This provides a mechanism for representing an IPFIX Information Model inline within an IPFIX Message stream. Data Records described by this template are referred to as Information Element type records. In deployments in which interoperability across vendor implementations of IPFIX is important, an Exporting Process exporting data using Templates containing enterprise-specific Information Elements SHOULD export an Information Element type record for each enterprise-specific Information Element it exports. Collecting Processes MAY use these type records to improve handling of unknown enterprise-specific Information Elements. Exporting Processes using enterprise-specific Information Elements to implement proprietary features MAY omit type records for those Information Elements. Information Element type records MUST be handled by Collecting Processes as scoped to the Transport Session in which they are sent; this facility is not intended to provide a method for the permanent definition of Information Elements. Similarly, for security reasons, type information for a given Information Element MUST NOT be re-defined by Information Element type records, and a Collecting Process MUST NOT allow an Information Element type record to replace its own internal definition of an Information Element. Information Element type records SHOULD NOT be duplicated in a given Observation Domain within a Transport Session. Once an Information Element type record has been exported for a given Information Element within a given Transport Session, all subsequent type records for that Information Element MUST be identical. Information Elements for which a Collecting Process receives conflicting semantic or type information MUST be ignored. Note that while this template MAY be used to export information about any Information Element, including those registered with IANA, Exporting Processes SHOULD NOT export any type records that could be reasonably assumed to duplicate type information available at the Collecting Process. This mechanism is not intended as a replacement for Exporting and Collecting Processes keeping up to date with changes to the IANA registry; such an update mechanism is out of scope for this document. The template SHOULD contain the following Information Elements as defined in the PSAMP Information Model and in this document, above: IE Description informationElementID The Information Element identifier of the Information Element described by this type record. This Information Element MUST be defined as a Scope Field. See the PSAMP Information Model for a definition of this field. privateEnterpriseNumber The Private Enterprise number of the Information Element described by this type record. This Information Element MUST be defined as a Scope Field. informationElementDataType The storage type of the specified Information Element. informationElementSemantics The semantic type of the specified Information Element. informationElementUnits The units of the specified Information Element. This element SHOULD be omitted if the Information Element is a unitless quantity, or a not a quantity or counter. informationElementRangeBegin The low end of the range of acceptable values for the specified Information Element. This element SHOULD be omitted if the beginning of the Information Element's acceptable range is defined by its data type. informationElementRangeEnd The high end of the range of acceptable values for the specified Information Element. This element SHOULD be omitted if the end Information Element's acceptable range is defined by its data type. informationElementName The name of the specified Information Element. informationElementDescription A human readable description of the specified Information Element. This element MAY be omitted in the interest of export efficiency.

Note that the informationElementSemantics values defined in section 3.2 of are primarily intended to differentiate semantic interpretation of numeric values, and that not all combinations of the informationElementDataType and informationElementSemantics Information Elements are valid; e.g., a counter cannot be encoded as an IPv4 address. The following are acceptable values of informationElementSemantics: Any value is valid for unsigned informationElementDataType values ("unsigned8", "unsigned16", "unsigned32", or "unsigned64"). Any value except "flags" is valid for signed informationElementDataType values ("signed8", "signed16", "signed32", or "signed64"). Any value except "identifier" or "flags" is valid for floating-point informationElementDataType values ("float32" or "float64"). Only "default" is valid for all other other informationElementDataType values ("octetArray", "boolean", "macAddress", "string", "dateTimeSeconds", "dateTimeMilliseconds", "dateTimeMicroseconds", "dateTimeNanoseconds", "ipv4Address", or "ipv6Address"). Information Element type records containing invalid combinations of informationElementSemantics and informationElementDataType MUST NOT be sent by Exporting Processes, and MUST be ignored by Collecting Processes. Future standards actions that modify the Information Element Data Type subregistry or the Information Element Semantics subregistry should contain a Data Type and Semantics Restrictions sections such as this one to define allowable combinations of type and semantics information.

The same security considerations as for the IPFIX Protocol apply. Attention must be paid to the handling of Information Element type records at the Collecting Process. Type information precedence rules defined above (a Collecting Process' current knowledge overrides type records; types are not redefinable during a session) are designed to minimize the opportunity for an attacker to maliciously redefine the data model. Note that Information Element type records may contain two strings describing Information Elements: informationElementName and informationElementDescription. IPFIX strings on the wire are length-prefixed and UTF-8 encoded, most often within an IPFIX variable-length Information Element, which mitigates the risk of unterminated-string attacks against IPFIX Collecting Processes. However, care should still be taken when handling strings within the type system of the Collecting Process. First, Collecting Processes should pay particular attention to buffer sizes converting between length-prefixed and null-terminated strings. Exporting Processes MUST NOT export, and Collecting Processes MUST ignore, any informationElementName or informationElementDescription content which contains null characters (U+0000) in order to ensure buffer and string lengths are consistent. Also, note that there is no limit to IPFIX string length beyond that inherent in the protocol. The maximum IPFIX string length is 65512 octets (maximum message length (65535), minus message header (16), minus set header (4), minus long variable length field (3)). Specifically, although the informationElementName of all IANA Information Elements at the time of this writing is less than about 40 octets, and the informationElementDescription is less than 4096 octets, either of these Information Elements may contain strings up to 65512 octets long.

This document specifies the creation of several new IPFIX Information Elements in the IPFIX Information Element registry as defined in section 3 above. IANA has assigned the following Information Element numbers for their respective Information Elements as specified below: Information Element Number TBD1 for the informationElementDataType Information Element Information Element Number TBD2 for the informationElementDescription Information Element Information Element Number TBD3 for the informationElementName Information Element Information Element Number TBD4 for the informationElementRangeBegin Information Element Information Element Number TBD5 for the informationElementRangeEnd Information Element Information Element Number TBD6 for the informationElementSemantics Information Element Information Element Number TBD7 for the informationElementUnits Information Element Information Element Number TBD8 for the privateEnterpriseNumber Information Element [NOTE for IANA: The text TBD1, TBD2, TBD3, TBD4, TBD5, TBD6, TBD7, and TBD8 should be replaced with the respective assigned Information Element numbers where they appear in this document.] IANA has created an Information Element Data Type subregistry for the values defined for the informationElementSemantics Information Element. Entries may be added to this subregistry subject to a Standards Action. [NOTE for IANA: Please create a new Information Element Data Type subregistry as specified in the paragraph above, with values taken from section 3.1 of this document.] IANA has created an Information Element Semantics subregistry for the values defined for the informationElementSemantics Information Element. Entries may be added to this subregistry subject to a Standards Action. [NOTE for IANA: Please create a new Information Element Semantics subregistry as specified in the paragraph above, with values taken from section 3.6 of this document.] IANA has created an Information Element Units subregistry for the values defined for the informationElementUnits Information Element. Entries may be added to this subregistry on an Expert Review basis. [NOTE for IANA: Please create a new Information Element Units subregistry as specified in the paragraph above, with values taken from section 3.7 of this document.]

Thanks to Paul Aitken and Gerhard Muenz for the detailed reviews, and to David Moore for first raising this issue to the IPFIX mailing list.

&rfc5101; &rfc5102; &rfc5477; &rfc2119; &rfc2434; &rfc3917; &rfc5470;

The following example illustrates how the type information extension mechanism defined in this document may be used to describe the semantics of enterprise-specific Information Elements. The Information Elements used in this example are as follows: initialTCPFlags, an example private IE 14, 1 octet, the TCP flags on the first TCP packet in the flow. unionTCPFlags, an example private IE 15, 1 octet, the union of the TCP flags on all packets after the first TCP packet in the flow. An Exporting Process exporting flows containing these Information Elements might use a Template like the following:

However, a Collecting Process receiving Data Sets described by this Template can only treat the enterprise-specific Information Elements as opaque octets; specifically, there is no hint to the collector that they contain flag information. To use the type information extension mechanism to address this problem, the Exporting Process would first export the Information Element Type Options Template described in section 3.9 above:

Then, the Exporting Process would then export two records described by the Example Information Element Type Options Template to describe the enterprise-specific Information Elements: