Reed-Solomon Forward Error Correction (FEC) Schemes for FECFRAME

The use of Forward Error Correction (FEC) codes is a classic solution to improve the reliability of unicast, multicast and broadcast Content Delivery Protocols (CDP) and applications . The document describes a generic framework to use FEC schemes with media delivery applications and in particular with real-time or streaming media applications based on the RTP real-time protocol. Similarly the document describes a generic framework to use FEC schemes with with objects (e.g., files) delivery applications based on the ALC and NORM reliable multicast transport protocols. For instance, the and FEC schemes introduce erasure codes based on sparse parity check matrices for object delivery protocols like ALC and NORM. These codes are efficient in terms of processing but not optimal in terms of erasure recovery capabilities when dealing with "small" objects. The Reed-Solomon FEC codes described in this document belong to the class of Maximum Distance Separable (MDS) codes that are optimal in terms of erasure recovery capability. It means that a receiver can recover the k source symbols from any set of exactly k encoding symbols. However they are limited in terms of maximum source block size and number of encoding symbols. Since the real-time constraints of media delivery applications usually limit the maximum source block size, this is not considered to be a major issue in the context of the FEC Framework for many (but not necessarily all) use-cases. Additionally, if the encoding/decoding complexity is higher with Reed-Solomon codes than it is with or codes, it remains reasonable for most use-cases, even in case of a software codec. Many applications dealing with reliable content transmission or content storage already rely on packet-based Reed-Solomon codes. In particular, many of them use the Reed-Solomon codec of Luigi Rizzo . The goal of the present document is to specify Reed-Solomon schemes that are compatible with this codec. The document introduced such Reed-Solomon codes, and the FEC schemes are compatible with the framework. The present document inherits from the specification of the core Reed-Solomon codes based on Vandermonde matrices, and specifies FEC schemes that are compatible with the framework. Therefore this document specifies only the information specific to the FECFRAME context and refers to for the core specifications of the codes. The present document introduces: the Fully-Specified FEC Scheme with FEC Encoding ID XXX that specifies the use of Reed-Solomon codes over GF(2^^m), with m in {2..16}, a global FEC encoding and for arbitrary packet flows; the Fully-Specified FEC Scheme with FEC Encoding ID XXX that specifies the use of Reed-Solomon codes over GF(2^^m), with m in {2..16}, the general case FEC encoding and for a single sequenced flow; the Fully-Specified FEC Scheme with FEC Encoding ID XXX is similar to Scheme XXX except that it is for a single sequenced flow; the Fully-Specified FEC Scheme with FEC Encoding ID XXX is similar to Scheme XXX except that it is for a single sequenced flow; Distinguishing FEC schemes with a global FEC encoding and FEC schemes with the general case FEC encoding derives from the small block nature of Reed-Solomon codes over GF(2^^^8), the default value. With the general case encoding, protecting a single source data payload block can require to perform several independent FEC encodings, over different sub-blocks. A dedicated interleaving solution is then used to assign the various symbols of a given source packet to the sub-blocks in an optimal way, so as to guaranty the highest possible erasure recovery capabilities.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 .

This document uses the same terms and definitions as those specified in . Additionally, it uses the following definitions: Source symbol: unit of data used during the encoding process. Encoding symbol: unit of data generated by the encoding process. With systematic codes, source symbols are part of the encoding symbols. Repair symbol: encoding symbol that is not a source symbol. Code rate: the k/n ratio, i.e., the ratio between the number of source symbols and the number of encoding symbols. The code rate belongs to a ]0; 1] interval. A code rate close to 1 indicates that a small number of repair symbols have been produced during the encoding process. Systematic code: FEC code in which the source symbols are part of the encoding symbols. The Reed-Solomon codes introduced in this document are systematic. Source block: a block of k source symbols that are considered together for the encoding. Source Packet: a data packet containing only source symbols. Repair Packet: a data packet containing only repair symbols. Packet Erasure Channel: a communication path where packets are either dropped (e.g., by a congested router, or because the number of transmission errors exceeds the correction capabilities of the physical layer codes) or received. When a packet is received, it is assumed that this packet is not corrupted. Source Data Flow: a flow of data payloads coming from an application (e.g., using an RTP encapsulation), and to which FEC protection is applied. Source Data Payload: a unit of data that belongs to a source data flow. Source Data Payload Block: a set of source data payloads that are considered together by the FECFRAME instance. This is the set of symbols over which FEC encoding will be performed, either in a global way, or separately, over each sub-block. Sub-Block: a block of k source symbols, in the general case FEC encoding scheme. FEC encoding is performed independently over each sub-block.

This document uses the following notations: k denotes the number of source symbols in a source block. max_k denotes the maximum number of source symbols for any source block. n_r denotes the number of repair symbols generated for a source block. n denotes the encoding block length, i.e., the number of encoding symbols generated for a source block. Therefore: n = k + n_r. B denotes the number of source data payloads per source block. max_B denotes the maximum number of source data payloads for any source block. E denotes the encoding symbol length in bytes. S denotes the symbol size in units of m-bit elements. When m = 8, then S and E are equal. m defines the length of the elements in the finite field, in bits. In this document, m belongs to {2..16}. q defines the number of elements in the finite field. We have: q = 2^^m in this specification. CR denotes the "code rate", i.e., the k/n ratio. a^^b denotes a raised to the power b.

This document uses the following abbreviations: ESI stands for Encoding Symbol ID. FFCI stands for FEC Framework Configuration Information. RS stands for Reed-Solomon. MDS stands for Maximum Distance Separable code. GF(q) denotes a finite field (also known as Galois Field) with q elements. We assume that q = 2^^m in this document.

This section introduces the procedures that are used during the source block creation, and that are common to all the FEC schemes introduced in this document.

Several aspects must be considered, that impact the source block creation: the distribution of source data payload sizes; the maximum source block size (k parameter) and encoding block size (n parameter), that are constrained by the finite field size (m parameter); the potential real-time constraints, that impact the maximum encoding block size, since the larger the block size, the larger the decoding delay; We now detail each of these aspects. In its most general form the FECFRAME framework and the RS FEC schemes are meant to protect a set of independent flows. Since the flows have no relationship to one another, the source data payloads of each flow will potentially vary significantly. Even in the special case of a single flow, the source data payload sizes may largely vary (e.g., the various frames of a "Group of Pictures (GOP) of an H.264 flow can have different sizes). This diversity must be addressed by the source block creation procedure since the RS FEC scheme requires a constant encoding symbol size (E parameter). The finite field size parameter, m, defines the number of non zero elements in this field which is equal to: q - 1 = 2^^m - 1. This q - 1 value is also the theoretical maximum number of encoding symbols that can be produced for a source block. For instance, when m = 8 (default) there is a maximum of 2^^8 - 1 = 255 encoding symbols. So: k < = n < = 255. Given the target FEC code rate (e.g., provided by the developer when starting the FECFRAME framework, and taking into account the (known or estimated) packet loss rate), the sender calculates: max_k = floor((2^^m - 1) * CR) This max_k value leaves enough room for the sender to produce the desired number of repair symbols. The source flows can have real-time constraints. It means that the maximum number of source transport payloads of a source block, which directly impacts the decoding delay, must not exceed a certain threshold. It is the role of the developer, who knows the flow real-time features, to define an appropriate upper bound to the source data payload block size, max_B. Another aspect is the appropriate way of performing FEC encoding over the source data payload block. Depending of the actual situation, two schemes are feasible: There can be situations where a sender needs to protect a small number of source data payloads. In that case, all source data payloads, along with additional information (flow ID, length, and padding), are virtually split into symbols and a global FEC encoding is performed. There can be situations where a sender needs to protect a large number of source data payloads. In that case the number of symbols is large and can easily exceed the max_k value. For this kind of situation, the present document introduces a general case encoding scheme, which defines sub-blocks over which independent FEC encoding is performed. These two encoding schemes are introduced in the following sections.

With the global encoding scheme, the source data payload block is encoded as a single source block. There are a total of B < = max_B source data payloads. For the source data payload i, with 0 < = i < = B-1, 3 bytes are prepended (): The first byte, FID[i] (Flow ID), contains the integer identifier associated to the source flow to which this source data payload belongs to. It is assumed that a single byte is sufficient, or said differently, that no more than 256 flows will be protected by a single instance of the FECFRAME framework. The following two bytes, L[i] (Length), contain the length of this source data payload, in network byte order (i.e., big endian). This length is for the data payload itself, without considering FID[i], L[i], or Pad[i]. Zero padding is also added, in field Pad[i], if needed, for alignment purposes on symbol boundaries. This can happen at most once per source data payload. Thanks to this padding, a symbol will never straddle several source data payloads, nor several source packets.

Enc Symbol Len (E) Enc Symbol Len (E) Enc Symbol Len (E) < ------------------ >< ------------------ >< ------------------ > +----+----+-----------------------+--------+ |F[0]|L[0]| R[0] | Pad[1] | +----+----+----------+------------+--------+ |F[1]|L[1]| R[1] | +----+----+----------+--------------------------------------+----+ |F[2]|L[2]| R[2] |P[2]| +----+----+----------+--------------------------------------+----+ |F[3]|L[3]| R[3] | P3| +----+----+------+---+ \_______________________________ _______________________________/ \/ global FEC encoding +--------------------+ | Repair 4 | +--------------------+ | Repair 5 | +--------------------+ | Repair 6 | +--------------------+ | Repair 7 | +--------------------+ Note that neither the initial 3 bytes nor the optional padding is sent over the network. However, they are considered when performing FEC encoding, which means that a receiver who lost a certain UDP datagram (if UDP is used) will be able to recover both the source data payload and the initial 3 bytes, as long as FEC decoding succeeds.

With the general case encoding scheme, the source data payload block is potentially split into several sub-blocks. Each sub-block leads to a different FEC encoding and the appropriate number of repair symbols are generated, for each sub-block, as specified by the target code rate. This constraint comes from the fact that the number of source symbols in the source data payload block exceeds the max_k value (). In this section we define an interleaving approach to distribute the source symbols of the various packets to sub-blocks in order to guaranty the best possible erasure protection. Note that this solution is preferable to the alternative approach consisting in considering smaller source data payload blocks, from an erasure recovery capability point of view. Note also that the source block creation with the global encoding scheme () can also be regarded as a special case, with a single sub-block. For the source data payload i, with 0 < = i < = B-1, 3 bytes are prepended and an optional zero padding appended, as in . Let: s[i] be the number of size of the source data payload i, along with the F[i], L[i], and Pad[i] fields, in units of symbols. k_tot be the total number of symbols in this source data payload block. In other words, k_tot is the sum of all s[i], with 0 < = i < = B-1. NSB be the number of sub-blocks for this source data payload block. NSB = Ceil(k_tot / max_k). Here we assume that k_tot > = max_k (the case where it is equal corresponds to ). The key point here is that maximum erasure recover capabilities require that the source symbols coming from a certain source packet be spread over the largest possible number of sub-blocks. Thereby, the loss of a single source packet will trigger a single symbol erasure on the corresponding sub-blocks, instead of deeply impacting a single sub-block which might compromise FEC decoding. This requirement leads to the definition of an appropriate interleaving scheme. Let us consider the example of .

+----------+ src pkt 0: | symb 0.0 | +----------+ +----------+ src pkt 1: | symb 1.0 | +----------+ +----------+----------+ src pkt 2: | symb 2.0 | symb 2.1 | +----------+----------+ +----------+----------+ src pkt 3: | symb 3.0 | symb 3.1 | +----------+----------+ +----------+----------+----------+ src pkt 4: | symb 4.0 | symb 4.1 | symb 4.2 | +----------+----------+----------+ The source data payload block consists of five packets (B = 5), whose size (in unit of symbols) is respectively 1, 1, 2, 2, and 3 symbols. Therefore k_tot = 9 symbols. If max_k = 3 symbols, then NSB = Ceil(9/3) = 3 sub-blocks. A good interleaving consists in creating the following sub-blocks: SB0 = {2.0; 3.0; 4.0}, SB1 = {2.1; 3.1; 4.1}, and SB2 = {0.0; 1.0; 2.0}. If the source packet 4 is lost during transmission, then it leads to a single symbol loss in the three sub-blocks, which will easily be recovered after FEC decoding. ----- Editor's note: The exact optimal interleaving algorithm is TBD. ----- Here also, neither the initial 3 bytes nor the optional padding is sent over the network. However, they are considered when performing FEC encoding, which means that a receiver who lost a certain UDP datagram (if UDP is used) will be able to recover both the source data payload and the initial 3 bytes, as long as FEC decoding succeeds.

The FEC Framework Configuration Information (or FFCI) includes information that MUST be communicated between the sender and receiver(s) for the FEC scheme to operate. It includes both mandatory elements and scheme-specific elements, as detailed below.

FEC Encoding ID: the value assigned to this fully-specified FEC scheme MUST be XXX, as assigned by IANA (). When SDP is used to communicate the FFCI, this FEC Encoding ID is carried in the 'encoding-id' parameter.

The FEC Scheme Specific Information (FSSI) includes elements that are specific to the present FEC scheme. More precisely: Encoding symbol length (E): a non-negative integer indicating the length of each encoding symbol in bytes. m parameter: it defines the length of the elements in the finite field, in bits. In this scheme, m belongs to {2..16}. The encoding format is the following 3 octet field:

0 1 2 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Encoding Symbol Length (E) | m | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ These elements are required both by the sender (RS encoder) and the receiver(s) (RS decoder). When SDP is used to communicate the FFCI, this FEC Encoding ID is carried in the 'fssi' parameter as an opaque octet string.

A source packet MUST contain an Explicit Source FEC Payload ID that is appended to the end of the packet as illustrated in . It contains the original source data payload, and therefore it can span several source symbols.

+--------------------------------+ | IP Header | +--------------------------------+ | Transport Header | +--------------------------------+ | Source Data Payload | +--------------------------------+ | Explicit Source FEC Payload ID | +--------------------------------+ More precisely, the Explicit Source FEC Payload ID is composed of the Source Block Number and the Encoding Symbol ID. The length of these two fields depends on the m parameter (which is transmitted separately in the FFCI, see below): The Source Block Number (SBN) (32-m bit field) identifies the source block to which the source packet belongs. There are a maximum of 2^^(32-m) blocks before a wrapping of this field occurs. The Encoding Symbol ID (ESI) (m bit field) identifies the first source symbol associated to this source packet in the source block. There are a maximum of 2^^m encoding symbols per block. The first k values (0 to k - 1) identify source symbols.

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Block Number (32-8=24 bits) | Enc. Symb. ID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Block Nb (16 bits) | Enc. Symbol ID (16 bits) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The format of the FEC Payload ID for m = 8 and m = 16 is illustrated in and respectively.

A repair packet MUST contain a Repair FEC Payload ID that is prepended to the repair symbol(s) as illustrated in . There can be several repair symbols per repair packet.

+--------------------------------+ | IP Header | +--------------------------------+ | Transport Header | +--------------------------------+ | Repair FEC Payload ID | +--------------------------------+ | Repair Symbol | +--------------------------------+ More precisely, the Repair FEC Payload ID is composed of the Source Block Number, the Encoding Symbol ID and the Source Block Length. The length of these fields depends on the parameter m (which is transmitted separately in the FFCI, see below): The Source Block Number (SBN) (32-m bit field) identifies the source block to which the repair packet belongs. There are a maximum of 2^^(32-m) blocks before a wrapping of this field occurs. The Encoding Symbol ID (ESI) (m bit field) identifies the first repair symbol contained in this repair packet. There are a maximum of 2^^m encoding symbols per block. The last n-k values (k to n - k - 1) identify repair symbols. The Source Block Length (SBL) (16 bit field) provides the actual size of the source block, in number of symbols. If 16 bits are too much when m < = 8, it is needed when 8 < m < = 16.

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Block Nb (16 bits) | Enc. Symbol ID (16 bits) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Block Length (SBL) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ The format of the FEC Payload ID for m = 8 and m = 16 is illustrated in and respectively.

The source block creation procedures are specified in . More precisely: the length indication for the source data payload i, used to compute the L[i] field, is the actual length of the source data payload i. It MUST NOT include FID[i], L[i], Pad[i], nor the Explicit Source FEC Payload ID. The SBN value is incremented for each new source block, starting at 0 for the first block. Wrapping to zero will happen for long sessions, after value 2^^(32-m)-1. The ESI of source symbols are managed sequentially, starting at 0 for the first symbol.

The present document inherits from the specification of the core Reed-Solomon codes based on Vandermonde matrices.

The FEC Framework Configuration Information (or FFCI) includes information that MUST be communicated between the sender and receiver(s) for the FEC scheme to operate. It include both mandatory elements and scheme-specific elements, as detailed below.

TBD

A source packet MUST contain an Explicit Source FEC Payload ID that is appended to the end of the packet as illustrated in . TBD

A repair packet MUST contain a Repair FEC Payload ID that is prepended to the repair symbol(s) as illustrated in . TBD

The source block creation procedures are specified in . More precisely: the length indication for the source data payload i, used to compute the L[i] field, is the actual length of the source data payload i. It MUST NOT include FID[i], L[i], Pad[i], nor the Explicit Source FEC Payload ID. The SBN value is incremented for each new source block, starting at 0 for the first block. Wrapping to zero will happen for long sessions, after value 2^^(32-m). The ESI of source symbols are managed sequentially, starting at 0 for the first symbol.

The present document inherits from the specification of the core Reed-Solomon codes based on Vandermonde matrices.

TBD

----- Editor's note: Section To Be Updated... -----

A content delivery system is potentially subject to many attacks: some of them target the network (e.g., to compromise the routing infrastructure, by compromising the congestion control component), others target the Content Delivery Protocol (CDP) (e.g., to compromise its normal behavior), and finally some attacks target the content itself. Since this document focuses on a FEC building block independently of any particular CDP (even if ALC and NORM are two natural candidates), this section only discusses the additional threats that an arbitrary CDP may be exposed to when using this building block. More specifically, several kinds of attacks exist: those that are meant to give access to a confidential content (e.g., in case of a non-free content), those that try to corrupt the object being transmitted (e.g., to inject malicious code within an object, or to prevent a receiver from using an object), and those that try to compromise the receiver's behavior (e.g., by making the decoding of an object computationally expensive). These attacks can be launched either against the data flow itself (e.g. by sending forged symbols) or against the FEC parameters that are sent either in-band (e.g., in an EXT_FTI or FDT Instance) or out-of-band (e.g., in a session description).

First of all, let us consider the attacks against the data flow.

Access control to the object being transmitted is typically provided by means of encryption. This encryption can be done over the whole object (e.g., by the content provider, before the FEC encoding process), or be done on a packet per packet basis (e.g., when IPSec/ESP is used ). If access control is a concern, it is RECOMMENDED that one of these solutions be used. Even if we mention these attacks here, they are not related nor facilitated by the use of FEC.

Protection against corruptions (e.g., after sending forged packets) is achieved by means of a content integrity verification/sender authentication scheme. This service can be provided at the object level, but in that case a receiver has no way to identify which symbol(s) is(are) corrupted if the object is detected as corrupted. This service can also be provided at the packet level. In this case, after removing all forged packets, the object may be in some case recovered. Several techniques can provide this source authentication/content integrity service: at the object level, the object MAY be digitally signed (with public key cryptography), for instance by using RSASSA-PKCS1-v1_5 . This signature enables a receiver to check the object integrity, once this latter has been fully decoded. Even if digital signatures are computationally expensive, this calculation occurs only once per object, which is usually acceptable; at the packet level, each packet can be digitally signed. A major limitation is the high computational and transmission overheads that this solution requires (unless Elliptic Curve Cryptography (ECC) is used, but ECC is the subject of proprietary patents). To avoid this problem, the signature may span a set of symbols (instead of a single one) in order to amortize the signature calculation. But if a single symbol is missing, the integrity of the whole set cannot be checked; at the packet level, a Group Message Authentication Code (MAC) scheme can be used, for instance by using HMAC-SHA-1 with a secret key shared by all the group members, senders and receivers. This technique creates a cryptographically secured (thanks to the secret key) digest of a packet that is sent along with the packet. The Group MAC scheme does not create prohibitive processing load nor transmission overhead, but it has a major limitation: it only provides a group authentication/integrity service since all group members share the same secret group key, which means that each member can send a forged packet. It is therefore restricted to situations where group members are fully trusted (or in association with another technique as a pre-check); at the packet level, TESLA is a very attractive and efficient solution that is robust to losses, provides a true authentication/integrity service, and does not create any prohibitive processing load or transmission overhead. Yet checking a packet requires a small delay (a second or more) after its reception; Techniques relying on public key cryptography (digital signatures and TESLA during the bootstrap process, when used) require that public keys be securely associated to the entities. This can be achieved by a Public Key Infrastructure (PKI), or by a PGP Web of Trust, or by pre-distributing the public keys of each group member. Techniques relying on symmetric key cryptography (group MAC) require that a secret key be shared by all group members. This can be achieved by means of a group key management protocol, or simply by pre-distributing the secret key (but this manual solution has many limitations). It is up to the developer and deployer, who know the security requirements and features of the target application area, to define which solution is the most appropriate. Nonetheless, in case there is any concern of the threat of object corruption, it is RECOMMENDED that at least one of these techniques be used.

Let us now consider attacks against the FEC parameters (or FEC OTI). The FEC OTI can either be sent in-band (i.e., in an EXT_FTI or in an FDT Instance containing FEC OTI for the object) or out-of-band (e.g., in a session description). Attacks on these FEC parameters can prevent the decoding of the associated object: for instance modifying the B parameter will lead to a different block partitioning at a receiver thereby compromising decoding; or setting the m parameter to 16 instead of 8 with FEC Encoding ID 2 will increase the processing load while compromising decoding. It is therefore RECOMMENDED that security measures be taken to guarantee the FEC OTI integrity. To that purpose, the packets carrying the FEC parameters sent in-band in an EXT_FTI header extension SHOULD be protected by one of the per-packet techniques described above: digital signature, group MAC, or TESLA. When FEC OTI is contained in an FDT Instance, this object SHOULD be protected, for instance by digitally signing it with XML digital signatures . Finally, when FEC OTI is sent out-of-band (e.g., in a session description) this latter SHOULD be protected, for instance by digitally signing it. The same considerations concerning the key management aspects apply here also.

Values of FEC Encoding IDs and FEC Instance IDs are subject to IANA registration. TBD

The authors want to thank Hitoshi Asaeda for his valuable comments.