Syslog Sending Policy Messages

Generally, syslog senders would send all generated messages to targeted receivers. But there would be a different story in some circumstances. For some reason, syslog system may buffer messages before sending them further. When some important messages have been hold in sending queue for a long time, the senders would like to send those urgent messages timely even out of order they are in sending queue or they were generated. When transport senders are going to drain their sending queue, they would not be able to send all the messages timely. In this case, senders would filter out less important messages to reduce queue overload or store new coming messages persistently in disk for sending them later. These scenarios may be not rare in real world. When congestion control is applied to syslog transport, some mechanisms will be employed to adjust sending rate based on feedback approach. When network congestion occurs, in order to reduce package loss in transit and improve network throughout, sending rate will slow down. Congestion control increase the latency of messages sending and the chance sending queue is full. Message burst due to malicious behaviors is likely to exhaust sending queue, too. When senders process messages before sending them, something would have happend, e.g, network congestion occurs or intrusion led to log messages burst. So the event that senders process messages before sending them might be an alert to operators. Additionally, the information how senders process messages before sending them is of interest to receivers and helpful for audit, e.g, the information which messages are more important than others may be of value to receivers. For an instance, when receivers can identify the more important messages from the sender's perspective, it could handle this event timely when many messages near-simultaneously arrive. For another instance, taking into account that the sender has dropped which messages during which period of time may help us do a more accurate audit. This document defines special messages called Sending Policy messages to notify this kind of event and deliver this information.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].

Message refer to syslog message whose on-the-wire format is defined in section 6, [RFC5424]. Sender refer to transport sender which is defined in section 3, [RFC5424]. Receiver refer to transport receiver which is defined in section 3, [RFC5424]. Sending queue is a conceptual buffer for buffering messages to be sent in sender's side. Receiving queue is a conceptual buffer for buffering messages to be received and processed in receiver's side.

A Sending Policy contains an action senders take on messages before sending them and criteria which are used to evaluate the importance of messages. Actions should be taken based on criteria.

According to what kind of actions senders would take, Sending Policies can be probably divided into three category:

When latency occurs, senders would like to send the more important messages timely even out of order the messages were generated or are in sending queue. How to evaluate the importance of messages depend on what criteria senders adopts.The criteria are of interest to the receiver, as the receiver can also pick up important messages to be processed timely when it starves, i.e, when so many near-simultaneously messages arrive. Priority Policy indicates messages would be sent based on level of importance even out of order they were generated or are in sending queue. Priority Policy also contains the criteria adopted to evaluate the importance of messages.

When the number of messages to be sent increases, it is possible to drain the sending queue. Senders may simply dropping messages to alleviate sending queue load. Comparing to messages dropped by routers in transit, senders has more sense what messages they'd like to abondon whilst the others preserved. We assume senders always drop less important messages. This policy may do not affect receiver's behavior very much. But when audit later, it would do more accurate analysis by taking criteria which are used to evaluate importance of messages into account. Filtering Policy indicates some messages would be dropped based on their importance. Filtering Policy also contains the criteria adopted to evaluate the importance of messages.

As an alternative to Filtering Policy, Senders may want to store the messages persistently for more reliability but less timeliness. This policy may not affect receiver's bahavior very much. The event that this policy has been applied may be of interest to receivers. It would be considered as an alert indicating congestion occurs or messages burst.

Persistency Policy do not care about importance of messages, since no messages dropping and no messages sent out of order. With regard to Priority Policy and Filtering Policy, they both need to identify the more important messages to be processed before sending them. There are commonly 2 ways to evaluate importance of syslog messages:

Priority Value consists of two parts: Facility and Severity. Facility value takes value from 0 to 23. Severity value takes value from 0 to 6. In both cases, the lower the value is, the more important the message is.

Timestamp Value is not mandatory in syslog messages, but when it shows up, it indicates the time when message was generated. When we filter out messages, we can simply dropping new coming messages or let new coming messages override the old ones(e.g, circular queue). The latter takes newer message more valuable, but the former takes newer message less important. Priority Policy would not base on these criteria.

Sending Policy Messages are used to contain policies specified in section 4 and the event when specific policy was applied or revoked. These information are carried as Sending Policy Blocks in Sending Policy Messages actually. There is a need to distinguish the Sendig Policy Block itself from the syslog message that is used to carry a Sending Policy Block. Sending Policy Blocks MUST be encompassed within completely formed syslog messages. Syslog messages that contain a Sending Policy Block are also referred to as Sending Policy messages. A Sending Policy message is identified by the presence of an SD ELEMENT with an SD-ID with the value "sending-policy". In addition, a Sending Policy message MUST contain valid APP-NAME, PROCID, and MSGID fields to be compliant with [RFC5424]. This specification does not mandate particular values for these fields; however, for consistency, originators MUST use the same values for APP-NAME, PROCID, and MSGID fields for every Sending Policy message that is sent, whichever values are chosen. To allow for the possibility of multiple originators per host, the combination of APP-NAME, PROCID, and MSGID MUST be unique for each such originator. If an originator daemon is restarted, it MAY use a new PROCID for what is otherwise the same originator but MUST continue to use the same APP-NAME and MSGID. The Sending Policy Block is carried as Structured Data within the Sending Policy Block message, per the definitions that follow in the subsection. The MSG part of a Sending Policy message SHOULD be any readable text for explaining the Sending Policy Block.

The Sending Policy Block MUST be encoded as an SD ELEMENT, as defined in [RFC5424]. The SD-ID MUST have the value of "sending-policy". The SDE contains the fields of the Sending Policy Block encoded as SD Parameters, as specified in the following. The Sending Policy Block is composed of the following fields. The value of each field MUST be printable ASCII. Field SD-PARAM-NAME Size in octets Version VER 2 Type TYPE 1 TimeType TT 1 TimeValue TV 1-50 Criteria CRI 1 Threshold THRE 1-10 The fields MUST be provided in the order listed if they were present. Each SD parameter MUST occur once in the Sending Policy Block. Version is mandatory. A Sending Policy Block is likely to be accordingly encoded as follows, where xxx denotes a placeholder for the particular values: [sending-policy VER="xxx" TYPE="xxx" TT="xxx" TV="xxx" CRI="xxx" THRE="xxx"] Values of the fields constitute SD parameter values and are hence enclosed in quotes, per [RFC5424]. The fields are separated by single spaces and are described in the subsequent subsections.

The Version field is a numeric value that has a length of 2 octets, which may include leading zeroes. The two octets contain a decimal character in the range of "0" to "9". The value in this field specifies the version of this Block with "01" as the value for the protocol version that is described in this document.

This field indicates which type of policies specified in section 4 the sender has adopted. The TYPE parameter may take any value from 0-2 inclusive. "0" -- Priority Policy has been adopted by the sender. "1" -- Filtering Policy has been adopted by the sender. "2" -- Persistency Policy has been adopted by the sender.

The two fields show us when the sender starts to apply the policy indicated in TYPE field or when the sender revokes it, that is dependent on TimeType Value. TT parameter may take any value from 0-1 inclusive "0" -- Subsequent TimeValue field represent start time. "1" -- Subsequent TimeValue field represent end time. When Priority Policy was applied, if TT Value equals "0", TV Value SHOULD be the TIMESTAMP Value of the first message out of order; if TT Value equals "1", TV Value SHOULD be the TIMESTAMP Value of the last message out of order. When Filtering Policy was applied, if TT Value equals "0", TV Value SHOULD be the TIMESTAMP Value of the first message dropped; if TT Value equals "1", TV Value SHOULD be the TIMESTAMP Value of the last message dropped. When Persistency Policy was applied, if TT Value equals "0" or "1", TV Value SHOULD be the system time when the Policy message was sent(I.e., TV Value equals to TIMESTAMP Value in message HEADER). sending-policy SDE can convey only criteria for importance evaluation, in this case, no indicator for policy has been applied or revoked, so this two field can be absent. TV parameter is a formalized timestamp derived from [RFC3339] and follow the restrictions specified in [RFC5424](section 6.2.3).

Criteria field indicates the criteria which are used to evaluate the importance of messages. Criteria parameter may take any value from 0-2 inclusive. "0" -- Evaluate importance of a message based on Severity Value which is a part of PRI Value contained in HEADER part. "1" -- Evaluate importance of a message based on Facility Value which is a part of PRI Value contained in HEADER part. "2" -- Evaluate importance of a message based on Timestamp Value contained in HEADR part. When Criteria field takes value "2", Threshold Value indicates whether earlier or older messages are considered more important. In this case, Threshold Value can be intepreted in this way: "0" -- the earlier messages were generated, more important they were. "1" -- the older messages were generated, more important they were. When Criteria field takes value "0" or "1", Threshold Value indicates the border of more important messages and less important messages. E.g, when Criteria parameter takes value "0", and, Threshold Value equals "3", that means any messages bearing severity Value greater than 3 may be dropped(when Filter Policy was adopted) or delayed their sending(when Priority Policy was adopted). When Persistency Policy was adopted, these two fields can be absent.

Sending Policy messages can only convey criteria for importance evaluation, in this case, Version, CRI and THRE MUST be present. E.g., senders would send this messages following the session is established. The following example shows the criteria are that messages bearing Severity Value which is less than or equals to 3 are more important. [sending-policy VER="01" CRI="0" THRE="3"] When Sending Policy messages are used as event notification, TT and TV MUST be present, CRI and THRE MUST be present unless criteria for importance evaluation have been delivered before. two Sending Policy messages are recommended to indicate event start and event end respectively. In this case, Sending Policy messages SHOULD have enough high priority to be sent timely and avoid being filtered out. The following example shows Filtering Policy has been adopted by the sender and the first affected message(i.e. the first message dropped) was generated on Feb 24 2009 05:14:15am, 3 microseconds into the next second( and the local time is -7 hours from UTC). The CRI and THRE are interpreted as last example [sending-policy VER="01" TYPE="1" TT="0" TV="2009-02-24T05:14:15.000003-07:00" CRI="0" THRE="3"]

This specification shares the same security considerations addressed in section 8.5, [RFC5424]. Additionally, two issues should be taken care of.

When Priority Policy was applied, syslog messages may be sent out of order they were generated. The receivers should not reconstruct sequence of events based on the order they were received. In this case, TIMESTAMP may be used to do this work. Besides, when Priority Policy message is used as real-time event notification, syslog-sign may not work well in this case, since the urgent message sent out of order can not be verified until the corresponding Signature message has been interpreted. By the same token, receivers would not process messages out of order they were received.

When Filtering Policy was applied, syslog messages may be lost by senders. In this case, Sending Policy messages could tell what kind of messages was dropped. If syslog-sign is also applied, syslog-sign can tell how many and which messages was dropped. It is recommended that Signature Block messages have enough high priority to avoid being filtered out by senders easily. If UDP-like congestion control is adopted for syslog transport, Sending Policy messages themselves would probably be lost in transit. Sending Policy messages should be sent several times in this case.

IANA is requested to register the SD-ID value "sending-policy" together with the PARAM-NAME values specified in Section 5.1 in the registry for SYSLOG structured data id values according to section 9 in [RFC5424]. SD-ID PARAM-NAME sending-policy VER mandatory TYPE optional TT optional TV optional CRI optional THRE optional

[RFC5424] Gerhards, R., "The syslog Protocol", RFC 5424, December 2008. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC3339] Klyne, G. and C. Newman, "Date and Time on the Internet: Timestamps", RFC 3339, July 2002.