next up previous contents index
Next: The icmp Analyzer Up: Analyzers and Events Previous: The SSL Analyzer   Contents   Index

Subsections


The weird Module

The weird module processes unusual or exceptional events. A number of these ``shouldn't'' or even ``can't'' happen, yet they do. The general design philosophy of Bro is to check for such events whenever possible, because they can reflect incorrect assumptions (either Bro's or the user's), attempts by attackers to confuse the monitor and evade detection, broken hardware, misconfigured networks, and so on.

Weird events are divided into three categories, namely those pertaining to: connections; flows (a pair of hosts, but for which a specific connection cannot be identified); and network behavior (cannot be associated with a pair of hosts). These categories have a total of four event handlers: conn_weird, conn_weird_addl, flow_weird, and net_weird, and in the corresponding sections below we catalog the events handled by each. In addition, we separately catalog the events generated by the standard scripts themselves (§ ). Finally, two more weird events have their own handlers, in order to associate detailed information with the event: rexmit_inconsistency and ack_above_hole.

weird_file is the logging file that the module uses to record exceptional events. It defaults to open_log_file("weird").

Note: While these events ``shouldn't'' happen, in reality they often do. For example, of the 73 listed below, a search of 10 months' worth of logs at LBNL shows that 42 were seen operationally. While some of the instances reflect attacks, the great majority are simply due to (i) buggy implementations, (ii) diverse use of the network, or (iii) Bro bugs or limitations. Accordingly, you may initially be inclined to log each instance, but don't be surprised to find that you soon decide to only record many of them in the weird file, or not record them at all. (For further discussion, see the section on ``crud'' in [Pa99].)


Actions for ``weird'' events


Table: Different types of possible actions to take for ``weird'' events.
Action Meaning
WEIRD_UNSPECIFIED No action specified.
WEIRD_IGNORE Ignore the event.
WEIRD_FILE Record the event to weird_file, if it has not been seen for these hosts before. (But see weird_do_not_ignore_repeats.)
WEIRD_LOG_ALWAYS Record the event to weird_file and generate a real-time alert each time the event occurs.
WEIRD_LOG_ONCE Record the event to weird_file; generate a real-time alert the first time the event occurs.
WEIRD_LOG_PER_CONN Record the event to weird_file; generate a real-time alert the first time it occurs for a given connection.
WEIRD_LOG_PER_ORIG Record the event to weird_file; generate a real-time alert the first time it occurs for a given originating host.


The general approach taken by the module is to categorize for each event the action to take when the event engine generates the event. Table [*] summarizes the different possible actions.

weird variables

The standard weird script provides the following redefinable variables:

[weird_action : table[string] of count] Maps different weird events to actions as given in Table [*].

Default: as specified in conn_weird, conn_weird_addl, flow_weird, net_weird, and § . As usual, you can change particular values using refinement. For example:

redef weird_action: table[string] of count += {
    [["bad_TCP_checksum", "bad_UDP_checksum"]] = WEIRD_IGNORE,
    ["fragment_overlap"] = WEIRD_LOG_PER_CONN,
};
would specify to ignore TCP and UDP checksum errors (rather than the default of WEIRD_FILE), and to alert on fragment overlaps once per connection in which they occur, rather than the default of WEIRD_LOG_ALWAYS.

[weird_action_filters : table[string] of function(c: connection): count] Indexed by the name of a weird event, yields a function that when called for a given connection exhibiting the event, returns an action from Table [*]. A return value of WEIRD_UNSPECIFIED means ``no special action, use the action you normally would.'' This variable thus allows arbitrary customization of the handling of particular events.

Default: empty, for the weird analyzer itself. The portmapper analyzer redefines this variable as follows:

    redef weird_action_filters += {
        [["bad_RPC", "excess_RPC", "multiple_RPCs", "partial_RPC"]] =
                RPC_weird_action_filter,
};
where RPC_weird_action_filter is a function internal to the analyzer that returns WEIRD_FILE if the originating host is in RPC_okay_nets, and WEIRD_UNSPECIFIED otherwise.

[weird_ignore_host : set[addr, string]] Specifies that the analyzer should ignore the given weird event (named by the second index) if it involves the given address (as either originator or responder host).

Default: empty.

[weird_do_not_ignore_repeats : set[string]] Gives a set of weird events that, if their action is WEIRD_FILE, should still be recorded to the weird_file each time they occur.

Default: the events relating to checksum errors, i.e., "bad_IP_checksum", "bad_TCP_checksum", "bad_UDP_checksum", and "bad_ICMP_checksum". These are recorded multiple times because it can prove handy to be able to track clusters of checksum errors.

weird functions

The weird analyzer includes the following functions:

[report_weird(t: time, name: string, id: string, action: count, no_log: bool) ] Processes an occurrence of the weird event name associated with the connection described by the string id (which may be empty if no connection is associated with the event). action is the action associated with the event. For report_weird, the only distinctions made between the different actions are that WEIRD_IGNORE causes the function to do nothing; any of WEIRD_LOG_XXX cause the function to log a message, unless no_log is true; and WEIRD_UNSPECIFIED causes the function to look up the action in weird_action. If the function does not find an action for the event, then it uses WEIRD_LOG_ALWAYS and prepends the log message with a pair of asterisks (``**'') to flag that this event does not have a specified action.

For WEIRD_FILE, report_weird only records the event once to the file, unless the given event is present in weird_do_not_ignore_repeats. Events with loggable actions are always recorded to weird_file.

[report_weird_conn(t: time, name: string, id: string, c: connection) ] Processes an occurrence of the weird event name associated with the connection c, which is described by the string id.

If report_weird_conn finds one of the hosts and the given event name in weird_ignore_host, then it does nothing. Then, if the event is in weird_action, then it looks up the event in weird_action_filters and invokes the corresponding function if present, otherwise taking the action from weird_action. It then implements the various flavors of WEIRD_LOG_XXX by not logging events more than once per connection, originator host, etc., though the events are still written to weird_file. Finally, the function invokes report_weird to do the actual recording and/or writing to weird_file.

[report_weird_orig(t: time, name: string, id: string, orig: addr) ] Processes an occurrence of the weird event name associated with the source address orig. id textually describes the flow from orig to the destination, for example using endpoint_id.

The function looks up the event name in weird_action and passes it along to report_weird.

Events handled by conn_weird

[conn_weird (name: string, c: connection)] Invoked for most ``weird'' events. name is the name of the weird event, and c is the connection with which it's associated.

conn_weird handles the following events, all of which have a default action of WEIRD_FILE:

[active_connection_reuse] A new connection attempt (initial SYN) was seen for an already-established connection that has not yet terminated. [bad_HTTP_reply] The first line of a reply from an HTTP server did not include HTTP/version. [bad_HTTP_version] The first line of a request from an HTTP client did not include HTTP/version. [bad_ICMP_checksum] The checksum field in an ICMP packet was invalid. [bad_rlogin_prolog] The beginning of an Rlogin connection had a syntactical error. [bad_RPC] A Remote Procedure Call was ill-formed. [bad_RPC_program] A portmapper RPC call did not include the correct portmapper program number. [bad_SYN_ack] A TCP SYN acknowledgment (SYN-ack) did not acknowledge the sequence number sent in the initial SYN. [bad_TCP_checksum] A TCP packet had a bad checksum. [bad_UDP_checksum] A UDP packet had a bad checksum. [baroque_SYN] A TCP SYN was seen with an unlikely combination of other flags (the URGent pointer). [blank_in_HTTP_request] The URL in an HTTP request includes an embedded blank. [connection_originator_SYN_ack] A TCP endpoint that originated a connection by sending a SYN followed this up by sending a SYN-ack. [data_after_reset] After a TCP endpoint sent a RST to terminate a connection, it sent some data. [data_before_established] Before the connection was fully established, a TCP endpoint sent some data. [excessive_RPC_len] An RPC record sent over a TCP connection exceeded 8 KB. [excess_RPC] The sender of an RPC request or reply included leftover data beyond what the RPC parameters or result value themselves consumed. [FIN_advanced_last_seq] A TCP endpoint retransmitted a FIN with a higher sequence number than previously. [FIN_after_reset] A TCP endpoint sent a FIN after sending a RST. [FIN_storm] The monitor saw a flurry of FIN packets all sent on the same connection. A ``flurry'' is defined as 1,000 packets that arrived with less than 1 sec between successive FINs. Deficiency: Clearly, this numbers should be user-controllable. [HTTP_unknown_method] The method in an HTTP request was not GET, POST or HEAD. [HTTP_version_mismatch] A persistent HTTP connection sent a different version number for a subsequent item than it did initially. [inappropriate_FIN] A TCP endpoint sent a FIN before the connection was fully established. [multiple_HTTP_request_elements] An HTTP request included multiple methods. [multiple_RPCs] A TCP RPC stream included more than one remote procedure call. [NUL_in_line] A NUL (ASCII 0) was seen in a text stream that is expected to be free of NULs. Currently, the only such stream is that associated with an FTP control connection. [originator_RPC_reply] The originator (and hence presumed client) of an RPC connection sent an RPC reply (either instead of a request, or in addition to a request). [partial_finger_request] When a Finger connection terminated, it included a final line of unanalyzed text because the text was not newline-terminated. [partial_ftp_request] When an FTP connection terminated, it included a final line of unanalyzed text because the text was not newline-terminated. [partial_ident_request] When an IDENT connection terminated, it included a final line of unanalyzed text because the text was not newline-terminated. [partial_portmapper_request] A portmapper connection terminated with an unanalyzed request because the data stream was incomplete. [partial_RPC] An RPC was missing some required header information due to truncation. [pending_data_when_closed] A TCP connection closed even though not all of the data in it was analyzed due to a sequence hole. [possible_split_routing] Bro appears to be seeing only one direction of some bi-directional connections (§ ). This can also occur due to certain forms of stealth-scanning. [premature_connection_reuse] A TCP connection tuple is being reused less than 30 sec after its previous use. (The standard requires waiting $2 \cdot \mbox{MSL} = 4$ minutes [RFC793, p. 27].) [repeated_SYN_reply_wo_ack] A TCP responder that replied to an initial SYN with a SYN-ack has subsequently sent a SYN without an acknowledgment. [repeated_SYN_with_ack] A TCP originator that sent an initial SYN has subsequently sent a SYN-ack. [responder_RPC_call] The responder (and hence presumed server) of an RPC connection sent an RPC request (either instead of a reply, or in addition to a reply). [rlogin_text_after_rejected] An Rlogin client sent additional text to an Rlogin server after the server already presumably rejected the client's service request. [RPC_rexmit_inconsistency] An RPC call was retransmitted, and the retransmitted call differed from the original call. This could reflect an attempt by an attacker to evade the monitor. Note: This type of inconsistency checking is not available for RPC replies because the transmission of the reply in general marks the end of the RPC connection, and the monitor deletes the connection state shortly afterward. [RST_storm] The monitor saw a flurry of RST packets all sent on the same connection. See FIN_storm for the definition of ``flurry.'' [RST_with_data] A TCP RST packet included data. This actually is allowed by the specification [RFC1122, 4.2.2.12]. Deficiency: This event should include the data. [simultaneous_open] The monitor saw a TCP simultaneous open, i.e., both endpoints sent initial SYNs to one another at the same time. While the specification allows this [RFC793, p. 30], none of the protocols analyzed by Bro should be using it. [spontaneous_FIN] A TCP endpoint sent a FIN packet without sending any previous packets. This event can reflect stealth-scanning, but can also occur when Bro has recently started up and has not seen other traffic on a connection and hence does not know that the connection already exists. [spontaneous_RST] A TCP endpoint sent a RST packet without sending any previous packets. As with spontaneous_FIN, this event can reflect either stealth scanning or a Bro start-up transient. [SYN_after_close] A TCP endpoint sent a SYN (connection initiation) after sending a FIN (connection termination), but before the connection fully closed. [SYN_after_partial] A TCP endpoint in a ``partial'' connection (§ ) sent a SYN. [SYN_after_reset] A TCP endpoint sent a SYN after sending a RST (reset connection). [SYN_inside_connection] A TCP endpoint sent a SYN during a connection (or partial connection) on which it had already sent data. [SYN_seq_jump] A TCP endpoint retransmitted a SYN or a SYN-ack, but with a different sequence number. [SYN_with_data] A TCP endpoint included data in a SYN packet it sent. Note, this can legitimately occur for T/TCP connections [RFC1644]. [TCP_christmas] A TCP endpoint sent a SYN packet that included the RST flag (a nonsensical combination). The term ``Christmas packet'' has been used in this context (particularly if other flags are set, too) because the packet's flags are ``lit up like a Christmas tree.'' [UDP_datagram_length_mismatch] The length field in a UDP header did not match the length field in the IP header. This could reflect an attempt by an attacker to evade the monitor. [unpaired_RPC_response] An RPC reply was seen for which no request was seen. This event could reflect a Bro start-up transient (it started running after the request was sent). [unsolicited_SYN_response] A TCP endpoint sent a SYN-ack without first receiving an initial SYN. This event could reflect a Bro start-up transient.

Events handled by conn_weird_addl

[conn_weird_addl (name: string, c: connection, addl: string)] Invoked for a few ``weird'' events that require an extra (string) argument to help clarify the event. Deficiency: It would likely be very handy if the general ``weird'' event handling was more flexible, with the ability to have various parameters associated with the events. Doing so will likely have to wait on general Bro mechanism for dealing with default parameters and/or polymorphic functions and event handlers.

conn_weird_addl handles the following events, all of which have a default action of WEIRD_FILE:

[bad_ident_reply] A reply from an IDENT server was syntactically invalid. [bad_ident_request] A request to an IDENT server was syntactically invalid. [ident_request_addendum] An IDENT request included additional text beyond that forming the request itself.

Events handled by flow_weird

[flow_weird (name: string, src: addr, dst: addr)] is invoked for ``weird'' events that cannot be associated with a particular connection, but only with a pair of hosts, corresponding to a flow of packets from src to dst. Presently, all of these events deal with fragments.

flow_weird handles the following events:

[excessively_large_fragment] A set of IP fragments reassembled to a maximum size exceeding 64,000 bytes. Note: Sizes between 64,000 and 65,535 bytes are allowed, strictly speaking, but are highly unlikely in legitimate traffic. Sizes above 65,535 bytes generally represent attempted denial-of-service attacks, due to IP implementations that crash upon receiving such impossibly-large fragment sets.

Default: WEIRD_LOG_ALWAYS.

[excessively_small_fragment] A fragment other than the last fragment in a set was less than 64 bytes in size. Note: The standard allows such small fragments, but their presence may reflect an attacker attempting to evade the monitor by splitting header information across multiple fragments.

Default: WEIRD_LOG_ALWAYS.

[fragment_inconsistency] A fragment overlaps with a previously sent fragment, and the two disagree on data they share in common. This event could reflect an attacker attempting to evade the monitor; it can also occur because Bro keeps previous fragments indefinitely (Deficiency: it needs to provide a means for flushing old fragments, otherwise it becomes vulnerable to a state-holding attack), and occasionally a fragment will overlap with one sent much earlier and long-since forgotten by the endpoints.

Default: WEIRD_LOG_ALWAYS.

[fragment_overlap] A fragment overlaps with a previously sent fragment. As for fragment_inconsistency, this event can occur due to Bro keeping previous fragments indefinitely. This event does not in general reflect a possible attempt at evasion.

Default: WEIRD_LOG_ALWAYS.

[fragment_protocol_inconsistency] Two fragments were seen for the same flow and IP ID which differed in their transport protocol (e.g., UDP, TCP). According to the specification, this is allowed [RFC791, p. 24], but its use appears highly unlikely.

Default: WEIRD_FILE, because it is difficult to see how an attacker can exploit this anomaly.

[fragment_size_inconsistency] A ``last fragment'' was seen twice, and the two disagree on how large the reassembled datagram should be. This event could reflect an attacker attempting to evade the monitor.

Default: WEIRD_FILE, since it is more likely that this occurs due to a high volume flow of fragments wrapping the IP ID space than due to an actual attack.

[fragment_with_DF] A fragment was seen with the ``Don't Fragment'' bit set in its header. While strictly speaking this is not illegal, and not impossible (a router could have fragmented a packet and then decided that the fragments should not be further fragmented), its presence is highly unusual.

Default: WEIRD_FILE, because it's difficult to see how this could reflect malicious activity.

[incompletely_captured_fragment] A fragment was seen whose length field is larger than the fragment datagram appearing on the monitored link.

Default: WEIRD_LOG_ALWAYS.

Events handled by net_weird

[net_weird (name: string)] is invoked for ``weird'' events that cannot be associated with a particular connection or set of hosts. Except as noted, the default action for all such events is WEIRD_FILE.

net_weird handles the following events:

[bad_IP_checksum] A packet had a bad IP header checksum.

[bad_TCP_header_len] The length of the TCP header (which is itself specified in the header) was smaller than the minimum allowed size.

[internally_truncated_header] A captured packet with a valid IP length field was smaller as actually recorded, such that the captured version of the packet was illegally small. This event may reflect an error in Bro's packet capture hardware or software.

Default: WEIRD_LOG_ALWAYS, because this event can indicate a basic problem with Bro's packet capture.

[truncated_IP] A captured packet either was too small to include a minimal IP header, or the full length as recorded by the packet capture library was smaller than the length as indicated by the IP header.

[truncated_header] An IP datagram's header indicates a length smaller than that required for the indicated transport type (TCP, UDP, ICMP).


Events generated by the standard scripts

The following events are generated by the standard scripts themselves:

[bad_pm_port] See pm_bad_port. Handled by conn_weird_addl, where the extra parameter is the text "port <bad-port>".

[Land_attack] A TCP connection attempt was seen with identical initiator and responder addresses and ports. This event likely reflects an attempted denial-of-service attack known as a ``Land'' attack. See check_spoof. Handled by conn_weird.


Additional handlers for ``weird'' events

In addition to the above, generalized events, Bro includes two specific events that are defined by themselves so they can include additional parameterization:

[rexmit_inconsistency (c: connection, t1: string, t2: string)] Invoked when a retransmission associated with connection c differed in its data from the contents transmitted previously. t1 gives the original data and t2 the different retransmitted data.

This event may reflect an attacker attempting to evade the monitor. Unfortunately, however, experience has shown that (i) inconsistent retransmissions do in fact happen due to (appalling) TCP implementation bugs, and (ii) once they occur, they tend to cascade, because often the source of the bug is that the two endpoints have become desynchronized.

The handler logs the message in the format "id rexmit inconsistency (<t1>) (<t2>)" . However, the handler only logs the first instance of an inconsistency, due to the cascade problem mentioned above.

Deficiency: The handler is not told which of the two connection endpoints was the faulty transmitter.

[ack_above_hole (c: connection, t1: string, t2: string)] Invoked when Bro sees a TCP receiver acknowledge data above a sequence hole. In principle, this should never occur. Its presence generally means one of two things: (i) a TCP implementation with an appalling bug (these definitely exist), or (ii) a packet drop by Bro's packet capture facility, such that it never saw the data now being acknowledged.

Because of the seriousness of this latter possibility, the handler logs a message "ack above a hole". Note: You can often distinguish between a truly broken TCP acknowledgment and Bro dropping packets by the fact that in the latter case you generally see a cluster of ack-above-a-hole messages among otherwise unrelated connections.

Deficiency: The handler is not told which of the two connection endpoints sent the acknowledgment.


next up previous contents index
Next: The icmp Analyzer Up: Analyzers and Events Previous: The SSL Analyzer   Contents   Index
Vern Paxson 2004-03-21