.\" .\" SEC version 2.3.3 - sec.pl.man .\" simple event correlation tool .\" .\" Copyright (C) 2000-2006 Risto Vaarandi .\" .\" This program is free software; you can redistribute it and/or .\" modify it under the terms of the GNU General Public License .\" as published by the Free Software Foundation; either version 2 .\" of the License, or (at your option) any later version. .\" .\" This program is distributed in the hope that it will be useful, .\" but WITHOUT ANY WARRANTY; without even the implied warranty of .\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the .\" GNU General Public License for more details. .\" .\" You should have received a copy of the GNU General Public License .\" along with this program; if not, write to the Free Software .\" Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. .\" .TH sec.pl 1 "March 2006" "SEC 2.3.3" .SH NAME sec.pl \- simple event correlator .SH SYNOPSIS .TP .B sec.pl -conf= ... .br [-input=[=] ...] .br [-input_timeout=] .br [-timeout_script=] .br [-reopen_timeout=] .br [-poll_timeout=] .br [-check_timeout=] .br [-blocksize=] .br [-log=] .br [-syslog=] .br [-debug=] .br [-pid=] .br [-dump=] .br [-cleantime=] .br [-bufsize=] .br [-evstoresize=] .br [-quoting | -noquoting] .br [-tail | -notail] .br [-fromstart | -nofromstart] .br [-detach | -nodetach] .br [-intevents | -nointevents] .br [-intcontexts | -nointcontexts] .br [-testonly | -notestonly] .br [-separator=] .SH DESCRIPTION SEC is a tool that was designed for solving various event correlation tasks in the domains of network and system management. Event correlation is a process where a stream of primitive events is processed in order to detect .I composite events that correspond to event patterns in the event stream. .PP After startup SEC reads lines from files, named pipes, or standard input, matches the lines with patterns (regular expressions, Perl subroutines, etc.) for recognizing input events, and correlates events according to the rules in its configuration file(s). In order to detect a composite event, rules are used sequentially in the same order as they are given in the configuration file. If there are two or more configuration files, rule sequences from different files are used virtually in parallel. SEC can be configured to produce its output by executing user-specified shell commands, and by using utilities like .BR snmptrap (1) or .BR snmpnotify (1), SEC can generate network management events as output. Other options for producing output events are described further in this man page. .PP Following rule types are currently implemented in SEC for detecting composite events: .PP .B Single - match input event and immediately execute an action. .PP .B SingleWithScript - match input event and depending on the exit value of an external script, execute an action. .PP .B SingleWithSuppress - match input event and execute an action immediately, but ignore following matching events for the next .I t seconds. .PP .B Pair - match input event, execute an action immediately, and ignore following matching events until some other input event arrives. On the arrival of the second event execute another action. .PP .B PairWithWindow - match input event and wait for .I t seconds for other input event to arrive. If that event is not observed within a given time window, execute an action. If the event arrives on time, execute another action. .PP .B SingleWithThreshold - count matching input events during .I t seconds and if a given threshold is exceeded, execute an action and ignore all matching events during the rest of the time window. .PP .B SingleWith2Thresholds - count matching input events during .I t1 seconds and if a given threshold is exceeded, execute an action. Then start the counting of matching events again and if their number per .I t2 seconds drops below the second threshold, execute another action. .PP .B Suppress - suppress matching input events (used to keep the events from being matched by later rules). .PP .B Calendar - execute an action at specific times. .PP If a rule defines the correlation of several events in a certain time window, the rule will start an event correlation operation when the first matching event occurs. In order to distinguish between different event correlation operations that have been started by the rules, every operation is assigned a unique key that is generated from the rule ID, the name of the configuration file the rule belongs to, and the textual description of the composite event that the correlation operation is trying to detect (see RULES AND EVENT CORRELATION OPERATIONS section for more information). .PP Rules allow not only shell commands to be executed as actions, but among other options it is also possible to create and delete contexts that decide whether a particular rule can be applied at the moment, generate synthetic events that will act as input for other rules, and reset correlation operations that have been started by other rules. This makes it possible to combine several rules and form more complex event correlation schemes. Contexts can not only be used for activating or deactivating a particular rule, but they can also serve as event stores (in .BR logsurfer (1) the contexts are used in a similar way). There are actions for adding event to a context, reporting all events associated with a context, etc. By using the features of SEC in an appropriate way, one can solve a wide range of event correlation and event consolidation tasks. .SH OPTIONS .TP .B "-conf=" expand to filenames (with Perl .BR glob () function) and read event correlation rules from every file. Multiple .B -conf options can be specified on commandline. Each time SEC receives a signal that forces a configuration reload, is re-evaluated. See also TIMING AND IPC section for a discussion about the rule processing order when multiple configuration files are involved. .TP .B "-input=[=]" expand to filenames (with Perl .BR glob () function) and use the files as event sources. An input file can be a regular file, named pipe, or standard input if .B - was specified. Multiple .B -input options can be specified on commandline. Each time SEC receives a signal that forces it to reopen its input files, is re-evaluated. If SEC experiences a system error when reading from an input file, it will close the file. If is given, SEC will set up the context each time it reads a line from input files that correspond to . This will help the user to write rules that match data from particular input source(s) only. When there is an .B -input option with specified, it will automatically enable the .B -intcontexts option. See TIMING AND IPC section for more information. .TP .B "-input_timeout=" .TP .B "-timeout_script=" if SEC has not observed new data in an input file during seconds (or the file was closed seconds ago), will be executed with commandline parameters 1 and . If fresh data become available again, will be executed with commandline parameters 0 and . .TP .B "-reopen_timeout=" if SEC has not observed new data in an input file during seconds (or the file was closed seconds ago), SEC will attempt to reopen this input file. If the attempt fails, SEC will try again after every seconds until open succeeds. .TP .B "-poll_timeout=" a real number that specifies how many seconds SEC will sleep when no new data were read from input files. Default is 0.1 seconds. .TP .B "-check_timeout=" if SEC has not observed new data in an input file, SEC will not poll the file (both for status and data) during the next seconds. .TP .B "-blocksize=" size of the block in bytes that SEC will attempt to read from input files at once. Default is 1024 (i.e., read from input files by 1KB blocks). .TP .B "-log=" use for logging SEC activities. Note that if the SEC standard error is connected to a terminal, messages will be logged there, in order to facilitate debugging. If the .B -detach option has also been specified, no logging to standard error will take place. .TP .B "-syslog=" use syslog for logging SEC activities. All messages will be logged with the facility , e.g., .I local0 (see .BR syslog (3) for possible facility values). Warning: be careful with this option if you use SEC for monitoring syslog logfiles, because it might cause message loops when SEC log messages are written to SEC input files. .TP .B "-debug=" set logging verbosity for the SEC logfile. Setting debuglevel to means that all messages from and lower levels are logged (e.g., if is 3, messages from levels 1, 2, and 3 are logged). The following levels are recognized by SEC: .br 1 - critical messages (severe faults that cause SEC to terminate, e.g., a failed system call) .br 2 - error messages (faults that need attention but don't cause SEC to terminate, e.g., an incorrect rule definition in the configuration file) .br 3 - warning messages (possible faults, e.g., a command forked from SEC terminated with a non-zero exit code) .br 4 - notification messages (normal system level events and interrupts, e.g., the reception of a signal) .br 5 - informative messages (information about external commands forked from SEC) .br 6 - debug messages (detailed information about all SEC activities) .br Default is 6 (i.e., log everything). .TP .B "-pid=" use for storing the process ID of SEC. .TP .B "-dump=" use as the SEC dumpfile. Default is /tmp/sec.dump (see SIGNALS section for more information). .TP .B "-cleantime=" time interval in seconds that specifies how often internal event correlation and context lists are processed to perform time-related tasks and to remove obsolete elements. Default is 1 second. .TP .B "-bufsize=" set input buffer to hold last lines that have been read from input files. The content of input buffer will be compared with patterns that are part of rule definitions (i.e., no more than lines can be matched by a pattern at a time). Default size is 10 (i.e., 10 lines). .TP .B "-evstoresize=" set an upper limit to the size of context event stores - if an event store already contains events, SEC will add no more events to that context, regardless of its configuration file directives. Specifying 0 as or omitting the .B -evstoresize option means that there is no limit to the size of event stores. .TP .B "-quoting" .TP .B "-noquoting" if the .B -quoting option is specified, event description strings that are supplied to external shell commands of .I shellcmd and .I spawn actions will be put inside apostrophes. All apostrophes ' that strings originally contain will be masked. This option prevents the shell from interpreting special symbols that event description strings might contain. Default is .BR -noquoting . .TP .B "-tail" .TP .B "-notail" if the .B -notail option is specified, SEC will process all data that are currently available in input files and exit after reaching all EOFs. Default is .B -tail - SEC will jump to the end of input files and wait for new lines to arrive. With the .B -tail option, SEC follows an input file by its name (and not merely by its i-node). If the input file is recreated or truncated, SEC will reopen it and process its content from the start. If the input file is removed (i.e., there is just an i-node left without name), SEC will keep the i-node open and wait for the input file to be recreated. If the .B "-reopen_timeout=" option was specified and no data have been added to the i-node during seconds, SEC will attempt to reopen the file without further waiting for the file to be recreated. .TP .B "-fromstart" .TP .B "-nofromstart" These flags have no meaning when the .B -notail option is also specified. When used in combination with .B -tail (or alone, since .B -tail is enabled by default), .B -fromstart will force SEC to read and process input files from the beginning to the end, before the 'tail' mode is entered. Default is .BR -nofromstart . .TP .B "-detach" .TP .B "-nodetach" if the .B -detach option is specified, SEC will disassociate itself from the controlling terminal and become a daemon (note that SEC will close its standard input, standard output, and standard error, and change its working directory to the root directory). Default is .BR -nodetach . .TP .B "-intevents" .TP .B "-nointevents" SEC will generate internal events when it starts up, when it receives certain signals, and when it terminates normally. Specific rules can be written to match those internal events, in order to take some action at SEC startup, restart, and shutdown. See TIMING AND IPC section for more information. Default is .BR -nointevents . .TP .B "-intcontexts" .TP .B "-nointcontexts" SEC will create an internal context when it reads a line from an input file. This will help the user to write rules that match data from particular input source only. See TIMING AND IPC section for more information. Default is .BR -nointcontexts . .TP .B "-testonly" .TP .B "-notestonly" if the .B -testonly option is specified, SEC will exit immediately after parsing the configuration file(s). If the configuration file(s) contained no faulty rules, SEC will exit with 0, otherwise with 1. Default is .BR -notestonly . .TP .B "-separator=" obsolete. .PP Note that one can introduce options both with a single dash (-) and double dash (--), and also use both an equal sign (=) and whitespace as a separator between the option name and the option value, e.g., .B "-conf=" and .B "--conf " are equivalent. .SH CONFIGURATION FILE The SEC configuration file consists of rule definitions. Each rule definition is made up of key=value pairs, one key and value per line. Values are case sensitive only where character case is important (like the values specifying regular expressions). \\-symbol may be used at the end of a line to continue the key=value pair on the next line. Lines which begin with #-symbol are treated as comments and ignored (whitespace characters may precede #-symbol). Any comment line, empty line, or end of the file will terminate the preceding rule definition. .PP Before describing each rule type in detail, patterns and pattern types, context expressions, and action lists are discussed, since they are important parts of the rule definition. .SS "PATTERNS AND PATTERN TYPES" SEC supports the following pattern types (if is omitted, 1 is assumed): .TP .I "SubStr[]" pattern is assumed to be a substring that will be searched in last input lines L1, L2, ..., L. The input lines are joined into a single string with the newline character acting as a separator, and the resulting string "L1\\nL2\\n...\\nL" is searched for the substring. (Note that if is 1, the last input line without a terminating newline is searched.) If the substring is found, the pattern matches. The backslash constructs \\t, \\n, \\r, \\s, and \\0 can be used in the pattern to denote tabulation, newline, carriage return, space character, and empty string, while \\\\ denotes backslash itself. As an example, consider the following pattern definition: .sp ptype=substr .br pattern=Backup done:\\tsuccess .sp The pattern matches lines containing "Backup done:success". .TP .I "RegExp[]" pattern is assumed to be a regular expression that last input lines L1, L2, ..., L are compared with. The input lines are joined into a single string with the newline character acting as a separator, and the resulting string "L1\\nL2\\n...\\nL" is compared with the regular expression pattern. (Note that if is 1, the last input line without a terminating newline is compared with the pattern.) If the pattern matches, backreference values will be assigned to the special variables $1, $2, ..., and the special variable $0 will be set to "L1\\nL2\\n...\\nL" (i.e., to the matching input line(s)). These special variables can be used in some other parts of the rule definition. All regular expression constructs that Perl allows are allowed in the pattern (see .BR perlre (1) for more information). As an example, consider the following pattern definition: .sp ptype=regexp .br pattern=(?i)(\\S+\\.mydomain).*printer: toner\\/ink low .sp The pattern matches "printer: toner/ink low" messages in a case insensitive manner from printers belonging to .mydomain. Note that the printer hostname is assigned to $1, while the whole message line is assigned to $0. As another example, the following pattern definition .sp ptype=regexp2 .br pattern=^AAA\\nBBB$ .sp produces a match if the last two input lines are AAA and BBB, and sets $0 to "AAA\\nBBB". .TP .I "PerlFunc[]" pattern is assumed to be a Perl function that last input lines L1, L2, ..., L are submitted to. The Perl function is compiled at SEC startup by calling Perl .BR eval () function, and .BR eval () must return a code reference for the pattern to be valid (also see VARIABLES AND EVAL section for more information). In order to check whether the pattern matches, SEC will call the function in list context and pass lines L1, L2, ..., L and the names of corresponding input sources S1, S2, ..., S to the function as parameters: .sp function(L1, L2, ..., L, S1, S2, ..., S) .sp (if the input line has been generated with the .I event action, its input source name will be set to 'undef'). If the function returns several values or a single value that is TRUE in boolean context, the pattern matches. If the pattern matches, return values will be assigned to the special variables $1, $2, ..., and the special variable $0 will be set to "L1\\nL2\\n...\\nL" (i.e., to the matching input line(s)). These special variables can be used in some other parts of the rule definition. As an example, consider the following pattern definition: .sp ptype=perlfunc2 .br pattern=sub { return ($_[0] cmp $_[1]); } .sp The pattern compares last two input lines in a stringwise manner ($_[1] holds the last line and $_[0] the preceding one), and matches if the lines are different. Note that the result of the comparison is assigned to $1, while the two input lines are concatenated (with the newline character between them) and assigned to $0. As another example, the following pattern definition .sp ptype=perlfunc .br pattern=sub { if ($_[0] =~ /(abc|def)/) { \\ .br return defined($_[1]) ? $_[1] : "SEC"; } return 0; } .sp produces a match if the input line contains either the string "abc" or the string "def", and sets $0 to the matching line and $1 to the name of the input source. .TP .I "NSubStr[]" like .IR "SubStr[]" , except the result of the match is negated. .TP .I "NRegExp[]" like .IR "RegExp[]" , except the result of the match is negated. .TP .I "NPerlFunc[]" like .IR "PerlFunc[]" , except the result of the match is negated. .TP .I "TValue" pattern is assumed to be a truth value with TRUE and FALSE being legitimate values for the pattern. TRUE always matches an input line while FALSE never matches an input line. .PP Note that since Pair and PairWithWindow rules have two pattern definitions, special variables $ set by the first pattern are shadowed when the second pattern matches and sets the variables. In order to access the shadowed variables, they must be referred to as % (e.g., instead of $1 one has to write %1). Also note that the second pattern of Pair and PairWithWindow rules may contain $ variables, if the second pattern is of type SubStr, NSubStr, Regexp, or NRegExp. The variables are substituted at runtime with the values set by the first pattern. .SS "CONTEXT EXPRESSIONS" Context expression is a logical expression that consists of context names, Perl miniprograms, and Perl functions as operands; operands are combined with operators ! (logical NOT), && (logical AND), || (logical OR), and parentheses. Context expressions are employed for activating or deactivating rules - the truth value of the rule's context expression must be TRUE for the rule to be applicable. .PP If the operand contains the arrow (->), the text following the arrow is considered to be a Perl function that will be compiled at SEC startup by calling Perl .BR eval () function, and .BR eval () must return a code reference for the operand to be valid (also see VARIABLES AND EVAL section for more information). If any text precedes the arrow, it is considered to be the list of parameters for the function. Parameters are separated by whitespace and may contain $ and % special variables. In order to evaluate the Perl function operand, the function with its parameter list will be called in scalar context. If the return value of the function is TRUE in boolean context, the truth value of the operand is TRUE, otherwise its truth value is FALSE. .PP If the operand begins with the equal sign (=), the following text is considered to be a Perl miniprogram. The miniprogram may contain $ and % special variables. In order to evaluate the Perl miniprogram operand, it will be executed by calling Perl .BR eval () function in scalar context (also see VARIABLES AND EVAL section for more information). If the return value of the miniprogram is TRUE in boolean context, the truth value of the operand is TRUE, otherwise its truth value is FALSE. .PP Note that since Perl functions, miniprograms, and parameter lists may contain strings that clash with the operators of the context expression (e.g., '!'), it is strongly recommended to enclose them in parentheses, e.g., .PP ($1 $2) -> (sub { return ($_[0] != $_[1]); }) .PP =({my($temp) = 0; return !$temp;}) .PP If the operand is not a Perl function or program, it is considered to be a context name. Context name may contain $ and % special variables. If the context name refers to an existing context, the truth value of the operand is TRUE, otherwise its truth value is FALSE. .PP If the whole context expression is enclosed in square brackets [ ], e.g., .BR "[MYCONTEXT1 && !MYCONTEXT2]" , SEC evaluates the expression _before_ pattern matching operation (normally the pattern is compared with input line(s) first, so that $ and % variables in the context expression could be replaced with their values). Evaluating context expression first could save CPU time when many of the input lines are known to match the pattern but not the context expression, and the expression does not contain any variables. .PP As an example, consider the following context expressions: .PP -> ( sub { my(@stat) = stat("/var/log/messages"); \\ .br return (!scalar(@stat) || time() - $stat[9] > 3600); } ) .PP ($1 $2) -> (sub { return ($_[0] != $_[1]); }) && C1 .PP !(C1 || C2) && =("$1" eq "myhost.mydomain") .PP The first expression is TRUE when the /var/log/messages file does not exist or was last modified more than 1 hour ago. The second expression is TRUE when variables $1 and $2 are numerically not equal and the context .B C1 exists. The third expression is TRUE when contexts .B C1 and .B C2 do not exist and the $1 variable equals to the string "myhost.mydomain". Note that the Perl code of the third expression is compiled each time the expression is evaluated. .SS "ACTION LISTS" Action list consists of action definitions that are separated by semicolons. Each action definition begins with a keyword specifying the action type, followed by additional parameters. Additional parameters that do not have a constant nature may contain $ and % special variables. Also, the following variables can be used in non-constant parameters: .PP %s - event description string (set either by the .I desc or .I desc2 parameter of the rule definition). .br %t - textual timestamp (as returned by .BR date (1)). .br %u - numeric timestamp (as returned by .BR time (2)). .br % - user-defined variables that can be set at SEC runtime with certain actions like .I assign and .I eval ( must begin with a letter and may contain letters, digits, and underscores). In order to disambiguate from the following text, must be enclosed in braces: %{}. .PP In order to use semicolons inside a non-constant parameter, the parameter must be enclosed in parentheses (the outermost set of parentheses will be removed by SEC during configuration file parsing). .PP The following actions are supported: .TP .I "none" No action. .TP .I "logonly" Event description (the value of the %s variable) is logged. .TP .I "write []" Event string and terminating newline are written to the file ( may not contain spaces). File may be a regular file, named pipe, or standard output if .B - was specified. If the file is a regular file, is appended to the end of the file. If the file does not exist, it is created as a regular file before writing. If is omitted, %s is assumed for its value. Though .I write can also be expressed through the .I shellcmd action, .I write does not involve .BR fork (2) like .I shellcmd does. .TP .I "shellcmd " Shell command is executed. If the .B -quoting option was specified, %s will be converted to '%s' before supplying it to shell command (see the .B -quoting and .B -noquoting options). .TP .I "spawn " Identical to the .I shellcmd action, except the following - every line from the standard output of is treated like SEC input line and matched against the rules. This is done by applying .I "event 0 " to every line from standard output (see the .I event action). Note that if outputs a large data set at once, SEC will process it all at once, so if enters an endless "busy write" loop, it will block SEC from doing anything other than processing its output. .TP .I "pipe '' []" Event string and terminating newline are fed to the standard input of shell command (apostrophes are used to mark the beginning and end of , in order to distinguish it from ). If is omitted and there is nothing between apostrophes, %s is assumed for . If is omitted, is written to standard output. .TP .I "create [ [