Build a Secure Logging Server with syslog-ng

You're not a mole, and our guide to building your own secure logging server will help you quit digging through logs like one so you can get to the information you need quickly.

By Carla Schroder | Posted Apr 4, 2006
Page 1 of 2
Print ArticleEmail Article
  • Share on Facebook
  • Share on Twitter
  • Share on LinkedIn

Managing Linux system and application logging is important and a bit tricksy. You want to capture important information, not bales of noise. You need to be able to find what you want in your logs without making it your life's work. The venerable old syslogd has served nobly for many years, but it's not quite up to meeting more complex needs. For this we have the next-generation Linux logger, syslog-ng.

Syslog-ng has a number of advantages over our old friend syslogd: better networking support, highly-configurable filters, centralized network logging, and lots more flexibility. For example, with syslogd all iptables messages get dumped in kern.log along with all the other kernel messages. Syslog-ng lets you direct iptables messages to a separate logfile. Syslogd uses only UDP; syslog-ng runs over UDP and TCP, so you can run it over encrypted network tunnels to a central logging server.

Installation

Syslog-ng is a standard package on all Linuxes. The current stable release is 1.6. If you are using apt-get or yum sysklogd should be removed automatically. But if it isn't, make sure to remove it and to stop the syslogd daemon. (sysklogd is the package name, syslogd is the daemon.)

Configuration File Structure

With all of this flexibility comes a bit of a learning curve. Syslog-ng's configuration file is /etc/syslog-ng/syslog-ng.conf, or on some systems /etc/syslog-ng.conf. You'll need man 5 syslog-ng.conf to understand all the different options and parameters. syslog-ng.conf has five possible sections:

options{}
Global options. These can be overridden in any of the next four sections

source{}
Message sources, such as files, local sockets, or remote hosts

destination{}
Message destinations, such as files, local sockets, or remote hosts

filter{}
Filters are powerful and flexible; you can filter on any aspect of a log message, such as standard syslogd facility names (man 5 syslog.conf), log level, hostname, and arbitrary contents like words or number strings

log{}
Log statements connect the source, destination, and filter statements, and tell syslog-ng what to do with them

Here are some typical global options:

options {

    sync (0);
    log_fifo_size (2048);
    create_dirs (yes);
    group (logs);
    dir_group (logs);
    perm (0640);
    dir_perm (0750);
    };

Options statements must use options as defined in /etc/syslog-ng.conf. This what the examples mean:

sync
How many lines of messages to keep in the output queue before writing to disk. (Logging output is called "messages.") Zero is the preferred option, since you want to be sure to capture everything, and not risk losing data in the event of a power failure

log_fifo_size
The maximum number of lines of messages in the output queue. The default is 100 lines of messages. You can do some calculations to figure out a suitable value, as this syslog-ng mail list post shows. To quote the relevant bit:

Each message source receives maximum 1024 bytes of data. A single log message is about 20 to 30 bytes. So on a single run each log source may emit 1024/20 = 50 messages. Your log fifo must be able to hold this number of messages for each source. So for two sources, you'll need at least 100 slots in your fifo.

It is easy to get confused by these two options. Incoming messages from remote hosts arrive in bursts, so what you need to do is make sure your log_fifo_size is large enough to handle these bursts. Otherwise you risk losing messages. Ultimately you will be limited by i/o and network speeds

create_dirs
Enable or disable automatic directory creation for destination files. In this article this value is "yes", so that remote host directories can be created on demand

group
dir_group
Set the group owner of logfiles and directories, so you don't have to read and analyze logs as the superuser

perm
dir_perm
Default permissions on new files and directories

Source, Destination, and Filter Statements

Source, destination, and filter names are arbitrary, like these examples show:

source s_local { internal(); unix-stream("/dev/log"); file("/proc/kmsg" log_prefix("kernel: ")); };
destination d_auth { file("/var/log/auth.log"); };
filter f_auth { facility(auth, authpriv); };

So "source s_local" could be "source frederick_remington_depew" if you so desired, and "destination d_auth" could be "destination moon." The convention is to prefix source names with "s", destination names with "d", and filter names with "f", but this is not required. All other elements of source, destination, and filter statements must use parameters as defined in man 5 syslog-ng.conf.

The "source s_local" example collects all locally-generated log messages into a single source statement. The "destination d_auth" statement directs authorization messages to /var/log/auth.log, which are defined by the "filter f_auth" statement. auth and authpriv are standard syslog facility names.

Log statements pull it all together:

log {source(s_local); filter(f_auth); destination(d_auth); };

So these four lines filter out all the auth and authpriv messages from all local messages, and write them to /var/log/auth.log.

Comment and Contribute
(Maximum characters: 1200). You have
characters left.
Get the Latest Scoop with Enterprise Networking Planet Newsletter