Enhance Security with a Linux Logging Server
Building a Linux logging server isn't hard, and it's a good way to protect your system logs from crackers, or figure out how they compromised your system in the first place.
Increasing security often means decreasing convenience. But when it comes to using a remote logging server, you get both security and convenience. All of your important logs are in one convenient-to-you location. If a host is compromised by a cracker, the logfiles are out of the varmint's reach. In the event of a system failure the logs are preserved, so you can conduct a proper post-mortem.
Syslogd and Syslog-ng
The sysklogd package has long been the standard logging utility for Linux. It contains syslogd (system logging daemon), klogd (kernel logging daemon) and the usual man pages and configuration files. syslogd captures and directs the logging output from local and remote processes, and from the kernel via klogd. Virtually all Linux distributions ship with a decent default setup that does the job for most users: startup scripts, rotations, and logfiles.
Syslog-ng (new generation) is the newfangled Unix system logger for admins who want finer-grained filtering and analysis, security, or who have complex needs that sysklogd cannot address. It whittles down the amount of "noise" and unimportant messages so you are not inundated with useless data, and do not have to resort to complex log-filtering and parsing to get to the bits you really want. Syslog-ng can traverse complex network links because it supports both TCP and UDP. (syslogd only supports UDP.)
sysklogd comes on virtually all Linux systems, and is simple to use, so all of our examples will use it. (Watch this space for a future Syslog-ng howto.)
Configuring the Syslog
Take a peek in your /etc/syslog.conf file. This is where you tell the syslog daemon where to send messages for all the different processes. For example:
# First some standard logfiles. Log by facility.
Every syslog.conf rule has two fields, separated by tabs: a selector field and an action field. The selector field has two parts, separated by a dot: facility and priority. On the first line, auth,authpriv are facilities. The asterisk is a wildcard that means "log messages of any priority."
"Facility" can be thought of as categories. These are listed in man 5 syslog.conf:
auth, authpriv, cron, daemon, ftp, kern, lpr, mail, mark, news, security (same as auth), syslog, user, uucp and local0 through local7.
Don't use security; it is obsolete. mark is for the exclusive use of the syslog daemon, so don't use it either. (See "What the Heck is —Mark—? Learn Linux Logging")
"Priority" sets the level at which a log message is triggered:
debug, info, notice, warning, warn (same as warning), err, error (same as err), crit, alert, emerg, panic (same as emerg).
error, warn and panic are obsolete and should not be used. The priorities are listed in order from lowest (debug) to highest (emerg). Your choice includes all the ones above it - for example, a priority of err means err + crit + alert + emerge. If you wish to select a single specific priority use =:
Use ! to exclude a priority. This means "all priorities except warning":
You may set only one priority per selector. Other choices are none, or *, which means "any."
/var/log/auth.log is the "action." Note that the action on the second line in the syslog.conf example is preceded by a hyphen. This means "do not synchronize the file after every write." Do this for busy processes, like mail and lpr. If you don't your system performance will suffer. There is a risk of losing a bit of data if the system crashes, but that's hardly worth jamming up a busy server.
Enabling remote logging is very simple. The most difficult part is keeping all the doggone names straight — syslog, syslogd, sysklogd — I tell ya, it's a deliberate form of geek torture. Then you need a dedicated logging server somewheres. Then you need to start the syslog daemon with the -r switch; this tells it to accept messages from remote processes. Then the various hosts need to be configured to send their log messages to your logging server.
If you're running a Debian logging server, edit /etc/init.d/sysklogd:
Then restart it:
# /etc/init.d/sysklogd restart
On Red Hat/Fedora/CentOS/White Box/etc. edit /etc/sysconfig/syslog:
# service syslog restart
Finally, on the hosts that are going to send log messages to your logging server, edit their individual /etc/syslog.conf files. Prepending the action with @ tells it to send to a remote host via UDP 514. Use hostnames, fully qualified domain names, or IPs. This example sends pretty much everything to the log server:
You can log both locally and remotely:
sysklogd has no built-in access controls at all — it happily accepts anything coming over UDP port 514. A simple method for allowing only packets from your own hosts is to use TCP wrappers, which is configured with the /etc/hosts.allow and /etc/hosts.deny files. A simple, paranoid configuration looks like this. In /etc/hosts.deny:
Then open the door to your allowed logging hosts in /etc/hosts.allow:
Or use IPs, like
Stealth Logging Server
Michael Bauer, author of the excellent "Linux Server Security", details an interesting scheme for setting up a stealth logging server. Since UDP is a one-way protocol your logging server doesn't need to have its own IP address, thereby making it more difficult for an intruder to find and possibly mess with it. Specify a fake address or hostname in your host configurations, then add a corresponding fake arp entry in the network startup script. To capture the log messages, grab all the UDP 514 packets with a packet sniffer.
Obviously this whole scheme has its limitations, and extracting the data from the raw packets isn't as fun as it sounds, but it is an interesting concept. If security is a major concern you're better off with Syslog-ng.