Build a Secure Logging Server with syslog-ng (Part 2)

Enterprise Networking Planet content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Build a Secure Logging Server with syslog-ng You’re not a mole, and our guide to building your own secure logging server will help you quit digging through logs like one so you can get to the information you need quickly.

In Part 1we learned the the basics of how syslog-ng works. Today we’ll look at some ways to manage remote client files, and learn how to securely encrypt syslog-ng’s network traffic.

Securing Your Log Server
For your log server to be trustworthy, it should have some good access controls in place. These example iptables rules, run on the logging server itself, illustrate how to allow in only log messages and SSH traffic from the LAN:

iptables -A INPUT -s 192.168.1.0/24 -p tcp --dport 514 --sport 1024:65535 -j ACCEPT
iptables -A INPUT -s 192.168.1.0/24 -p tcp --dport 22 --sport 1024:65535 -m state --state NEW -j ACCEPT

Presumably your default policies are DENY, so you need rules only to allow authorized bits.

You definitely want /varon its own partition, or even a separate drive. If something goes haywire and your log files are flooded with data, you don’t want the entire system to go down.

And of course you know to pay attention to hardening the entire system and application-level security.

Organizing Client Messages
Your starting point is the default syslog-ng.confcreated by the installer. Typically, though it depends on the distribution, this replicates a standard syslogd configuration. This saves you a bit of labor setting up all the standard facility logs, and gives you a nice model to study.

Syslog.ng lets you slice and dice your log files finely. Don’t make yourself crazy and create all manner of separate files for every last little individual criterion. I suggest starting with broader configurations, then directing certain types of messages to their own individual files as you need them. Start with directing unfiltered client messages to the server and let the server sort them out. Then fine-tune the client configurations as you need to. Just keep in mind it’s easier to test and debug a single server configuration file than bales of of client files.

The example server configuration from Part 1can be refined in a number of useful ways. This line creates a new client logfile every day. The client’s directory is named for the remote host, then each logfile uses the date for its name:

destination d_clients { file("/var/log/HOSTS/$HOST/$DATE"); };

Other options are MONTH, WEEKDAY, HOUR, FACILITY, PRIORITY, and several others that you can find in the Syslog-ng Reference Manual. Dumping everything into a single file is rather messy, so you may wish to duplicate a stock configuration by creating files by facility, like this:

destination d_clients { file("/var/log/HOSTS/$HOST/$DATE_$FACILITY"); };

This example gives you another way to sort by date, which is helpful if you need to archive log files for a long time:

destination d_clients { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DATE_$FACILITY"); };

Be sure to have the create_dirs (yes);option set as shown in Part 1.

This introduces a new issue: the default logrotate configuration probably doesn’t have a clue about your new log configurations. If you want your logs to be rotated instead of archived, you’ll have to tweak logrotate.

Logging Both Remotely and Locally
There’s no downside to logging remote clients both locally and remotely. Adding these lines to a stock syslog-ng.conf adds remote logging to the existing standard local logging configuration:

source s_local { internal(); unix-stream(“/dev/log”); file(“/proc/kmsg” log_prefix(“kernel: “)); };
destination d_loghost {tcp(“192.168.1.10” port(514));};
log { source(s_local); destination(d_loghost); };

Of course you must use your own loghost IP and port number. 514 is the standard syslog-ng port, but you may use any unused port that you desire. Check /etc/services to find unused ports, then be sure to edit /etc/servicesto include the port you are using.

Encryption With Stunnel
For additional protection you can use stunnel with syslog-ng. Syslog-ng has no native encryption, so stunnel fills this role quite ably. Stunnel must be installed on the server and on all clients. You’ll also need OpenSSL, since Stunnel requires it. You’ll need to create and distribute SSL certificates, which is the usual many-step dance: create your certificate authority, create .pemfiles for server and clients, sign the certificates, edit the derned things to remove the private keys, then distribute them.

If you do not already have your own SSL certificate authority, you must now create one. (The following instructions work on most Linux distributions, except Fedora/Red Hat etc., which as usual do things their own weird way. I’m not sure what to do with these, so I’m afraid you’re on your own.) Find openssl.conf, and find and edit these lines:

[ CA_default ]
dir   =   ./masterCA    # Where everything is kept
countryName_default             = US
stateOrProvinceName_default     = Oregon
0.organizationName_default      = Alrac Logging Co.
stateOrProvinceName		= Oregon
stateOrProvinceName_default	= OR
localityName			= Portland
organizationalUnitName_default	= CutEmAllDown
commonName			= Alrac AceAdmin
emailAddress			= carla@boardfeet.com

Now find the CA.sh script, and edit this line to tell CA.shwhere to put your new certificates, giving it any name you like:

CATOP=./masterCA

Now change to the directory you want to store your CA in, typically /etc/ssl/, and run the CA.shscript:

# CA.sh -newca

This creates the /etc/ssl/masterCA/ directory, and populates it with all manner of files and directories, including your new server CA, private/cakey.pem. You’ll be asked to create a passphrase, which you must write down and keep in a safe place.

Now create the certificate requests for the server, and individual ones for each client:

# openssl req -new -nodes -out logserver_req.pem -keyout logserver.pem
# openssl req -new -nodes -out client1_req.pem -keyout client1.pem
# openssl req -new -nodes -out client2_req.pem -keyout client2.pem

Now each one must be signed by your certificate authority:

# openssl ca -out logserver_cert.pem -infiles logserver_req.pem
# openssl ca -out client1_cert.pem -infiles client1_req.pem
# openssl ca -out client2_cert.pem -infiles client2_req.pem

There are a couple more steps before your shiny new certificates are usable. First, place logserver_cert.pem in /etc/stunnel on the server, and put every client key into each client’s /etc/stunneldirectory.

Each certificate contains a private key and the certificate itself. The server and client certificate portions need to be exchanged, so that the server has a copy of every client certificate, and every client has a copy of the server certificate. So first copy the certificate portion from every client certificate, and place them all into a single file for the server and store it in /etc/stunnel. The certificate is the bit between

—–BEGIN CERTIFICATE—–
—–END CERTIFICATE—–

You can call this file anything you want; I use client_cert.pem. Then you need to delete the private key from a copy of logserver_cert.pem, and place copies on every client in /etc/stunnel.

Configure Stunnel
Finally we get to configure Stunnel. You’ll need to edit /etc/stunnel/stunnel.conf. If your installer didn’t create one, look for an example file to use. Add these options, using your own server IP, port numbers, and certificate names:

#stunnel.cnf for the logserver
cert = /etc/stunnel/logserver_cert.pem
CAfile = /etc/stunnel/client_cert.pem
verify = 3
[5140]
    accept = 192.168.1.10:5140
    connect = 127.0.0.1:514

#stunnel.cnf for all clients
client = yes
cert = /etc/stunnel/client_cert.pem
CAfile = /etc/stunnel/logserver_cert.pem
verify = 3
[5140]
    accept = 127.0.0.1:514
    connect = 192.168.1.10:5140

Syslog-ng.conf
Finally, your syslog-ng.conffile will need a few additions on both clients and server. (These lines reference the examples in both part 1 and part 2.) On the clients, add these lines:

destination d_stunnel {tcp("127.0.0.1" port(514));};
log {source(s_local);destination(d_stunnel);};

The server needs these lines:

source s_stunnel {tcp(ip("127.0.0.1");port(514));};
log {source(s_stunnel); destination(d_clients);};

Start up stunnel by running the stunnel command, and fire up syslog-ng if it isn’t running already. Run some tests with the logger command (see Part 1), and there you are, running your own nicely secured remote logging server.

Resources

Get the Free Newsletter!

Subscribe to Daily Tech Insider for top news, trends, and analysis.

Latest Articles

Follow Us On Social Media

Explore More