Is Linux frustrating for you? Do you find yourself spending time trying to figure out why package updates broke or undid changes you’ve made? You’re not alone, but we have good news: it doesn’t have to be that way. Let us show you how to start getting along with your Linux distro.
What we’re talking about is how easy it is to paint yourself into a corner with Linux. You can modify anything, and most often people turn to Google for solutions to problems that arise. Those solutions may work, but they also lead down a path of doom.
Do Things the Distro Way
Yes, you can change random configuration files in /etc/ and make something work, but is that what the Linux distro intended? It is possible that the file is owned by a package, so the next time the package is updated, your change will be overwritten. Package management systems don’t generally care about collisions with modified files, with the exception of FreeBSD which will prompt you at upgrade time with a diff of each file. Most Linux package managers happily tromp over your changes to ensure the supported defaults are installed, which makes things much more supportable from the distro’s standpoint.
Once you gain an understanding of how your Linux distro wants you to modify configuration files (and how the various packages you use break those rules), things should become easier. In the meantime, you can quickly find out if a configuration file you’re modifying is owned by a package manager with the following commands:
In Red Hat / CentOS: rpm -fq /path/to/file
In Debian / Ubuntu: apt-file search /path/to/file
The RPM package can define how to handle configuration file overwrites, but instead of reading the RPM SPEC file, most of the time it’s best to assume you wish to avoid writing to a package’s managed files.
It may be useful to know how to list all files in a package, too:
In Red Hat / CentOS: rpm -ql packagename
In Debian / Ubuntu: apt-file list packagename
In the “getting to know you” stage, when you start running a new package, it is useful to poke around and see where everything is laid out. If we wanted to understand what a package did, instead of reading the .spec file, we could run: rpm -qa |grep name to determine the actual package name of the software, followed by rpm -ql packagename to get the full list of files the package installed (in RPM-based systems).
Each Linux distribution will have its own ideas about the use of single configuration files versus a directory of configuration snippets. This also varies by the particular software package you’re installing. Check for the following directories, always, before editing a “.conf” file directly in /etc, as a general rule:
- /etc/[softwarename]/config.d
- /etc/defaults/[softwarename]
- /etc/[softwarename].d/
Read Up
Linux is not “Linux.” The Red Hat way of doing things is far different from the SUSE or Debian way. Just because you’ve run Debian servers for years doesn’t mean you should start supporting a new distro without reading as much as possible.
Various software packages, Apache for example, have their own ways of doing things. The layout of /etc/apache (or /etc/httpd, depending on the distro!) is very different between each of the distros. If you compile Apache yourself, it again is different. Apache is an ubiquitous service, and one that often breaks due to people who “know Apache” editing the wrong configuration files within the confines of a distro-specific configuration structure.
The challenge of supporting and being knowledgeable about multiple Linux distros goes beyond learning how to work with different package managers or knowing where to look for network interface configuration files. It’s about understanding what rules are imposed on package maintainers who wish to add software to the system. Therefore, looking at package requirements is a great place to start. You don’t often find a document directed at sysadmins that explains how configurations are laid out, because it changes often. The definitive source of that information is often in developer (package committer) documents.
Best Practices and Configuration Mangement
Do not ever make a system change to quickly “make something work.” We all run into situations where production services are down, most likely due to a change of some sort, and they need to be brought back up quickly. It’s tempting to modify things and forget about them, or promise to document the changes later, but fail to do so. All changes must go into your configuration management system, e.g. Puppet.
One important thing to mention here is that you absolutely must not use Puppet to fight with your package manager. Configuration management wasn’t invented to repeatedly overwrite configuration files when software packages stomp over them. If you’re using automation tools to do exactly what we’re saying shouldn’t be done, you’re doing it wrong.
What, then, is the right way to stomp over configuration files that you deem unworthy? First, examine why you’re doing this. If it is to replace a “.conf” file when in reality you could just place your configuration snippet in a “.conf.d” directory, then simply re-implement properly and life will be much happier.
There are times when your software of choice just isn’t configured properly, or when you need software that isn’t available in your distro’s package repositories. Do not download random RPMs off the Internet (outside of the distro-supported or community-supported repositories). If you find something on rpmfind.net, even if it says it’s for your distro, it’s risky to install. Did you know packages can execute arbitrary shell scripts during install? At any rate, the answer is: Build your own packages and run a local repository for your custom builds.
Some of these things may seem like a lot of work. Configuration management is work. Building your own packages is work. Learning way more about your Linux distro than you ever though possible, is definitely work. But in the end, you can be certain that servers will be configured exactly as you expect. Security patches or even distro version upgrades (sometimes) can be applied with little worry that all the services you’ve configured will break. In the end, it’s a lot less work.
When he’s not writing for Enterprise Networking Planet or riding his motorcycle, Charlie Schluting works as the COO at Elevation Fitness, a Web-based fitness management platform. He also operates Longitude Technologies, which offers world-wide Linux & Network support and consulting services. Charlie also wrote Network Ninja, a must-read for every network engineer. Follow Charlie on twitter: http://twitter.com/cschluti
