What is configuration management, you ask? While some people would say that it is solely the act of “managing configuration files,” others believe it encompasses much more. Configuration management is necessary these days; there should be no question about that. We’ll talk about “why” and “what” in this article, and next week cover “how” with an introduction to a fairly new open source configuration management system.
Just a few years ago it was perfectly acceptable to recommend installation automation as the sole method of creating some semblance of coherence across a network of servers. The problem is quite obvious, though—configurations of each server will drift over time. Even after a known base installation, servers will have different roles, which will evolve over time. Documenting these changes only goes so far. Groups of servers will still become silos, and only some people will know about the one-off nuances of each server and service.
Over time, admins invent fancy ways of dealing with the need for central configurations. They store them all on a master server and use scp or rdist to manually copy them out to other boxes. This is useful, but there are much better ways. As Luke Kanies, the author of puppet once said, “ssh in a for loop just doesn’t cut it anymore.”
“But ITIL says that automating repeatable system builds is very important,” you argue. Yes, but that’s only a small piece of the overall jigsaw. It’s actually uncommon for IT shops of any size to not use kickstart or jumpstart these days. It’s just too labor-intensive to manually use a CD and click through an installer. Often these automated installation systems begin to evolve and support many different configurations for various types of servers. Shortly after the initial install, however, the configurations begin to drift.
There are two motivating factors, for sysadmins, when it comes to configuration management. First, with an effective system, the need to actually login to a server to change some aspect of its configurations diminishes. Second, the obvious ones: centrally managed configuration files, documentation, and automated changes. Those are some pretty vague concepts, but don’t worry, some examples are coming.
Configuration management systems also provide a mechanism to ensure the propagation of changes. In the old model of pushing out changes with scp, unavailable servers never get the change. It’s important to have a server get all site-wide changes immediately when it returns to service.
Configuration management software can be thought of as subscribing to one of two schools of thought. The tool can make you specify a bunch of actions that need to be taken, in hopes of obtaining some final state. This is very close to scripting changes, but with a bit of help to guarantee it happens frequently. Alternatively, the configuration management too can figure out what to do based on the final state you describe. Cfengine is an example of the former, while puppet is the latter.
Systems that know what to do, for example, when you say, “make sure user Bob exists,” need to understand each operating system they run on. The tools that are close to manually scripting things are able to run everywhere without “porting” the application. Of course this means that you’ll be copying the passwd and shadow files (or modifying the contents) instead of saying, “make Bob.” Which type system you use is a personal decision, but hopefully the benefits of higher-level specifications are clear.
What configuration management buys you is yet another level of abstraction, automation, and sanity. Service management, for example, is highly desirable. It may not be enough to simply ensure that Apache (the web server) is running—you may want to make sure that certain virtual host configurations exist on certain servers. This is about the limits of cfengine, for example, and it takes some know-how to get it set up. The goal is to have the ability to move websites to different servers, by simply changing one line in a configuration file. This works fine for virtual hosts, but if you’re wanting to do this with an SSL-based website, there’s going to be an IP address associated with it. The higher-level configuration management systems can ensure that Apache is installed, the IP address is configured, the configuration file is present, and then restart Apache. There’s quite a difference in capabilities between these two types of systems.
It’s also important to recognize that configuration management provides documentation, as mentioned above, but the type of documentation is unique and more useful than one might expect. With an effective system, lower-level admins can quickly figure out what makes a server unique. Documenting the fact that an Oracle server lives on a certain box, to many people, means that they simply need to ask the Oracle guru if a problem arises. A configuration management systems will allow other admins to see precisely what services and files make a server unique.
Don’t forget: effective change management requires proper configuration management. If an unauthorized change sneaks in, it will either be overwritten the next time the configuration management tool runs, or if it was implemented within the constraints of system, it’s easily identifiable. It’s also important to cross-reference actual changes with approved changes, both for auditing and troubleshooting.
The reasons for using a good configuration management system are plenty. Everyone certainly understands configuration drift, and hopefully the IT industry is beginning to see the need to stop manually managing servers. Every once in a while some paradigm, dare I use the word, will cause people to completely rethink the way they operate. If you’ve already implemented cfengine, as many people painstakingly did, you’re ahead of the game. You’re also behind in some aspects.
Next week we’ll dive into a quick introduction to puppet, the configuration management system of choice. It can make managing large numbers of Linux, Solaris, and even OS X servers feel like you’ve been fishing all day, instead of sitting at work.
