Recently, Cisco started saying that virtualization doesn’t actually save money due to increased management costs involved with running a virtual infrastructure. Sure, in the same glossy Cisco was selling its “unified computing” system and management tools, but it may have had a point. Sometimes it may seem like running a virtualized environment is more work.
It’s certainly different, which means there is a one-time learning curve. The ramp-up costs associated with learning management tools for a given virtualization environment are one-time costs, or sunk costs, that wouldn’t factor into the calculation of yearly IT management costs.
There are fundamental differences in the way virtualized servers are managed, however. If these differences, compared to running bare-metal servers, prove to add a substantial management overhead, there may be something to this notion. Let’s talk about four aspects of managing a virtual environment: deployment, managing changes, monitoring and tuning.
Deploying and Maintaining Changes
A deployment server for physical and virtual machines is the same thing: a network boot image that starts the kickstart (or FAI, or jumpstart) process, and configures a minimal environment. Most often it sets up the configuration management software, so that when the new OS is booted for the first time it will fetch its configuration from a central server and apply whatever changes are necessary.
Deploying virtual machines is a tad different, in reality. Your deployment system needs to be aware of the virtual environment, and it needs to know how to configure Xen or KVM’s storage and other critical details (VMware has its own thing that handles everything for you). Using Cobbler , you can deploy virtual machines with a single command, and it centralizes all your kickstart configurations for easy management of all physical and virtual machines. Regardless of the chosen tool, they exist and make deployments very easy.
After initial deployment, maintaining changes across all these new servers is really no different. You have global configuration updates that need to go everywhere, including the virtual machines. Some virtual machines will be grouped together for various configurations, such as the services they run, and you’re still only making changes in one place (your configuration management tool). When the number of virtual machines grows due to the number of applications growing, you’re going to need to evaluate whether or not a new virtual machine is warranted; more on that in a moment.
Monitoring and Tuning
As the number of server instance increases, the daily maintenance tasks do as well. When problems arise due to storage or network outages, a greater number of virtual servers may be affected. It’s critical that your monitoring tools are configured sanely to alert you with distilled information that is useful for pinpointing the actual source of the outage. It is also just as critical that newly created virtual machines are automatically added to your monitoring tool in the right group.
More virtual machines means more demand on overall infrastructure. People often enjoy running five, ten, or more virtual machines on new servers with super fast CPUs, but they never thought about upgrading the storage or network infrastructure. It’s quite easy, in fact, to saturate a gigabit network link with just one virtual machine running over iSCSI. These types of problems can be anticipated and minimized, but most new virtualization environments forget to factor in these components.
Tuning, then, becomes a more regular task. When everything ran on underutilized physical servers, most applications had plenty of resources to spare. Now it takes careful planning to ensure all virtual machines run effectively. Many virtual infrastructure management tools provide a way to automatically migrate virtual machines when CPU and other resources dwindle. Automating the powering on of new physical servers to allow more iron for the virtual machines to migrate to is highly recommended. These rules and automation tasks can be time consuming at first, but they definitely keep the environment running at top performance on minimal hardware.
To Sprawl or Not
Both of the above sections assume that because you’re virtualized, you are no doubt going to run more operating system instances. You don’t necessarily have to, but when deploying a new virtual machine is as easy as clicking a button, it’s a great method for testing or isolating services to their own servers.
When each application can be cordoned off on its own, why not? It’s generally safer and easier to manage a single-purpose operating system instance. You don’t need to worry about upgrades or changes effecting more than you expected, because each virtual machine will run one application. It greatly simplifies how we deploy applications, in many cases.
In “Get a Handle on VM Sprawl” I wrote about how managing virtual machine instances is no different than managing physical servers. When virtual machines grow uncontrolled, you’re going to have to deal with it. If you’re set up with configuration management integrated into your deployment too, which also automatically adds the new instance to your monitoring tool, the extra management overhead during deployment is zero.
When upgrade time comes, you will hopefully not be logging in and manually updating each operating system. The great thing about application-centric virtual machines is that they are generally disposable. With the application installation and configuration fully automated with your configuration management tool of choice, an upgrade should consist of deploying a new virtual machine with the latest OS.
Ongoing maintenance, admittedly, will be slightly more difficult. Load distribution among virtual machine servers needs to be carefully monitored and rules for auto-migration when loads get too high need to be constantly updated. That additional overhead, however, does not negate power, cooling, and hardware savings. It’s difficult to quantify, but my hunch is those that say it does are either trying to sell you a management tool, or are a sysadmin resisting change. The many positive factors, many of which were not even mentioned here, far outweigh the learning curve and shift in sysadmin focus that accompany virtualized environments.