Move Your E-Mail Hosting to the Cloud
Moving from self-hosted to cloud-hosted e-mail services makes a lot of people nervous, but with a little planning and good communication with your user base, there's no reason it has to be hard.
The thought of switching a large (or even small) business's e-mail to hosted providers like Google or Microsoft often meets with a lot of resistance. You might think security would rank as the number one concern, but in practice the big show stopper usually ends up being implementation. Moving millions of old e-mail messages and thousands of accounts is a daunting task.
Migrating all of your existing e-mail doesn't have to be difficult. The big three providers, Google, Microsoft, and Yahoo, all provide many options to help alleviate the pain associated with moving an entire company's e-mail infrastructure.
The decision to outsource e-mail, even to a free provider, is a big one. Once all the naysayers have been satisfied with regards to performance, security, and maintenance, the next big step is to devise a migration mechanism that won't break and allow for anyone to question the decision.
To create a seamless migration from your local mail storage to the cloud, user accounts must be moved in-tact. It's more than just the account (user and password), however. Every single mail folder must be migrated, and message timestamps need to be preserved. In short, users should not be able to detect that anything has changed.
The savings of not running 50 or more servers won't be realized if account migration and routine maintenance tasks aren't carefully considered. The big providers have APIs for creating, deleting, and generally adjusting user accounts. Most organizations have identity management systems that create accounts on various systems, including e-mail servers. Before making the big switch, make sure to extend these automation tools to begin managing e-mail users via the new provider's API.
There are four basic steps, once all the planning and other tasks mentioned above are complete. They are:
- Calculate how much time the actual transfer of mail boxes will take.
- Create the user accounts in bulk via your new provider's API or bulk upload function.
- Change MX records in DNS to point at your new provider.
Calculating how much time it will take to transfer e-mail is critical, and difficult. If users are actively using e-mail, some "sent mail" may be left behind if the timing is off, so it is important to plan when to point users at the new service. Say you have 4,000 e-mail accounts, averaging 1GB each, and your provider will allow 100 transfers per batch. You can estimate that it should take one hour per batch, so in 40 hours the migration will be complete. Ideally, don't let users access e-mail during this time, at all.
Next, you will want to create the user accounts at your provider. It is recommended that you initially set the password to something you know, to make sure you have as many options as possible for the transfer mechanism. We'll cover that in a moment.
You will want to update MX records so that new mail ends up at the provider, but only after the user accounts exist, and preferably during a large window of time that your users are dormant. Make sure to lower the TTL records well before time so then changeover is quick.
Finally, the transfer itself. With most providers, there are three options for transferring all e-mail (and folders):
- Have users do it themselves
- IMAP automation, using provider tools or your own scripts
- API insertion of mail
Having your users do it themselves really only works for small offices full of fairly technical people. Nevertheless, this is how: simply add the new IMAP account from your provider to a mail client, and drag & drop folders between accounts until everything has been copied over.
IMAP automation is the preferred method. There are two options here: do it yourself with imapsync, or use the provider's tools. Google, for example, provides IMAP mail migration for education and Premiere Google Apps accounts. You must provide Google with remote access to your IMAP server, including a list of usernames and passwords for each user to migrate. Once it has that, Google's sync tool will automatically transfer all IMAP folders over.
Getting your user's passwords can be tricky. The best method is to get them to "register" for their new mail account, at which time you can gather the plain text version of their current password. If you're authenticating users via LDAP, however, there is a cool way to work around this. Simply write a script that records a user's password hash and then sets their password to something you know. Provide that known password to Google, and then restore their previous hash once the migration is complete.
Finally, most providers have an API to facilitate creating accounts and other administrative tasks. They might also allow for the transfer of e-mail via API. If this is the case with your new provider, API transfer of e-mail folders is a good idea. You're going to need to know the provider's API to create or delete accounts anyway, so this is a good time to practice.
What Won't Work
Mail filters are the big issue. Unix-based procmail or sieve filters will be lost. If you've historically provided a Web interface to allow users to create filters, this one will probably bite you. Users that run Outlook or other mail clients, however, probably have defined filters from the client side. If filters were created server-side, you're probably going to lose them, since Google doesn't provide a migration path for filters.
When he's not writing for Enterprise Networking Planet or riding his motorcycle, Charlie Schluting works as the COO at Elevation Fitness, a Web-based fitness management platform. He also operates Longitude Technologies, which offers world-wide Linux & network support and consulting services. Charlie also wrote Network Ninja, a must-read for every network engineer. Follow Charlie on twitter: http://twitter.com/cschluti