Considering Deduplication for a Midrange Data Center? Seven Questions You Should Ask

IT managers in most midrange data centers typically have limited staff and few backup specialists, and it can be hard to figure out how deduplication might fit into their situation.

 By Steve Mackey
Page 1 of 2
Print Article
Deduplication has been one of the hottest technologies in the storage industry for almost three years. During that time, it has generated marketing wars, industry consolidation, and comments and controversy from vendors. IT managers in most midrange data centers typically have limited staff and few backup specialists, and it can be hard to figure out how deduplication might fit into their situation. Following are important questions for IT managers to ask as they consider deploying deduplication in a midrange datacentre.

1. Is data deduplication now a mainstream technology?

Yes. Deduplication appliances have absolutely made the transition from experimental to mainstream. Analysts tell us that a little over 30 per cent of IT departments use it for at least part of their data, and vendors now offer products with a couple of technology generations behind them that are optimised for simplified, non-disruptive deployment.    

However, this doesn't mean that every solution is equal. Most deduplication vendors go through a learning curve, so it pays to ask about experience, references, and support when evaluating solutions. 

2. What does deduplication really do?

Generally, deduplication is a method for finding redundant data at a sub-file level, and substituting a pointer for the repeated data. It can be used to reduce disk requirements as well as the bandwidth needed to transmit data. 

There are several different and legitimate ways of doing that—block level deduplication is the most typical, but some products find differences between file-sets at a Byte level. Different approaches may have implications for performance, the amount of working space required, how easily they can support different software applications, and ease of setting up replication. The specific approach is less important than proven results and how well the approach matches with the problem you are trying to solve.   

3. What problems are best addressed by deduplication?

The greatest leverage, and the most widespread adoption, involves backup data. That's natural since backups contain more redundancy than any other datasets and get retained longer. Most common types of office data—including email, databases, and flat files—benefit from high deduplication rates. 

Quantum recently surveyed users of its DXi-Series appliances to quantify results on the effects of deduplication when it is added to users' backup strategies. Compared to traditional storage systems, users reported an average increase in backup speeds of 125 per cent, an 87 per cent reduction in failed backups, and a huge change in restore profiles—restores that used to take several hours or days are typically reduced to minutes using deduplication. Costs are also reduced, often dramatically. Users reported that overall removable media costs dropped by an average of nearly half, the costs of retrieving tape from offsite storage were deduced by 97 per cent, and the amount of time required to manage backups was reduced by 63 per cent.  

Users that adopted remote replication for disaster recovery (DR) protection saw an increase in recovery points, automating the process and eliminating tape (and tape management) in smaller offices.  

Slide Show

10 Way to Improve Data Backup

Every aspect of the data center environment can stand a little improvement. But if your backup capabilities are like most, they are in dire need of an upgrade.

4. Does it matter what backup software I use?

Most deduplication vendors have tested their systems with different backup applications and achieved effective results. Some vendors can even optimise data storage for more than one backup application. It is worth asking a deduplication supplier whether there are applications that they have optimised around. 

Be sure to check for support for specific backup software interfaces. Symantec, for example, has developed an OpenStorage interface that works with backup appliances to provide an additional level of operational advantages—increased performance, better replication management, even direct, off-line tape creation. Ask deduplication appliance vendors about their strategic relationships with backup application suppliers. You will want to understand how closely they work together, and what their plans are for interoperability and integration in the future.

This article was originally published on Aug 23, 2010
Get the Latest Scoop with Networking Update Newsletter