Data Domain Intros New Dedupe, Threatens EMC

The company's new appliance also threatens to steal some of the thunder from the expected move by EMC to unveil new deduplication technology based on products OEM'd from Quantum.

EMC, of Hopkinton, Mass., is widely believed in the storage industry to unveil the relationship with Quantum, of San Jose, Calif., possibly as early as next week at the EMC World conference in Las Vegas.

Data Domain, the Santa Clara, Calif.-based data deduplication pioneer, on Monday unveiled its DD690, one unit of which delivers up to 600 Gbytes per hour throughput, as well as aggregated throughput of up to 1.4 Tbytes per hour over multiple backup policies.

That compares to a maximum aggregate of 800 Gbytes per hour with the previous model, the DD580, said Brian Biles, vice president of product management.

Up to 16 DD690 controllers can be configured into a Data Domain DDX array, increasing throughput to over 22 Tbytes per hour and maximum capacity to up to 28 petabytes, Biles said. "This makes this the fastest in-line dedupe appliance," he said.

The DD690 supports "fan-in" replication, under which data from up to 60 remote sites can be replicated and deduped to a central site, Biles said. It also allows global dedupe of data across all sites, which means that similar data replicated from two or more sites is not stored multiple times, he said.

The DD580 will remain in production, he said.

Deduplication, also called "dedupe," removes duplicate information as data is backed up or archived. It can be done on the file level, where duplicate files are replaced with a marker pointing to one copy of the file, and/or at the sub-file or byte level, where duplicate bytes of data are removed, resulting in a significant decrease in storage capacity requirements.

Dedupe products can be classified in a couple different ways. The primary difference between them lies in where the de-dupe process takes place.

Some products dedupe the data as it is being sent across a LAN or WAN. Known as in-line dedupe, this results in fewer files and less data being sent over the network, but can affect the performance of the backup because of the processing overhead caused by the dedupe process. Data Domain features in-line dedupe technology.

Other products use post-processing dedupe in which the full data to be backed up is copied onto a destination drive, after which it is deduped. This mitigates the bottleneck by accepting the full data set and then eliminating duplicates as it is stored, but in this case the customer must have enough storage capacity to temporarily store the entire data set.

Brooks Byerly, president of Soccour Solutions LP, a Dallas-based storage solution provider and Data Domain partner, said the DD690 is a larger, more enterprise-class offering.

"We were proposing a deal where two DD580s would work, but the customer preferred a single system," Byerly said. "So we proposed a DD690, and got the early business with it."

Data Domain is known for making sure its performance rides that of the latest processors, and the DD690, with its Intel quad-core processor, is no exception, Byerly said.

Keith Norbie, director of the storage division of Nexus Information Systems, a Plymouth, Minn.-based solution provider, said the DD690's increase in scalability for the price is another notch in Data Domain's evolution.

While there is still an upper limit to how much data can be deduped and stored with the DD690, Norbie said that he would argue that Data Domain is the most scalable of the proved dedupe vendors. "There's no real challenge to Data Domain in terms of the size of referencible accounts," he said.

Biles said that, while he respects EMC as a potential competitor, he is not concerned about the possibility of that vendor doing an OEM deal with dedupe competitor Quantum. "If EMC goes with Quantum, they start with a weak hand," he said. "Quantum uses post-process de-dupe technology, so there is much more administration involved."