Virtual Tape Libraries come of age

Quantum’s Steve Mackey examines the marriage between the two mediums and looks at how the Virtual Tape Library market is coming of age, offering end-users new options and greater utilisation of storage space and networks.

Will disk-to-disk backup supersede tape backup? That's the age-old question that has been hotly debated by the storage industry for many years. During the course of the debate we’ve seen both sides of the argument come out fighting. In one corner there was the tape media makers stating the case for tape’s long-term survival, in the other corner there was the disk vendors heralding the death of tape.

Looking at the data storage market today we see the two technologies co-existing, both finding their place in a tired storage environment.

Users agree tape remains a necessity in the data centre. For the majority of businesses, achieving a state of tapelessness is far from reality. Even organisations that embrace emerging technologies admit they still have tape libraries for archiving and disaster recovery.

The need to retain data for longer periods of time, in multiple locations and the obligation to comply with government legislation and industry regulations means that a disk-based storage solution is impractical, being both expensive and non portable.

Additionally, although the cost of disk has fallen dramatically with advances in technology and the introduction of low-cost ATA drives, comparing cost per gigabyte for large-scale solutions, disk is still more expensive than tape.

The adoption of pure disk-based backup solutions will not replace tape until the problems of long-term retention, cost and portability can be solved.

The corporate backup problem

Whilst the market for tape-based data storage solutions is continuing to grow at a steady pace, with storage analysts Freeman Reports stating global revenue for tape libraries rose 10% during 2005, virtual tape libraries (VTLs) are gaining growing acceptance in a market demanding greater speed and reliability from storage solutions.

Data proliferation and the need for shorter backup windows are leading to demands on storage administrators to balance reliable data management against the pressures of cost and compliance for their organisations. The ability to recover large volumes of data within an acceptable timeframe is also proving to be a major backup and recovery challenge for businesses.

VTLs offer a number of advantages over traditional tape backup and disk-to-disk-to-tape. The most appealing of these benefits to storage administrators is that VTLs offer faster backups and restores as the demand grows for shorter backup and recovery times and greater reliability.

Customers want to consolidate backup into a single device. The advantage of VTLs is that it is a non-disruptive, easy to manage technology. As VTLs are designed to look and feel like tape, end-users can add disk to their backup process without having to radically overhaul their backup software, where data is located and how it is replicated between sites. All procedures of which were originally designed to write to tape.

The corporate backup problem that storage administrators now face is that data is growing rapidly and is widely distributed in branch offices, remote sites and regional data centres. Today the common solution is a series of disconnected tape devices, although isolated ATA disk devices are also starting to appear. These remote locations tend to have silos of data because there may not be enough network bandwidth to let users move backup data easily and cost effectively between locations.

In addition management is a challenge because the multiple sites often have very different levels of IT expertise, many remote sites or smaller branches don’t have dedicated IT resources at all.

Storage vendors need to address the issues of eliminating redundant data and improving the utilisation of disk resources and data transport networks. Data de-duplication, wide area file services (WAFS), WAN optimisation and remote data management and protection have been some of the hottest areas in the storage industry with key acquisitions and OEM relationships being announced in the past year such as ADIC’s acquisition of Rocksoft and the EMC acquisition of Avamar.

New technologies are coming

Storage analysts see the future looking bright for VTL adoption. In its Worldwide Virtual Tape Library 2006-2011 Forecast and Analysis report, IDC forecast that the worldwide VTL market will double in value over the next five years, reaching $1.4 billion by 2011. The total VTL system terabytes shipped will increase rapidly from 99,325 TB in 2006 to 690,082 TB in 2011, a compound annual growth rate of 47.4% and open system VTL products will drive market growth and eclipse enterprise mainframe VTL market value in the year ahead.

In the same report, IDC state that in order for the market to grow, “suppliers must educate customers about the potential benefits and value proposition of VTL systems and not merely tout them solely as an alternative to tape-based data protection”.

This points to new technologies are on their way and VTLs on the market today are in for many changes.

Symantec’s recent release of NetBackup OpenStorage API, a native application programming interface (API) for the next version of NetBackup, presents the first steps to bridging the divide between the integration of VTLs and backup software.

Integration between intelligent disk devices, such as VTLs and backup software has been a challenge for the industry. Since major backup applications haven’t provided a purpose-built API for integration to date, it has been difficult for intelligent disk backup devices to offer advanced functionality beyond basic interoperability.

Symantec believes the NetBackup OpenStorage API solves vendor integration problems by allowing vendors to natively integrate with NetBackup. OpenStorage API will allow NetBackup to manage VTLs and other secondary storage devices, giving users a single view of their tiered backup environment.

The investment by Symantec in the VTL market demonstrates that the technology has a future and is more than just a short-term solution to end-users’ problems with backing up to tape.

Data de-duplication is key

Data de-duplication is considered by many industry analysts to be one of the most important storage technologies to emerge in more than a decade. Enterprise Strategy Group is convinced that eliminating data redundancy will be one of the most important technologies for all tiers of storage. Likewise The Taneja Group says this is core technology that can help fuel a range of new offerings in the backup and archival spaces.

De-duplication is powerful technology that can change the economics of the way that backup and archive data is stored on disk and moved over networks. Backup, replication, and archive operations, which include very high levels of redundancy, such as VTLs, are the first targets for the new technology.

Quantum’s distributed backup and recovery initiative will offer end-users new options enabled by Blocklets? technology, which Quantum acquired along with its developer Rocksoft, in early August 2006.

Derived from Ross Williams’ pioneering 1990’s work in encryption and compression Blocklets de-duplication technology divides datasets into variable-length elements based on natural boundaries and identifies the elements that are repeated. Blocklets then eliminates the duplicated data within files and between files, storing or transmitting each unique element a single time.

Put simply, Blocklets looks for repeated patterns and when it finds them, it stores unique elements only once. Using this technique, de-duplication allows 20 to 40 times greater utilisation of storage and networks.

The basic idea is shown by imagining how you could store all the data you need to build a garden, one with lots of repetition, ie: earth, grass, paving, yellow flowers, blue flowers and trees. You could write everything over and over and take up lots of space, or you could write the common elements once, along with a notation system to show you where to introduce the unique features.

Blocklets can turn this simple idea into a powerful tool for managing data by intelligently finding variable length blocks of data (Blocklets) and remembering what it has seen. This technique lets Blocklets-enabled solutions find the repeated blocks of data over and over again, wherever they are, ie: inside one file, in a different file, in a different type of file.

The technology, which is being integrated into Quantum products, will provide end users new options for protecting their data more effectively across multiple sites using a strategy that combines disk, tape and replication.

Blocklets technology will provide important capabilities in distributed environments. As discussed, today most backups in these environments are carried out by isolated tape or disk devices. In the Blocklets-enabled replication model, sites will have option of deploying disk-backup appliances holding up to 40 times more data than conventional disk. Data de-duplication can also replicate backup data to a central disk backup appliance for disaster recovery protection.

In the data center, the Blocklets de-duplication technology will provide equally powerful advantages. Adding the technology to an enterprise disk backup solution that already provides a fully integrated tape component, Blocklets will increase the effective amount of data that can be stored on disk by up to 40 times and still provide tape support in a single, integrated system.

Data de-duplication seems to be a technology that’s here to stay because the benefits are so powerful. Storage space is at an all-time premium and shrinking the size of the stored data volumes seems to be one of the few options left.

Storage vendors need to allow end users get value out of their existing systems, whilst taking advantage of new technologies such as data de-duplication. Users need to be vigilant about selecting the right data management solution for their needs, whether it’s disk, tape, replication, or a combination of any of them.