US20120084523A1 - Data recovery operations, such as recovery from modified network data management protocol data - Google Patents

Data recovery operations, such as recovery from modified network data management protocol data Download PDF

Info

Publication number
US20120084523A1
US20120084523A1 US13241625 US201113241625A US2012084523A1 US 20120084523 A1 US20120084523 A1 US 20120084523A1 US 13241625 US13241625 US 13241625 US 201113241625 A US201113241625 A US 201113241625A US 2012084523 A1 US2012084523 A1 US 2012084523A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
data
storage
ndmp
chunk
stream
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US13241625
Other versions
US9244779B2 (en )
Inventor
Duncan A. Littlefield
Vimal K. Nallathambi
Girish Chanchlani
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CommVault Systems Inc
Original Assignee
CommVault Systems Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1448Management of the data involved in backup or backup restore
    • G06F11/1453Management of the data involved in backup or backup restore using de-duplication of the data
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1458Management of the backup or restore process
    • G06F11/1469Backup restoration techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601Dedicated interfaces to storage systems
    • G06F3/0602Dedicated interfaces to storage systems specifically adapted to achieve a particular effect
    • G06F3/0614Improving the reliability of storage systems
    • G06F3/0619Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601Dedicated interfaces to storage systems
    • G06F3/0628Dedicated interfaces to storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/064Management of blocks
    • G06F3/0641De-duplication techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601Dedicated interfaces to storage systems
    • G06F3/0628Dedicated interfaces to storage systems making use of a particular technique
    • G06F3/0646Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/065Replication mechanisms
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601Dedicated interfaces to storage systems
    • G06F3/0628Dedicated interfaces to storage systems making use of a particular technique
    • G06F3/0662Virtualisation aspects
    • G06F3/0665Virtualisation aspects at area level, e.g. provisioning of virtual or logical volumes
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from or digital output to record carriers, e.g. RAID, emulated record carriers, networked record carriers
    • G06F3/0601Dedicated interfaces to storage systems
    • G06F3/0668Dedicated interfaces to storage systems adopting a particular infrastructure
    • G06F3/067Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1448Management of the data involved in backup or backup restore
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/84Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

The systems and methods herein permit storage systems to correctly perform data recovery, such as direct access recovery, of Network Data Management Protocol (“NDMP”) backup data that was modified prior to being stored in secondary storage media, such as tape. For example, as described in greater detail herein, the systems and methods may permit NDMP backup data to be encrypted, compressed, deduplicated, and/or otherwise modified prior to storage. The systems and methods herein also permit a user to perform a precautionary snapshot of the current state of data (e.g., primary data) prior to reverting data to a previous state using point-in-time data.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • [0001]
    This application claims the benefit of the assignee's pending U.S. Patent Application No. 61/388,554, filed Sep. 30, 2010, entitled DATA RECOVERY OPERATIONS, SUCH AS RECOVERY FROM MODIFIED NETWORK DATA MANAGEMENT PROTOCOL DATA (attorney docket number 606928084US), which is hereby incorporated herein by reference in its entirety.
  • BACKGROUND
  • [0002]
    Current storage management systems employ a number of different methods to perform storage operations on electronic data. For example, data can be stored in primary storage as a primary copy that includes production data, or in secondary storage as various types of secondary copies including, as a backup copy, a snapshot copy, a hierarchical storage management copy (“HSM”), as an archive copy, and as other types of copies.
  • [0003]
    A primary copy of data is generally a production copy or other “live” version of the data that is used by a software application and is generally in the native format of that application. Primary copy data may be maintained in a local memory or other high-speed storage device that allows for relatively fast data access. Primary copy data is typically intended for short term retention (e.g., several hours or days) before some or all of the data is stored as one or more secondary copies, for example to prevent loss of data in the event a problem occurred with the data stored in primary storage.
  • [0004]
    Secondary copies include point-in-time data and are typically intended for long-term retention (e.g., weeks, months or years depending on retention criteria), before some or all of the data is moved to other storage or is discarded. Secondary copies may be indexed so users can later browse, search and restore the data. After primary copy data is backed up, a pointer or other location indicia such as a stub may be placed in the primary copy to indicate the current location of that data. Further details may be found in the assignee's U.S. Pat. No. 7,107,298, filed Sep. 30, 2002, entitled SYSTEM AND METHOD FOR ARCHIVING OBJECTS IN AN INFORMATION STORE (Attorney Docket No. 60692-8003US1).
  • [0005]
    One type of secondary copy is a backup copy. A backup copy is generally a point-in-time copy of the primary copy data stored in a backup format as opposed to in native application format. For example, a backup copy may be stored in a backup format that is optimized for compression and efficient long-term storage. Backup copies generally have relatively long retention periods and may be stored on media with slower retrieval times than other types of secondary copies and media (e.g., on magnetic tape), or be stored at on offsite location.
  • [0006]
    Another form of secondary copy is a snapshot copy. From an end-user viewpoint, a snapshot may be thought as a bitmap or instant image of the primary copy data at a given point in time. A snapshot may capture the directory structure of a primary copy volume at a particular moment in time, and may also preserve file attributes and contents. In some embodiments, a snapshot may exist as a virtual file system, parallel to the actual file system. Users may gain a read-only access to the record of files and directories of the snapshot. By electing to restore primary copy data from a snapshot taken at a given point in time (e.g., via a reversion process), users may also return the current file system to the prior state of the file system that existed when the snapshot was taken.
  • [0007]
    A snapshot may be created instantly, using a minimum of file space, but may still function as a conventional file system backup. A snapshot may not actually create another physical copy of all the data, but may simply create pointers that map files and directories to specific disk blocks and that indicate which blocks have changed. The snapshot may be a copy of a set of files and/or directories as they were at a particular point in the past. That is, the snapshot is an image, or representation, of a volume of data at a point in time. A snapshot may be as a secondary copy of a primary volume of data, such as data in a file system, an Exchange server, a SQL database, an Oracle database, and so on. The snapshot may be an image of files, folders, directories, and other data objects within a volume, or an image of the blocks of the volume.
  • [0008]
    Snapshots may be created using various techniques, such as copy-on-write, redirect-on-write, split mirror, copy-on-write with background copy, log structure file architecture techniques, continuous data protection techniques, and/or other techniques. Once a snapshot has been taken, subsequent changes to the file system typically do not overwrite the blocks in use at the time of a snapshot. Therefore, the initial snapshot may use only a small amount of disk space to record a mapping or other data structure representing or otherwise tracking the blocks that correspond to the current state of the file system. Additional disk space is usually only required when files and directories are actually modified later. Furthermore, when files are modified, typically only the pointers which map to blocks are copied when taking a new snapshot, not the blocks themselves. For example in the case of copy-on-write snapshots, when a block changes in primary storage, the block is copied to secondary storage before the block is overwritten in primary storage and the snapshot mapping of file system data is updated to reflect the changed block(s) at that particular point in time, e.g., the pointer in that snapshot now points to the old block now in secondary storage.
  • [0009]
    Data storage systems may utilize snapshots for a variety of reasons. One typical use of snapshots is to copy a volume of data without disabling access to the volume for a long period. After performing the snapshot, the data storage system can then copy the data set by leveraging the snapshot of the data set. As another example, a data storage system may use a snapshot and/or other point-in-time secondary copies (e.g., copies generated from a snapshot) to permit a user to revert data back to its state at a specific point in time during a reversion process.
  • [0010]
    An HSM copy is generally a copy of the primary copy data, but which typically includes only a subset of the primary copy data that meets a certain criteria and is usually stored in a format other than the native application format. For example, an HSM copy might include only that data from the primary copy that is larger than a given size threshold or older than a given age threshold and that is stored in a backup format. Often, HSM data is removed from the primary copy, and an address, pointer or stub is stored in the primary copy to indicate its new location. When a user requests access to the HSM data that has been removed or migrated, systems use the stub to locate the data and often make recovery of the data appear transparent even though the HSM data may be stored at a location different from the remaining primary copy data.
  • [0011]
    An archive copy is generally similar to an HSM copy, however the data satisfying criteria for removal from the primary copy is generally completely removed with no stub left in the primary copy to indicate the new location (i.e., where it has been moved to). Archive copies of data are generally stored in a backup format or other non-native application format. In addition, archive copies are generally retained for very long periods of time (e.g., years) and in some cases are never deleted. Such archive copies may be made and kept for extended periods in order to meet compliance regulations or for other permanent storage applications.
  • [0012]
    Application data over its lifetime typically moves from more expensive quick access storage to less expensive slower access storage. This process of moving data through these various tiers of storage is sometimes referred to as information lifecycle management (“ILM”). This is the process by which data is “aged” from more expensive forms of secondary storage with faster access/restore times down through less expensive secondary storage with slower access/restore times, for example, as the data becomes less important or mission critical.
  • [0013]
    In some embodiments, storage management systems may perform additional operations upon copies, including deduplication, content indexing, data classification, data mining or searching, electronic discovery (E-discovery) management, collaborative searching, encryption and compression.
  • [0014]
    One example of a system that performs storage operations on electronic data that produce such copies is the Simpana storage management system by CommVault Systems of Oceanport, New Jersey. The Simpana system leverages a modular storage management architecture that may include, among other things, storage manager components, client or data agent components, and media agent components as further described in U.S. Pat. No. 7,246,207, filed Apr. 5, 2004, entitled “SYSTEM AND METHOD FOR DYNAMICALLY PERFORMING STORAGE OPERATIONS IN A COMPUTER NETWORK.” The Simpana system also may be hierarchically configured into backup cells to store and retrieve backup copies of electronic data as further described in U.S. Pat. No. 7,395,282, filed Jul. 15, 1999, entitled “HIERARCHICAL BACKUP AND RETRIEVAL SYSTEM.”
  • [0015]
    The Simpana system and other storage systems may perform backup and Direct Access Recovery (“DAR”) storage operations under the Network Data Management Protocol (“NDMP”), an open standard protocol for backups of heterogeneous network-attached storage across an enterprise. Under the NDMP standard, during backup, an NDMP data server is responsible for creating backup data and sending it to an NDMP mover in a data stream format specified by the NDMP protocol. To the NDMP mover, the data stream may appear to be simply a raw stream of bytes or bits. The NDMP mover is then responsible for writing the data stream to backup or secondary storage media, such as tape. The NDMP mover may be on the same physical machine as the data server, or different machine. During a restore or recovery of a backed-up data object, the NDMP data server is responsible for requesting NDMP-formatted backup data from the mover and restoring the data object to a target location from that backup data, e.g., a target location in primary storage. To request a backup copy of a data object, the NDMP data server sends an offset and length that identify the location of the data object in the original NDMP data stream that was sent to the NDMP mover at backup. Using the offset and length information provided by the NDMP data server, the NDMP mover retrieves the desired data from the backup media and returns it to the NDMP data server in the form of an NDMP-formatted data stream.
  • [0016]
    Unfortunately, NDMP standards do not readily facilitate restore operations if the NDMP mover modified the NDMP data stream via encryption, compression, deduplication, etc., before writing the data to tape or other secondary storage media. These modification techniques may alter the data in an unpredictable way. For example, when an NDMP data stream is deduplicated and/or compressed, the total size of the modified data that must be stored is typically much smaller than the size of the original NDMP data stream. However, the modified data is not simply a linearly “scaled down” version of the original data stream. Instead, the original data stream is scaled down unevenly in a manner that depends on the contents of the original data stream and/or the types of modification techniques that are applied to the original data stream. Since these modification techniques alter the data in an unpredictable manner, at the time of restore, the NDMP mover can no longer use the offset and length provided by the NDMP data server to correctly retrieve and return requested data objects. For example, if a data object was originally represented in an original NDMP backup data stream at offset OF1 and length L1, the modified version of that object may instead be stored in modified form with an offset OF2 and length L2; furthermore, there may be no closed-form mathematical relationship to automatically derive OF2 and L2 from OF1 and L1. Thus, if the data mover receives a request from an NDMP data server to retrieve an object using offset and length values OF1 and L1, the data mover may be unable to fulfill the request.
  • [0017]
    The Simpana system and other storage systems may also permit users to perform a reversion operation in order to return client data to a previous state at a specified point in time by using a previously obtained point-in-time copy, such as a snapshot copy or other secondary copy. However, this reversion operation will effectively erase all changes to that data that were made after the specified point in time. Thus, such a reversion operation is irreversible, since a user cannot undo the reversion operation in order to return data to its state at the time the reversion operation was performed.
  • [0018]
    The need exists for systems and methods that overcome the above problems, as well as systems and methods that provide additional benefits. Overall, the examples herein of some prior or related systems and methods and their associated limitations are intended to be illustrative and not exclusive. Other limitations of existing or prior systems and methods will become apparent to those of skill in the art upon reading the following detailed description.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • [0019]
    FIG. 1 is a block diagram illustrating an environment in which aspects of the invention may be configured to operate.
  • [0020]
    FIG. 2 is a block diagram illustrating components of a data stream that may be used in a suitable data storage system.
  • [0021]
    FIG. 3 is a flow diagram illustrating a process for performing an NDMP backup operation, wherein the NDMP data stream is modified prior to storage.
  • [0022]
    FIG. 4 is an example chunk mapping table that shows how a secondary storage computing device may provide a mapping between physical and logical locations of modified NDMP backup data.
  • [0023]
    FIG. 5 is a flow diagram illustrating a process for restoring NDMP data that has been modified prior to storage.
  • [0024]
    FIG. 6 shows a process for reverting data to a previous state, wherein the reversion process is reversible.
  • [0025]
    FIG. 7 illustrates an example graphical interface for permitting a user to perform a reversion operation from point-in-time data that is reversible.
  • [0026]
    FIG. 8 is a block diagram illustrating an example of a data storage system.
  • [0027]
    In the drawings, the same reference numbers and acronyms identify elements or acts with the same or similar functionality for ease of understanding and convenience.
  • DETAILED DESCRIPTION
  • [0028]
    The headings provided herein are for convenience only and do not necessarily affect the scope or meaning of the claimed invention.
  • Overview
  • [0029]
    The systems and methods described herein permit storage systems to correctly perform direct access recovery of NDMP backup data that was modified prior to its storage in secondary storage media, such as tape. For example, as described in greater detail herein, the systems and methods may permit NDMP backup data to be encrypted, compressed, deduplicated, and/or otherwise modified prior to storage. During a direct access recovery operation, the systems and methods described herein permit an NDMP data server to use index information (such as logical offsets and/or lengths) generated by the NDMP data server in order to request an unmodified version of NDMP data that was previously stored in modified form.
  • [0030]
    The systems and methods described herein also permit a user to perform a precautionary snapshot of the current state of data (e.g., primary data) prior to reverting data to a previous state using point-in-time data. In this way, a reversion process becomes reversible, instead of irreversible, because data that would otherwise be erased or overwritten by a reversion process is first captured via a snapshot operation.
  • [0031]
    Various examples of the invention will now be described. The following description provides specific details for a thorough understanding and enabling description of these examples. One skilled in the art will understand, however, that the system may be practiced without many of these details. Additionally, some well-known structures or functions may not be shown or described in detail, so as to avoid unnecessarily obscuring the relevant description of the various examples.
  • [0032]
    The terminology used in the description presented below is intended to be interpreted in its broadest reasonable manner, even though it is being used in conjunction with a detailed description of certain specific examples of the system. Certain terms may even be emphasized below; however, any terminology intended to be interpreted in any restricted manner will be overtly and specifically defined as such in this Detailed Description.
  • Suitable Environments
  • [0033]
    FIG. 1 is a block diagram illustrating an environment 100 in which aspects of the invention may be configured to operate. The environment 100 includes one or more clients 130, one or more primary data stores 160, a secondary storage computing device 165 (or alternatively “media agent”), and one or more storage devices 115. Each of the clients 130 is a computing device, examples of which are described herein. The clients 130 are each connected to one or more associated primary data stores 160 and to the secondary storage computing device 165. The secondary storage computing device is connected to a storage device 115. The primary data stores and storage device may each be any type of storage suitable for storing data, such as Directly-Attached Storage (DAS) such as hard disks, a Storage Area Network (SAN), e.g., a Fibre Channel SAN, an iSCSI SAN or other type of SAN, Network-Attached Storage (NAS), a tape library, or any other type of storage. The clients 130 and the secondary storage computing device 165 typically include application software to perform desired operations and an operating system on which the application software runs. The clients 130 and the secondary storage computing device 165 typically also include a file system that facilitates and controls file access by the operating system and application software. The file system facilitates access to local and remote storage devices for file or data access and storage.
  • [0034]
    The clients 130, as part of their functioning, utilize data, which includes files, directories, metadata (e.g., access control lists (ACLs), descriptive metadata, and any creation/edit dates associated with the data), and other data objects, which may be stored in an associated primary data store 160. (More details as to the storage operations involving ACLs may be found in the assignee's U.S. patent application Ser. No. 12/058,518, entitled “SYSTEM AND METHOD FOR STORAGE OPERATION ACCESS SECURITY” (Attorney Docket No. 60692.8042US02), the entirety of which is incorporated by reference herein.)) The data of a client 130, which is stored in a primary data store 160, is generally a primary copy (e.g., a production copy). Although described as a “client” of the secondary storage computing device 165, a client 130 may in fact be a production server, such as a file server or Exchange server, which provides live production data to multiple user workstations as part of its function. During a copy, backup, snapshot, archive or other storage operation, the clients 130 send a copy of data objects in a primary data store 160 to the secondary storage computing device 165.
  • [0035]
    Some clients 130, such as client 1 and client 2, may include an NDMP data server 106 configured to permit the client to perform NDMP backup and direct access recovery (DAR) (or “restore”) storage operations, as described in greater detail herein. Some clients, such as client 1 and client N, may include a snapshot component 108 configured to permit the client to perform snapshot operations, including taking snapshots and performing data reversions from a snapshot or other point-in-time data, as described in greater detail herein. As shown in FIG. 1, a single client may comprise an NDMP data server 106, a snapshot component 108, and/or both. Of course, clients may also contain other components that perform other functions.
  • [0036]
    The secondary storage computing device 165 may include various components that perform various functions. These components include an NDMP mover 170, a control module 168, a media daemon 172, a snapshot module 174, and an interface module 176. The NDMP mover permits the secondary storage computing device to perform NDMP backup and direct access recovery (DAR) storage operations, as described in greater detail herein. The NDMP mover is configured to communicate with the NDMP data server 106 during NDMP backup and restore operations. The control module is configured to control NDMP backup and restore operations, and the media daemon is configured to store and retrieve a modified version of an NDMP data stream in the storage device 115. The NDMP mover and/or media daemon are also configured to modify data received from the NDMP data server via operations such as deduplication, encryption, modification, indexing, and/or the addition of metadata, before the data is stored in the storage device 115. The snapshot module 174 is configured to facilitate snapshot operations upon data (e.g., data stored in primary data stores 160), and to permit a client 130 to revert to earlier versions of data using snapshots or other point-in-time data. The interface module 176 is configured to present user interfaces that permit a user to initiate a reversion operation and select options associated with that reversion. The functionality of each of these components is described in greater detail herein.
  • [0037]
    Together, the NDMP data server 106 on a client 130 and the NDMP mover 170 on the secondary storage computing device 165 facilitate NDMP backup and restore operations of the primary data in the client's associated primary data store 160. Turning to an NDMP backup operation first, generally speaking, the control module 168 communicates with the NDMP data server in order to configure and initiate an NDMP copy operation or “backup job” of a set of primary data from the primary data store (such as a volume, sub-client, or file system). For example, the control module may instruct the NDMP data server to package the set of primary data into an NDMP backup data stream and to send the NDMP data stream to a specified data port on the secondary storage computing device. The configuration of the backup operation, including which set of data is backed up and other configuration parameters, may be specified by a storage policy and/or schedule policy, as described in greater detail herein. The control module may also cause the NDMP mover 170 to begin listening for the incoming NDMP backup data stream on the specified port. In accordance with the control module's instructions and NDMP protocols, the NDMP data server packages up the specified set of primary data from the primary data store into an NDMP-formatted backup data stream and begins sending the data stream to the NDMP mover, which receives the data stream, e.g., at the specified port. The precise format of the NDMP data stream may be dependent upon the type of NDMP data server that is sending the stream, so that effectively the NDMP mover cannot parse or discern a file-level or directory-level organization within the NDMP data stream. Instead, from the NDMP mover's perspective, the NDMP data stream may be seen simply as an ordered collection, or stream, of raw bytes or bits that needs to be stored. Any contiguous subset of data within the NDMP data stream may be described by a “logical offset” and “logical length” that together indicate the position of the subset within the larger stream (e.g., at an offset from the origin or head of the stream).
  • [0038]
    As the NDMP backup data stream is received, the NDMP mover 170 takes additional steps to modify the data stream and store the modified data in a manner that permits later retrieval and restoration of a portion of the original received NDMP backup data stream. The NDMP mover 170 may set up a data pipeline from the NDMP mover to the media daemon 172 to convey the received data stream (or a modified form thereof) to the media daemon. The NDMP mover and/or the media daemon may perform modifying operations upon the received NDMP data stream (e.g., encryption, deduplication, compression, indexing, adding metadata). The NDMP mover may perform modifying operations before sending the modified data to the media daemon and/or the media daemon may modify the data it receives from the NDMP mover. The media daemon writes the modified data to a storage device 115 in chunks. As described in greater detail herein, the media daemon also generates mapping information for each chunk. The mapping information associates each chunk of modified data stored with (1) a “physical offset” and a “physical length” that reflect where the chunk of modified data is physically stored within the storage device, and (2) a logical offset and a logical length that are associated with the contiguous subset of the unmodified NDMP backup data stream that was received by the NDMP mover and subsequently modified to create the chunk of modified data. In other words, the mapping information correlates the original NDMP offset and length parameters with new physical locations on the storage device.
  • [0039]
    While it is packaging data and sending the NDMP backup data stream during a backup job, the NDMP data server 106 may also locally generate and/or store index information or file history information; alternatively, or additionally, the NDMP data server may embed such information within the NDMP backup data stream (e.g., at the beginning or end of the data stream). The index information or file history information may associate each data object (e.g., file, directory, or sub-file data object) embodied within the data stream with a logical offset and logical length of the data object within the original data stream (these values indicate the object's position or location in the stream) and/or provide information about directory structures or other logical organizations that may also be defined by logical offset and logical length descriptors. The NDMP data server may also intermittently or periodically provide such index information to the NDMP mover 170; for example, it may send the index information for a batch of objects in the backup job. The media daemon 172 may store this index information, e.g., in the storage device 115. However, the NDMP data server typically provides its index information post-hoc, after the related portion of the data stream has already been received, modified, and/or stored by the NDMP mover and/or media daemon. Thus, typically, the media daemon cannot use the received index information to generate a mapping of data objects to physical offsets in real-time as the modified data is being written to the storage device 115.
  • [0040]
    Turning now to an NDMP direct access recovery or restore operation, at a later time, the NDMP data server 106 may restore data objects that were backed up during a backup operation to a target location (e.g., a target location in primary data store 160). As part of the restoration, the NDMP mover may retrieve and echo back stored information about the NDMP backup operation or job, such as indexing information or file history information that was generated and sent by the NDMP data server 106 at the time of the backup operation. Using this index information (or other index information, e.g., information stored locally by the NDMP data server), the NDMP data server may send one or more requests for a portion of a specified NDMP backup stream by providing a desired logical offset and a logical length that identify a desired contiguous subset of the original, unmodified NDMP backup data stream sent during the backup job. The NDMP mover 170 may then send the provided logical offset and logical length to the media daemon, which utilizes the mapping information to translate the provided logical offset and logical length into physical offsets and physical lengths that reflect where a modified version of the requested subset of the data stream has been physically stored in the storage device 115. Using those physical offsets and lengths, the media daemon may retrieve or read the modified data from the storage device 115, convert the retrieved modified data back into an NDMP-formatted data stream that contains the backup data associated with the logical offset and logical length, and send the desired portion of the NDMP-formatted data stream back to the NDMP data server, e.g., via the NDMP mover. More details about NDMP backup and restore operations, including mapping information, are provided herein with respect to FIGS. 3, 4, and 5.
  • [0041]
    Normally, a reversion process that involves reverting data to a previous state using point-in-time data such as a snapshot is “irreversible” because the reversion process may erase and/or overwrite all or portions of a table that indicates the current state of data (e.g., an active block table or block map) at the moment just before the reversion process is initiated and/or may overwrite blocks of data. Thus, effectively, a reversion process irreversibly erases or overwrites current data, since that data is no longer accessible to a user. As described in greater detail herein, the secondary storage computing device 165, including its snapshot module 174 and/or interface module 176, and/or the snapshot component 108 on a client 130 may permit a user to perform a precautionary snapshot of the current state of data (e.g., in a primary data store 160) prior to reverting data to a previous state using point-in-time data. In this way, the reversion process becomes reversible instead of irreversible, because data that would otherwise be erased or overwritten by a reversion process is captured via a snapshot.
  • [0042]
    The snapshot component 108 on the client 130 creates snapshots of the client's data. The snapshot component 108 includes software components and may also include hardware and/or firmware components. The snapshot component 108 may be provided in its entirety by a single entity (for example, a single vendor), or the snapshot component 108 may include sub-components that are provided by different entities (such as multiple vendors). In some examples, the snapshot component includes a Microsoft Volume Shadow Copy Service (VSS) sub-component and a software-based VSS provider sub-component that is provided by the assignee of the present application, CommVault Systems, Inc. In these examples, a data agent (not shown) associated with the client interacts with the Microsoft VSS sub-component to create snapshots. In other examples, in addition to or as an alternative to the software-based VSS provider sub-component, the snapshot component 108 includes other software-based VSS provider sub-components, such as a Microsoft system software provider, a Microsoft Data Protection Manager provider sub-component or a NetApp SnapManager provider sub-component. In other examples, in addition to the Microsoft VSS sub-component, the snapshot component 108 includes one or more hardware-based VSS provider sub-components, such as those provided by vendors such as Hewlett-Packard, EMC, NetApp, IBM, and other vendors. Those of skill in the art will understand that the snapshot component 108 may include various software-based and/or hardware-based sub-components, and may interact with other components in various ways in order to create snapshots of a client's data.
  • [0043]
    The snapshot component 108 may create snapshots using various techniques, such as copy-on-write, redirect-on-write, split mirror, copy-on-write with background copy, log structure file architecture techniques, continuous data protection techniques, and/or other techniques. The snapshot component 108 may also perform operations relating to reversion operations. The snapshot component 108 may store the created snapshots on a particular volume of the client 130.
  • [0044]
    The snapshot module 174 may also copy snapshots from the client 130 to another storage device, such as the storage device 115, and/or maintain index information for snapshots. The snapshot module may also utilize snapshots from a client to create other point-in-time secondary copies of a client's data. The snapshot module may also be configured to perform functions similar to the snapshot component 108.
  • [0045]
    The secondary storage computing device 165 also includes the interface module 176. The interface module 176 provides access to the copied snapshot data. The interface module 176 can be used to access data objects created in other types of secondary copies, such as backup copies, archive copies, and other types of copies. The interface module 176 can also be used to display to a user available snapshots or point-in time-copies of the data that can be used for reversion, recovery, or other purposes.
  • [0046]
    Snapshot operations and reversions from snapshots and other secondary copies are described in greater detail in commonly assigned applications U.S. Provisional Application No. 61/291,805, filed Dec. 31, 2009, entitled “SYSTEMS AND METHODS FOR ANALYZING SNAPSHOTS” (Attorney Docket No. 60692.8072US00), U.S. Provisional Application No. 61/291,803, filed Dec. 31, 2009, entitled “SYSTEMS AND METHODS FOR PERFORMING DATA MANAGEMENT OPERATIONS USING SNAPSHOTS” (Attorney Docket No. 60692.8074US00), and U.S. application Ser. No. 12/558,947, filed Sep. 14, 2009, entitled “USING A SNAPSHOT AS A DATA SOURCE,” now U.S. Published Application No. US 20100070726 (Attorney Docket No. 60692.8053US01), all of which are hereby incorporated herein in their entirety.
  • [0047]
    The above system may be incorporated within a data storage system and may be subjected to or receive a data stream during a data copy operation. Referring to FIG. 2, a block diagram illustrating components of a data stream 110 utilized by a suitable data storage and recovery system is shown. The data stream 110 may include client 130, a secondary storage computing device 165, and a storage device 115. For example, in storage operations, the system may store, receive, and/or prepare data to be stored, copied, or backed up at a client. The system may then transfer the data to be stored to the secondary storage computing device, which may then refer to storage policies, schedule policies, and/or retention policies (and other policies) in order to choose a storage device. The secondary storage computing device may include or be associated with an intermediate component, to be discussed herein.
  • [0048]
    The storage device 115 receives the data from the secondary storage computing device 165 and stores the data as a secondary copy, such as a backup copy, archive copy, and/or snapshot copy. Secondary storage devices may be magnetic tapes, optical disks, USB and other solid-state memory media, disk, and tape drives, and so on.
  • NDMP Backup Operation
  • [0049]
    FIG. 3 is a flow diagram illustrating a process 300 for performing an NDMP backup operation, wherein the NDMP backup data stream is modified prior to its storage. The processes shown in FIGS. 3 and 5 may be performed by components of the secondary storage computing device 165 (e.g., NDMP mover 170 and/or media daemon 172), and/or other systems or components.
  • [0050]
    Before beginning the process 300, a component of the secondary storage computing device 165, such as the control module 168, may trigger or initiate the backup operation or job in accordance with applicable storage and/or schedule policies, and/or at the request of other system components (such as a storage manager, discussed herein). For example, the control module may instruct an NDMP data server 106 resident on a client 130 to initiate an NDMP backup operation upon a subset of the client's data storage (e.g., a volume, sub-client, and/or file system) in an associated primary data store 160 and to send an NDMP backup data stream to a specified port. As another example, the control module may start an NDMP mover 170 process for the backup job and direct the NDMP mover to listen for an NDMP backup data stream at a specified port and/or may start a media daemon 172 process for the job. The control module may also take other steps, such as further configuring the NDMP backup operation (e.g., identifying a storage device 115 to use during the operation).
  • [0051]
    As shown, the process 300 begins at block 305, when the NDMP mover 170 receives an unmodified NDMP backup data stream from an NDMP data server 106 resident on a client 130, such as in response to an instruction sent to the NDMP data server. For example, the NDMP mover 170 may receive the stream by listening at a specified port.
  • [0052]
    At block 310, the NDMP mover 170 and/or another component such as the media daemon 172 modifies the received NDMP backup data stream by various modification procedures, such as the modification procedures described below. The examples of data modification procedures provided below are intended to be illustrative, not exhaustive, and any suitable data modification techniques may be employed at block 310. Furthermore, at block 310, the NDMP mover 170 or other components may perform any combination of modification procedures, such as those described below, and may perform modification procedures in any order. For example, more details regarding how the secondary storage computing device 165 can simultaneously support encryption, compression, and/or deduplication may be found in the assignee's U.S. application Ser. No. 12/145,342, filed Jun. 24, 2008, entitled “APPLICATION-AWARE AND REMOTE SINGLE INSTANCE DATA MANAGEMENT,” now U.S. Published Application No. US 20090319585 (Attorney Docket No. 60692.8056US00). The NDMP mover 170 may perform some or all of the modification procedures before sending the data to the media daemon 172. Alternatively, or additionally, the media daemon may receive the NDMP data stream (either unmodified or modified) from the NDMP mover and perform modification procedures upon the data stream it receives.
  • [0053]
    As one example, at block 310, the NDMP mover 170 or the media daemon 172 may encrypt the received data stream using well-known techniques. For example, the system may employ encryption techniques that satisfy Federal Information Processing Standards (FIPS). Further details about encryption and encrypting copies of data may be found in commonly assigned U.S. Patent Publication No. US2008-0320319A1, filed on Mar. 31, 2008, entitled SYSTEM AND METHOD FOR ENCRYPTING SECONDARY COPIES OF DATA (Attorney Docket No. 60692-8041US3). As another example, the NDMP mover may compress the received data stream using well-known techniques. For example, the system may employ many different well-known techniques or applications for compressing data, including Lempel-Ziv (LZ) techniques, DEFLATE techniques, and LZ-Renau (LZR) techniques. As yet another example, at block 310, the NDMP mover may add additional metadata to the data stream.
  • [0054]
    As yet another example, at block 310, the NDMP mover 170 or the media daemon 172 may perform deduplication upon the received data stream, either before or after encryption and/or compression. Generally speaking, deduplication refers to storing a single or reduced number of instances of a unique data object or unique data block (or other multiple-byte or multiple-bit unit of data) in secondary storage. For example, the system may store in secondary storage one or more instances, but fewer than the total number of instances, of each unique data object or unique data block.
  • [0055]
    Overall, at block 310, the NDMP mover 170 or the media daemon 172 may perform object-level, sub-object-level, block- level or multi-byte-/multi-bit-level deduplication procedures and/or encryption procedures and/or compression procedures via techniques that create “chunk folders” and similar deduplication data structures.
  • [0056]
    In examples described in more detail in the assignee's U.S. patent application Ser. No. 12/565,576, filed Sep. 23, 2009, entitled “SYSTEMS AND METHODS FOR MANAGING SINGLE INSTANCING DATA,” now U.S. Published Application No. US 20100082672 (Attorney Docket No. 60692.8067US01), the NDMP mover 170 or the media daemon 172 stores the modified data in a data structure comprising a chunk folder when performing object- or sub-object-level deduplication. Contained within the chunk folder are three container files: 1) a metadata file; 2) an “N” file; and 3) an “S” file. The three files are each logical containers of data. The “S” file stores deduplicated data (e.g., deduplicated files). The “N” file stores data that is not deduplicated (e.g., metadata, such as descriptive metadata associated with deduplicated files). The metadata file stores references to the location(s) of the data objects in the “S” file and the “N” file. Note that although three container files are described (S, N, and metadata index), a chunk folder may comprise more than one “S” file (e.g., S1, S2 . . . Sy, where y is an integer) to store deduplicated data and/or more than one “N” file (e.g., N1, N2 . . . Nz, where z is an integer). While described as being stored on the storage device 115, the “N” and metadata files may alternatively or additionally be stored elsewhere, such as on the secondary storage computing device and/or with a storage manager, described herein. The chunk folder and the container files may be equivalent to a directory and files (or folder and files) on a file system. For example, the chunk folder may be a directory and the container files may be files located within the directory. As another example, the chunk folder may be a file and the container files may be portions of the file. As another example, the container files may be collections of blocks, bytes, or bits grouped together. Those of skill in the art will understand that the chunk folder and the container files may be comprised in various data structures and are not limited to a directory and files within the directory.
  • [0057]
    During deduplication, an index file may be created within a data structure that consists of one or more stream headers and stream data. The stream header describes a data object contained in an “N” file or an “S” file (e.g., its location, its size, an offset within the file, etc.). The stream data contains the pointer to the data object contained in the “N” file or the “S” file. For example, the pointer may give its location within the “N” file or the “S” file. The location of the data object may be given by offsets within the “N” file or the “S” file. For example, its location may be given by a starting offset, and its length or size. As another example, its location may be given by a starting offset and an ending offset. As previously mentioned, the data object may be in an “S” file in another chunk folder, and the stream data would point to this “S” file in the other chunk folder (e.g., give its location in the “S” file in the other chunk folder). Each time the NDMP mover 170 or media daemon 172 places a data object in the “S” file, it may add a stream header and corresponding stream data to the index file.
  • [0058]
    In examples where block-level deduplication is performed at block 310, the data structures used to store modified data may include one or more volume folders, one or more chunk folders within a volume folder, and multiple files within a chunk folder. Each chunk folder includes a metadata file, a metadata index file, one or more container files, and a container index file. The metadata file stores non-deduplicated data blocks as well as links to deduplicated data blocks stored in container files. The metadata index file stores an index to the data in the metadata file. The container files store deduplicated data blocks. The container index file stores an index to the container files. Among other things, the container index file stores an indication of whether a corresponding block in a container file is referred to by a link in a metadata file. Those of skill in the art will understand that the volume folder and its constituent folders and files may be comprised in various data structures and are not limited to a directory and files within the directory; instead, e.g., they may be collections of blocks, bytes, or bits grouped together.
  • [0059]
    In some examples, chunk folders, volume folders, and/or their constituent files are stored in an archive file format. The archive file may be stored on various storage devices 115, such as on disk drives or on magnetic tapes, and may contain chunk folders and/or volume folders. The archive file may include, for example, a “chunk folder 0” located at offset 0, a “chunk folder 1” located at offset 5, a “chunk folder 2” located at offset 10, a “chunk folder 3” located at offset 15, and so on until a “chunk folder n” located at offset 65. The offsets may be expressed in relation to the start of the archive file. More details regarding a suitable archive file format may be found in the assignee's U.S. application Ser. No. 11/950,376, filed Dec. 4, 2007, entitled “SYSTEMS AND METHODS FOR CREATING COPIES OF DATA, SUCH AS ARCHIVE COPIES,” now U.S. Published Application US 20080229037 (Attorney Docket No. 60692-8037.US01), the entirety of which is incorporated by reference herein. An archive file may be considered as a container of data objects.
  • [0060]
    At block 310, modification procedures may be performed separately upon different subsets of the NDMP backup data stream. For example the procedures may be performed separately upon different subsets of the NDMP backup data stream that are associated with different “chunks,” as described in greater detail herein. For example, the NDMP mover 170 may perform a modification procedure or procedures (e.g., compression or encryption) upon the first 3 GB of an NDMP data stream as a subset, and then perform the same modification procedure upon the next 3 GB of an NDMP data stream as a subset.
  • [0061]
    Generally speaking, the modification operations performed at block 310 will create an unknown and unpredictable change in the volume of data that is needed to store the modified form of the received NDMP backup data stream. For example, if the received unmodified NDMP data stream is 1 GB, the volume of data needed to store the modified version of the NDMP data stream may be 1 GB, 0.75 GB, 0.2 GB, or any other value. The degree of change will depend on the contents of the data stream, the configuration settings, and the types and/or order of the modification operations that are performed at block 310. For example, deduplication will change (e.g., reduce) the volume of data that is stored during a backup or other storage operation; however, the amount of change is a function of the amount of redundancy that is present in a primary data set. As an example, if a received data set of 1 GB has 50% redundancy (e.g., each unique block appears twice), block-level deduplication may reduce the amount of data that is stored by approximately 50%. As another example, the level of compression achieved by a compression procedure may depend on the particular compression technique utilized. As yet another example, the addition of metadata prior to storage may increase the volume of the data that is stored. Furthermore, the modification procedures may create uneven volume changes through the entire NDMP data stream. For example, a deduplication procedure might reduce the first half of an NDMP data stream by a factor of two, but might reduce the second half of the NDMP data stream by a factor of three. Since the modification procedures change the volume of data in an unpredictable and/or uneven fashion, the logical offsets associated with the received original NDMP data stream may have no closed-form or predictable relationship to the offsets of a post-modification NDMP data stream or image.
  • [0062]
    At block 315, the NDMP mover 170 or media daemon 172 writes the modified data in “chunks” to the storage device 115. For example, the media daemon may write chunks of modified data to tape media. Each chunk written is a modified version of a contiguous subset of the received NDMP backup data stream that is associated with a certain logical offset and logical length.
  • [0063]
    In some examples, each “chunk” written at block 315 is a separate “chunk folder” and/or “volume folder” such as those deduplication data structures described previously. Alternatively, a “chunk” may be a sub-division of one of these deduplication data structures (e.g., a logical sub-division, a constituent file or folder, or an arbitrary sub-division (e.g., each chunk folder may be divided into approximately 0.5 GB chunks), a modified version of one of these deduplication data structures (e.g., a compressed version of a chunk folder), or a group of multiple deduplication data structures. In other examples, a “chunk” may simply refer to a subset of the modified data which is addressable (i.e., readable) by a media location, a physical offset, and a physical length (or a physical starting offset and physical ending offset). The length of each such chunk may be configurable. For example, at approximately every 1 GB “checkpoint” along the modified data stream, the media daemon 172 may simply define a new chunk.
  • [0064]
    In some examples, for each chunk, the media daemon 172 writes approximately 2 GB of modified data to the storage device 115. However, any chunk size may be used, and chunk size may be configurable, e.g., to tune the performance of restore operations. A storage policy may specify chunk size. Furthermore, during a single backup operation, chunk size may vary from one chunk to the next. Chunk size is typically expressed in terms of the amount of modified data that is written per chunk; however, the size of a chunk may instead be specified by or based on the amount of unmodified data that is associated with a given chunk and/or other characteristics of the unmodified data that is used to create a chunk. For example, each chunk may be chosen to correspond to approximately 5 GB of unmodified data, resulting in unevenly sized chunks being written during an operation. When chunks are stored in physical media, the series of chunks may or may not be stored sequentially or contiguously. Chunks also may or may not be demarcated or separated by separators such as file marks or separate file marks.
  • [0065]
    As described previously, the modifying operations performed at block 310 may destroy any relationship between the logical offset used by the NDMP data server 106 to identify a subset of the original NDMP data stream and the offsets of a post-modification NDMP data stream or image. Accordingly, at block 320, the NDMP mover 170 or media daemon 172 associates each chunk written with (1) the logical offset and logical length of the contiguous subset of the received original NDMP data stream that is now represented in modified form by the stored chunk, and (2) the physical offset and physical length indicative of where the chunk is stored within the storage device 115 (and possibly a physical media identifier that indicates the physical media on which the chunk is stored, such as a particular tape cassette). For example, at block 320, the NDMP mover or media daemon 172 may populate a table, such as the one shown in FIG. 4.
  • [0066]
    At a later time, the modified data that is written at block 315 may be moved from one physical location to a different physical location during a subsequent storage operation. For example, during a scheduled archive storage operation, some portion of the modified data written at block 315 may be copied or transferred to a new archive file on a new tape. Thus, at decision block 325, the NDMP mover or media daemon 172 determines whether one or more chunks of data that were written at block 315 are being moved to a changed physical location during a storage operation. If not, then the process 300 returns. Otherwise, if one or more chunks are being moved, then at block 330 the NDMP mover or media daemon updates the physical offsets, physical lengths, and/or physical media identifiers to indicate the new physical storage locations of those chunks. The process 300 then returns.
  • [0067]
    FIG. 4 shows an example chunk mapping table 400 that shows how a secondary storage computing device 165 may provide a mapping between physical and logical locations of modified NDMP backup data. Each row 425, 430, and 435 corresponds to a different chunk that has been written to a storage device 115 during an NDMP backup operation or job. The chunk mapping table may comprise one or more columns, including:
  • [0068]
    (1) a job identifier (ID) column 402 indicative of the NDMP backup operation or job during which the chunk was written,
  • [0069]
    (2) a chunk ID column 405 indicative of a unique chunk identifier within an NDMP backup job,
  • [0070]
    (3) a logical offset and length column 410 indicative of the portion of the unmodified NDMP backup data stream that was received during a backup job, and that is now represented in a modified form by the physical chunk;
  • [0071]
    (4) a physical media ID column 415 indicative of the media upon which the chunk is stored (e.g., a tape ID number), and
  • [0072]
    (5) a physical offset and length column 420 indicative of a portion of the indicated physical media (e.g., tape) on which the chunk is physically stored.
  • [0073]
    The logical offset and length in the table are typically the same as the logical offset and length that the NDMP data server 106 would associate with the same subset of the unmodified NDMP backup data stream. For example, row 425 conveys mapping information for the first modified data chunk that was written as part of NDMP backup job number “A5629.” This chunk corresponds to a modified version of the subset of the unmodified NDMP backup data stream for the job that is identified by the logical offset LO1 and the logical length LL1. As shown in row 430, the chunk of modified data was written to physical media in the storage device 115 (e.g., tape) identified as “Media A,” with a physical offset PO1 and physical length PL1. Of course, the secondary storage computing device 165 may store such mapping information in any other suitable format (e.g., a relational database), and/or some of the mapping information shown in Table 4 may be stored in other components within a storage system (e.g., such as in a storage manager described herein).
  • NDMP Restore Operation
  • [0074]
    FIG. 5 is a flow diagram illustrating a process 500 for restoring original NDMP backup data that was modified prior to storage. Prior to the beginning of the process, the NDMP data server 106 may determine a logical offset and logical length that it wishes to request from the NDMP mover 170 by using index information or file history information that relates backed up data objects to their logical offset and logical length within an NDMP backup data stream and/or provides information about directory structures or similar organizational information, etc. In some examples, the NDMP data server consults an index comprising this information that is stored at the NDMP data server to determine the logical offset and logical length needed to restore a data object or objects. In other examples, the NDMP data server may request that the control module provide such index information that was sent to the control module and stored during a particular job, and the NDMP mover may provide the requested index information to the NDMP data server. The NDMP data server may then use the provided index information to determine the logical offset and logical length needed to restore a data object or set of data objects. As yet another example, the NDMP data server may request the first N bytes (e.g., at logical offset zero) of the NDMP backup data stream for the job (or another portion of the NDMP data stream) if the NDMP data server embedded indexing information directly into the NDMP data stream. The NDMP data server may then parse out the embedded indexing information to determine the logical offset and logical length needed to restore specific data objects.
  • [0075]
    The process begins at block 505, when the NDMP mover 170 receives a request (e.g., a read request) to send a subset of an unmodified NDMP backup data stream associated with a particular backup operation or job. In order to identify the specific subset needed, the request may specify for example, a backup job identifier and/or a logical offset and logical length. The logical offset requested (“LOR”) at block 505 may correspond to or fall at the beginning, end, or middle of a data chunk that was created and stored during the backup operation. Furthermore the logical offset may correspond to or fall within the first, last, or an intermediate data chunk created during the specified job. Similarly, the specified logical offset and the logical length requested (“LLR”) may span more than one data chunk from the specified job.
  • [0076]
    At block 510, the NDMP mover 170 or media daemon 172 looks up the first chunk that is needed to satisfy the read request using the logical offset and logical length and any job identifier. To do so, the NDMP mover may use a job identifier and the logical offset and logical length as a lookup into a chunk mapping table or database, such as the example table shown in FIG. 4, to determine the chunk identifier associated with the chunk that spans the first part of the subset of the NDMP backup data stream that was requested. In the example shown in FIG. 4, if the NDMP mover receives a request to access the NDMP backup stream associated with job A5629 at a logical offset LOR that falls between LO2 and LO3, the NDMP mover may determine that the beginning of the desired subset of the data stream is stored somewhere in the chunk having the chunk ID 2, which is described by row 430 in the table 400.
  • [0077]
    At block 515, the NDMP mover 170 or media daemon 172 uses the chunk identifier determined at block 510 to look up the physical media identifier, physical offset, and physical length necessary to access the first needed chunk. To do so, the NDMP mover may use the chunk identifier as a lookup into a chunk mapping table or database, such as the example table shown in FIG. 4, to determine the physical media identifier, physical offset, and physical length associated with the first needed chunk. Continuing the previous example of Table 4, if the NDMP mover receives a request to access a logical offset LOR that falls between LO2 and LO3, the NDMP mover may access row 430 to determine that a modified form of the requested data is stored in the chunk stored on media associated with the identifier “Media B” at physical offset PO2 and physical length PL2.
  • [0078]
    At block 520, the NDMP mover 170 or media daemon 172 uses the retrieved media identifier, physical offset, and physical length retrieved at block 515 in order to read the chunk of modified data. In the previous example, the NDMP mover or media daemon may access a tape B and begin reading at physical offset PO2 until it has read data corresponding to physical length PL2.
  • [0079]
    At block 525, the NDMP mover 170 and/or media daemon 172 converts the chunk of modified data that is read at block 520 back into an unmodified NDMP data stream format by undoing or reversing each of the modifying procedures that was performed upon the original NDMP data stream. For example, the NDMP mover or media daemon may decompress compressed data, decrypt encrypted data, “rehydrate” deduplicated data by restoring multiple instances of data objects to the data stream, and/or remove metadata that was added during the backup operation. Various methods for reversing these modifying procedures are well-known in the art and/or are described in the commonly-assigned applications related to modification techniques that are incorporated by reference herein.
  • [0080]
    At block 530, the NDMP mover 170 or media daemon 172 scans the converted version of the chunk until it arrives at the logical offset needed to satisfy the data request. In the previous example, if the received requested logical offset LOR is equal to LO2+217, the NDMP mover may scan the converted version of chunk 2 until it reaches position 217 within the converted chunk. The NDMP mover may discard the preceding portion of the converted version of the chunk or cache it for later use (e.g., in local memory in order to respond quickly to anticipated read requests).
  • [0081]
    At block 535, the NDMP mover 170 uses the converted chunk to return an unmodified NDMP data stream starting at the requested logical offset, e.g., by sending it to a requesting NDMP data server 106. In the previous example, if the received requested logical offset LOR is equal to LO2+217, the NDMP mover may send a copy of the converted chunk starting at position 217. The NDMP mover will continue to send the unmodified NDMP data stream until the read request has been satisfied (i.e., the logical length requested has been reached) and/or the end of the converted chunk is reached.
  • [0082]
    Of course, blocks 520-535 may be performed in parallel to the extent that the modified data can be read, converted, scanned, and sent in a parallel manner. For example, the NDMP mover 170 and/or media daemon 172 may be reading the end of a stored chunk at the same time that it is converting a middle portion of the same chunk and sending the converted version of the first part of the same chunk. Similarly, the data that is being read, converted, scanned, and/or returned, may be buffered in any manner at any of blocks 520-535.
  • [0083]
    At decision block 540, the NDMP mover 170 or media daemon 172 determines whether it is necessary to read, convert, scan, and return a portion of another data chunk. For example, the NDMP mover may determine that the requested logical offset and logical length implicates more than one data block, because the requested subset of the NDMP data stream spans more than one chunk. If the NDMP mover determines that it needs to read another data chunk to satisfy the read request, the process may be repeated beginning at block 510, this time using an adjusted logical offset and logical length that reflect how a portion of the last chunk was read, converted, and sent to partially satisfy the read request.
  • [0084]
    Of course, different iterations of blocks 520-535 may be performed in parallel. For example, the NDMP mover 170 and/or media daemon 172 may be reading the beginning of a third physical chunk at the same time that it is converting a middle portion of a second physical chunk and sending the converted version of a first physical chunk.
  • [0085]
    In some examples, the NDMP mover 170 anticipates future read requests by buffering unmodified NDMP backup data that is obtained by reading and converting stored chunks. For example, at block 535, once the entire subset of the requested NDMP backup data stream corresponding to the requested logical offset LOR and logical length LLR has been sent, the NDMP mover may continue to buffer the contents of the converted chunk until it reaches the end of the converted chunk (or another point in the converted chunk). As another example, even if the read request does not require that the NDMP mover repeat steps 510-535 for another chunk (e.g., because the read request has been fully satisfied), the NDMP mover, at the conclusion of block 535, may look up the chunk that corresponds to the next or otherwise adjacent section of the NDMP backup data stream for the same job and perform blocks 515-535. While doing so, it may buffer the NDMP data generated at block 535. By doing so, the NDMP mover may be able to satisfy subsequent read requests more quickly, since often a series of read requests related to a restore operation implicate several subsets of a backup data stream that are near to one another.
  • [0086]
    Although the description of NDMP operations herein have primarily referred to sets of data as being associated with or addressable by an offset (e.g., either a physical offset or logical offset) and length (e.g., either a physical length or logical length), one having skill in the art will appreciate that this is equivalent to associating or addressing a subset of data by a starting offset and an ending offset, since these two quantities together inherently indicate a length.
  • Reversion Operation
  • [0087]
    FIG. 6 shows a process 600 for reverting data, such as primary data stored in a primary data store 160, to a previous state, wherein the reversion process is reversible. The process may be performed by the secondary storage computing device 165 or components thereof (such as the snapshot module 174 and/or interface module 176), other components (such as a snapshot component 108 on a client 130), and/or other systems.
  • [0088]
    As shown, the process begins at block 605, when the secondary storage computing device 165 receives a request to revert data to its earlier state at a previous time using previously obtained point-in-time data, such as an earlier snapshot copy or backup copy. For example, a user may utilize a graphical user interface, (e.g., provided by the interface module 176) to browse previous snapshots of client data. Those snapshots and other secondary copies available to the user may be stored in a primary data store 160 and/or may be stored in snapshot format or another secondary format in a secondary storage device, such as storage device 115. The user may indicate that he wishes to revert to all of the data captured by a particular snapshot or secondary copy or only a particular subset of the data captured by a particular snapshot or secondary copy (e.g., a particular logical unit, disk volume, file server volume, file, email object, etc.).
  • [0089]
    At block 610, the secondary storage computing device 165 presents the user with the option to take a snapshot of the current state of data that will be partially or wholly overwritten or erased as a result of the requested reversion process, e.g., a snapshot of a logical unit of data that will be partially or wholly overwritten. For example, using the interface module 176, the secondary storage computing device may present the user with an interface such as the example interface 700 shown in FIG. 7. As shown in FIG. 7, the interface may include a warning message 705 that indicates that the requested reversion operation may erase or overwrite data (and may indicate what data will be erased/overwritten). The interface may provide a message 710 asking the user whether he wants to take a snapshot of the current state of the data that will be overwritten or erased and provides the user with input means 715, 720 such as check boxes, drop-down menus, “right-click” or similar interactive menus, or similar means to indicate whether a current snapshot should be taken before the requested reversion operation. The interface may also include a message and input means 725 for the user to select other options related to reversion operations, such as the option of always taking a current snapshot before reverting.
  • [0090]
    Referring again to FIG. 6, at decision block 615, the secondary storage computing device determines whether the user has requested that a current snapshot be taken. If yes, the process 600 proceeds to block 620; otherwise, the process proceeds to block 625. At block 620, the secondary storage computing device 165 takes (or requests the taking of) a snapshot of the current state of the data that will be erased or overwritten by the reversion operation (and possibly additional primary data; e.g., during a granular reversion operation, other data in the same logical unit as the data that will be overwritten/erased will also be captured by the precautionary snapshot). For example, the secondary storage computing device may instruct the snapshot module 174 on the secondary storage computing device 165 and/or the snapshot component 108 on a client 130 to initiate a snapshot, e.g., of a logical unit that is being reverted. Prior to taking a snapshot of the current state of the data that will be erased or overwritten by the reversion operation (or data in a related logical unit), the data may be put into a consistent state and brought offline (i.e., so that it is unavailable for modifications).
  • [0091]
    At block 625, the secondary storage computing device 165 reverts the requested data using the point-in-time data (e.g., the snapshot or other secondary copy) indicated at block 605 using procedures such as those described previously and/or other techniques known in the art. For example, the secondary storage computing device may instruct the snapshot module 174 on the secondary storage computing device 165 and/or the snapshot component 108 on a client 130 to initiate a reversion operation, e.g., of a logical unit that is being reverted. After the reversion, the requested data may be brought back online.
  • [0092]
    Although not shown in FIG. 6, after the conclusion of the process 600, a user may “undo” the reversion process. For example, a user may utilize a graphical user interface, (e.g., provided by the interface module 176) to indicate that he wishes to undo the reversion operation. As another example, the user may browse previous snapshots and other point-in-time copies of a client's data, including the precautionary snapshot taken in conjunction with a reversion operation during the process 600. The user may then indicate that he wishes to revert using the precautionary snapshot. In either example, the secondary storage computing device 165 may revert the data a second time using the precautionary snapshot (or alternatively, the secondary storage computing device may perform the process 600 again, this time using the precautionary snapshot as the requested reversion point).
  • Suitable System
  • [0093]
    FIG. 8 illustrates an example of one arrangement of resources in a computing network, comprising a data storage system 250. The resources in the data storage system 250 may employ the processes and techniques described herein. The system 250 includes a storage manager 205, one or more data agents 295, one or more secondary storage computing devices 265, one or more storage devices 215, one or more computing devices 230 (called clients 230), one or more data or information stores 260 and 262, a single instancing database 223, an index 211, a jobs agent 220, an interface agent 225, and a management agent 231. The system 250 may represent a modular storage system such as the CommVault QiNetix system, and also the CommVault GALAXY backup system, available from CommVault Systems, Inc. of Oceanport, NJ, aspects of which are further described in the commonly-assigned U.S. patent application Ser. No. 09/610,738, now U.S. Pat. No. 7,035,880, the entirety of which is incorporated by reference herein. The system 250 may also represent a modular storage system such as the CommVault Simpana system, also available from CommVault Systems, Inc.
  • [0094]
    The system 250 may generally include combinations of hardware and software components associated with performing storage operations on electronic data. Storage operations include copying, backing up, creating, storing, retrieving, and/or migrating primary storage data (e.g., data stores 260 and/or 262) and secondary storage data (which may include, for example, snapshot copies, backup copies, hierarchical storage management (HSM) copies, archive copies, and other types of copies of electronic data stored on storage devices 215). The system 250 may provide one or more integrated management consoles for users or system processes to interface with in order to perform certain storage operations on electronic data as further described herein. Such integrated management consoles may be displayed at a central control facility or several similar consoles distributed throughout multiple network locations to provide global or geographically specific network data storage information.
  • [0095]
    In one example, storage operations may be performed according to various storage preferences, for example, as expressed by a user preference, a storage policy, a schedule policy, and/or a retention policy. A “storage policy” is generally a data structure or other information source that includes a set of preferences and other storage criteria associated with performing a storage operation. The preferences and storage criteria may include, but are not limited to, a storage location, relationships between system components, network pathways to utilize in a storage operation, data characteristics, compression or encryption requirements, preferred system components to utilize in a storage operation, a single instancing or variable instancing (or deduplication) policy to apply to the data, and/or other criteria relating to a storage operation. For example, a storage policy may indicate that certain data is to be stored in the storage device 215, retained for a specified period of time before being aged to another tier of secondary storage, copied to the storage device 215 using a specified number of data streams, etc.
  • [0096]
    A “schedule policy” may specify a frequency with which to perform storage operations and a window of time within which to perform them. For example, a schedule policy may specify that a storage operation is to be performed every Saturday morning from 2:00 a.m. to 4:00 a.m. In some cases, the storage policy includes information generally specified by the schedule policy. (Put another way, the storage policy includes the schedule policy.) A “retention policy” may specify how long data is to be retained at specific tiers of storage or what criteria must be met before data may be pruned or moved from one tier of storage to another tier of storage. Storage policies, schedule policies and/or retention policies may be stored in a database of the storage manager 205, to archive media as metadata for use in restore operations or other storage operations, or to other locations or components of the system 250.
  • [0097]
    The system 250 may comprise a storage operation cell that is one of multiple storage operation cells arranged in a hierarchy or other organization. Storage operation cells may be related to backup cells and provide some or all of the functionality of backup cells as described in the assignee's U.S. patent application Ser. No. 09/354,058, now U.S. Pat. No. 7,395,282, which is incorporated herein by reference in its entirety. However, storage operation cells may also perform additional types of storage operations and other types of storage management functions that are not generally offered by backup cells.
  • [0098]
    Storage operation cells may contain not only physical devices, but also may represent logical concepts, organizations, and hierarchies. For example, a first storage operation cell may be configured to perform a first type of storage operations such as HSM operations, which may include backup or other types of data migration, and may include a variety of physical components including a storage manager 205 (or management agent 231), a secondary storage computing device 265, a client 230, and other components as described herein. A second storage operation cell may contain the same or similar physical components; however, it may be configured to perform a second type of storage operation, such as storage resource management (SRM) operations, and may include monitoring a primary data copy or performing other known SRM operations.
  • [0099]
    Thus, as can be seen from the above, although the first and second storage operation cells are logically distinct entities configured to perform different management functions (i.e., HSM and SRM, respectively), each storage operation cell may contain the same or similar physical devices. Alternatively, different storage operation cells may contain some of the same physical devices and not others. For example, a storage operation cell configured to perform SRM tasks may contain a secondary storage computing device 265, client 230, or other network device connected to a primary storage volume, while a storage operation cell configured to perform HSM tasks may instead include a secondary storage computing device 265, client 230, or other network device connected to a secondary storage volume and not contain the elements or components associated with and including the primary storage volume. (The term “connected” as used herein does not necessarily require a physical connection; rather, it could refer to two devices that are operably coupled to each other, communicably coupled to each other, in communication with each other, or more generally, refer to the capability of two devices to communicate with each other.) These two storage operation cells, however, may each include a different storage manager 205 that coordinates storage operations via the same secondary storage computing devices 265 and storage devices 215. This “overlapping” configuration allows storage resources to be accessed by more than one storage manager 205, such that multiple paths exist to each storage device 215 facilitating failover, load balancing, and promoting robust data access via alternative routes.
  • [0100]
    Alternatively or additionally, the same storage manager 205 may control two or more storage operation cells (whether or not each storage operation cell has its own dedicated storage manager 205). Moreover, in certain embodiments, the extent or type of overlap may be user-defined (through a control console) or may be automatically configured to optimize data storage and/or retrieval.
  • [0101]
    Data agent 295 may be a software module or part of a software module that is generally responsible for performing storage operations on the data of the client 230 stored in data store 260/262 or other memory location. Each client 230 may have at least one data agent 295 and the system 250 can support multiple clients 230. Data agent 295 may be distributed between client 230 and storage manager 205 (and any other intermediate components), or it may be deployed from a remote location or its functions approximated by a remote process that performs some or all of the functions of data agent 295.
  • [0102]
    The overall system 250 may employ multiple data agents 295, each of which may perform storage operations on data associated with a different application. For example, different individual data agents 295 may be designed to handle Microsoft Exchange data, Lotus Notes data, Microsoft Windows 2000 file system data, Microsoft Active Directory Objects data, and other types of data known in the art. Other embodiments may employ one or more generic data agents 295 that can handle and process multiple data types rather than using the specialized data agents described above.
  • [0103]
    If a client 230 has two or more types of data, one data agent 295 may be required for each data type to perform storage operations on the data of the client 230. For example, to back up, migrate, and restore all the data on a Microsoft Exchange 2000 server, the client 230 may use one Microsoft Exchange 2000 Mailbox data agent 295 to back up the Exchange 2000 mailboxes, one Microsoft Exchange 2000 Database data agent 295 to back up the Exchange 2000 databases, one Microsoft Exchange 2000 Public Folder data agent 295 to back up the Exchange 2000 Public Folders, and one Microsoft Windows 2000 File System data agent 295 to back up the file system of the client 230. These data agents 295 would be treated as four separate data agents 295 by the system even though they reside on the same client 230.
  • [0104]
    Alternatively, the overall system 250 may use one or more generic data agents 295, each of which may be capable of handling two or more data types. For example, one generic data agent 295 may be used to back up, migrate and restore Microsoft Exchange 2000 Mailbox data and Microsoft Exchange 2000 Database data while another generic data agent 295 may handle Microsoft Exchange 2000 Public Folder data and Microsoft Windows 2000 File System data, etc.
  • [0105]
    Data agents 295 may be responsible for arranging or packing data to be copied or migrated into a certain format such as an archive file. Nonetheless, it will be understood that this represents only one example, and any suitable packing or containerization technique or transfer methodology may be used if desired. Such an archive file may include metadata, a list of files or data objects copied, the file, and data objects themselves. Moreover, any data moved by the data agents may be tracked within the system by updating indexes associated with appropriate storage managers 205 or secondary storage computing devices 265. As used herein, a file or a data object refers to any collection or grouping of bytes of data that can be viewed as one or more logical units.
  • [0106]
    Generally speaking, storage manager 205 may be a software module or other application that coordinates and controls storage operations performed by the system 250. Storage manager 205 may communicate with some or all elements of the system 250, including clients 230, data agents 295, secondary storage computing devices 265, and storage devices 215, to initiate and manage storage operations (e.g., backups, migrations, data recovery operations, etc.).
  • [0107]
    Storage manager 205 may include a jobs agent 220 that monitors the status of some or all storage operations previously performed, currently being performed, or scheduled to be performed by the system 250. (One or more storage operations are alternatively referred to herein as a “job” or “jobs.”) Jobs agent 220 may be communicatively coupled to an interface agent 225 (e.g., a software module or application). Interface agent 225 may include information processing and display software, such as a graphical user interface (“GUI”), an application programming interface (“API”), or other interactive interface through which users and system processes can retrieve information about the status of storage operations. For example, in an arrangement of multiple storage operations cells, through interface agent 225, users may optionally issue instructions to various storage operation cells regarding performance of the storage operations as described and contemplated herein. For example, a user may modify a schedule concerning the number of pending snapshot copies or other types of copies scheduled as needed to suit particular needs or requirements. As another example, a user may employ the GUI to view the status of pending storage operations in some or all of the storage operation cells in a given network or to monitor the status of certain components in a particular storage operation cell (e.g., the amount of storage capacity left in a particular storage device 215).
  • [0108]
    Storage manager 205 may also include a management agent 231 that is typically implemented as a software module or application program. In general, management agent 231 provides an interface that allows various management agents 231 in other storage operation cells to communicate with one another. For example, assume a certain network configuration includes multiple storage operation cells hierarchically arranged or otherwise logically related in a WAN or LAN configuration. With this arrangement, each storage operation cell may be connected to the other through each respective interface agent 225. This allows each storage operation cell to send and receive certain pertinent information from other storage operation cells, including status information, routing information, information regarding capacity and utilization, etc. These communications paths may also be used to convey information and instructions regarding storage operations.
  • [0109]
    For example, a management agent 231 in a first storage operation cell may communicate with a management agent 231 in a second storage operation cell regarding the status of storage operations in the second storage operation cell. Another illustrative example includes the case where a management agent 231 in a first storage operation cell communicates with a management agent 231 in a second storage operation cell to control storage manager 205 (and other components) of the second storage operation cell via management agent 231 contained in storage manager 205.
  • [0110]
    Another illustrative example is the case where management agent 231 in a first storage operation cell communicates directly with and controls the components in a second storage operation cell and bypasses the storage manager 205 in the second storage operation cell. If desired, storage operation cells can also be organized hierarchically such that hierarchically superior cells control or pass information to hierarchically subordinate cells or vice versa.
  • [0111]
    Storage manager 205 may also maintain an index, a database, or other data structure 211. The data stored in database 211 may be used to indicate logical associations between components of the system, user preferences, management tasks, media containerization and data storage information or other useful data. For example, the storage manager 205 may use data from database 211 to track logical associations between secondary storage computing device 265 and storage devices 215 (or movement of data as containerized from primary to secondary storage).
  • [0112]
    Generally speaking, the secondary storage computing device 265, which may also be referred to as a media agent, may be implemented as a software module that conveys data, as directed by storage manager 205, between a client 230 and one or more storage devices 215 such as a tape library, a magnetic media storage device, an optical media storage device, or any other suitable storage device. In one embodiment, secondary storage computing device 265 may be communicatively coupled to and control a storage device 215. A secondary storage computing device 265 may be considered to be associated with a particular storage device 215 if that secondary storage computing device 265 is capable of routing and storing data to that particular storage device 215.
  • [0113]
    In operation, a secondary storage computing device 265 associated with a particular storage device 215 may instruct the storage device to use a robotic arm or other retrieval means to load or eject a certain storage media, and to subsequently archive, migrate, or restore data to or from that media. Secondary storage computing device 265 may communicate with a storage device 215 via a suitable communications path such as a SCSI or Fibre Channel communications link. In some embodiments, the storage device 215 may be communicatively coupled to the storage manager 205 via a SAN.
  • [0114]
    Each secondary storage computing device 265 may maintain an index, a database, or other data structure 261 that may store index data generated during storage operations for secondary storage (SS) as described herein, including creating a metabase (MB). For example, performing storage operations on Microsoft Exchange data may generate index data. Such index data provides a secondary storage computing device 265 or other external device with a fast and efficient mechanism for locating data stored or backed up. Thus, a secondary storage computing device index 261, or a database 211 of a storage manager 205, may store data associating a client 230 with a particular secondary storage computing device 265 or storage device 215, for example, as specified in a storage policy, while a database or other data structure in secondary storage computing device 265 may indicate where specifically the data of the client 230 is stored in storage device 215, what specific files were stored, and other information associated with storage of the data of the client 230. In some embodiments, such index data may be stored along with the data backed up in a storage device 215, with an additional copy of the index data written to index cache in a secondary storage device. Thus the data is readily available for use in storage operations and other activities without having to be first retrieved from the storage device 215.
  • [0115]
    Generally speaking, information stored in cache is typically recent information that reflects certain particulars about operations that have recently occurred. After a certain period of time, this information is sent to secondary storage and tracked. This information may need to be retrieved and uploaded back into a cache or other memory in a secondary computing device before data can be retrieved from storage device 215. In some embodiments, the cached information may include information regarding format or containerization of archives or other files stored on storage device 215.
  • [0116]
    One or more of the secondary storage computing devices 265 may also maintain one or more single instance databases 223. Single instancing, a method of deduplication, generally refers to storing in secondary storage only a single instance of each data object (or data block) in a set of data (e.g., primary data). More details as to single instancing may be found in one or more of the following commonly-assigned U.S. patent applications: 1) U.S. application Ser. No. 11/269,512, filed Nov. 7, 2004, entitled “SYSTEM AND METHOD TO SUPPORT SINGLE INSTANCE STORAGE OPERATIONS,” now U.S. Published Application No. US 20060224846 (Attorney Docket No. 60692.8023US00); 2) U.S. application Ser. No. 12/145,347, filed Jun. 24, 2007, entitled “APPLICATION-AWARE AND REMOTE SINGLE INSTANCE DATA MANAGEMENT,” now U.S. Published Application No. US 20090319534 (Attorney Docket No. 60692.8057US00); 3) U.S. Application No. 12/145,342, filed Jun. 24, 2008, entitled “APPLICATION-AWARE AND REMOTE SINGLE INSTANCE DATA MANAGEMENT,” now U.S. Published Application No. US 20090319585 (Attorney Docket No. 60692.8056US00), 4) U.S. application Ser. No. 11/963,623, filed Dec. 21, 2007, entitled “SYSTEM AND METHOD FOR STORING REDUNDANT INFORMATION,” now U.S. Published Application No. US 20080243879 (Attorney Docket No. 60692.8036U502); and 5) U.S. application Ser. No. 11/950,376, filed Dec. 4, 2007, entitled “SYSTEMS AND METHODS FOR CREATING COPIES OF DATA SUCH AS ARCHIVE COPIES,” now U.S. Published Application No. US 20080229037 (Attorney Docket No. 60692.8037US01), each of which is incorporated by reference herein in its entirety.
  • [0117]
    In some examples, the secondary storage computing devices 265 maintain one or more variable instance databases. Variable instancing, a method of deduplication, generally refers to storing in secondary storage one or more instances, but fewer than the total number of instances, of each data block (or data object) in a set of data (e.g., primary data). More details as to variable instancing may be found in the commonly-assigned U.S. Provisional Application No. 61/164,803, filed Mar. 30, 2009, entitled “STORING A VARIABLE NUMBER OF INSTANCES OF DATA OBJECTS” (Attorney Docket No. 60692.8068US00).
  • [0118]
    In some embodiments, certain components may reside and execute on the same computer. For example, in some embodiments, a client 230 such as a data agent 295, or a storage manager 205, coordinates and directs local archiving, migration, and retrieval application functions as further described in the previously-referenced U.S. patent application Ser. No. 09/610,738. This client 230 can function independently or together with other similar clients 230.
  • [0119]
    As shown in FIG. 8, each secondary storage computing device 265 has its own associated metabase or index 261. Each client 230 may also have its own associated metabase 270. However in some embodiments, each “tier” of storage, such as primary storage, secondary storage, tertiary storage, etc., may have multiple metabases or a centralized metabase, as described herein. For example, rather than a separate metabase or index associated with each client 230 in FIG. 8, the metabases on this storage tier may be centralized. Similarly, second and other tiers of storage may have either centralized or distributed metabases. Moreover, mixed architecture systems may be used if desired, that may include a first tier centralized metabase system coupled to a second tier storage system having distributed metabases and vice versa, etc.
  • [0120]
    Moreover, in operation, a storage manager 205 or other management module may keep track of certain information that allows the storage manager 205 to select, designate, or otherwise identify metabases to be searched in response to certain queries as further described herein. Movement of data between primary and secondary storage may also involve movement of associated metadata and other tracking information as further described herein.
  • [0121]
    In some examples, primary data may be organized into one or more sub-clients. A sub-client is a portion of the data of one or more clients 230, and can contain either all of the data of the clients 230 or a designated subset thereof. As depicted in FIG. 8, the data store 262 includes two sub-clients. For example, an administrator (or other user with the appropriate permissions; the term administrator is used herein for brevity) may find it preferable to separate email data from financial data using two different sub-clients having different storage preferences, retention criteria, etc.
  • Conclusion
  • [0122]
    Systems and modules described herein may comprise software, firmware, hardware, or any combination(s) of software, firmware, or hardware suitable for the purposes described herein. Software and other modules may reside on servers, workstations, personal computers, computerized tablets, PDAs, and other devices suitable for the purposes described herein. Modules described herein may be executed by a general-purpose computer, e.g., a server computer, wireless device, or personal computer. Those skilled in the relevant art will appreciate that aspects of the invention can be practiced with other communications, data processing, or computer system configurations, including: Internet appliances, hand-held devices (including personal digital assistants (PDAs)), wearable computers, all manner of cellular or mobile phones, multi-processor systems, microprocessor-based or programmable consumer electronics, set-top boxes, network PCs, mini-computers, mainframe computers, and the like. Indeed, the terms “computer,” “server,” “host,” “host system,” and the like, are generally used interchangeably herein and refer to any of the above devices and systems, as well as any data processor. Furthermore, aspects of the invention can be embodied in a special purpose computer or data processor that is specifically programmed, configured, or constructed to perform one or more of the computer-executable instructions explained in detail herein.
  • [0123]
    Software and other modules may be accessible via local memory, a network, a browser, or other application in an ASP context, or via another means suitable for the purposes described herein. Examples of the technology can also be practiced in distributed computing environments where tasks or modules are performed by remote processing devices, which are linked through a communications network, such as a Local Area Network (LAN), Wide Area Network (WAN), or the Internet. In a distributed computing environment, program modules may be located in both local and remote memory storage devices. Data structures described herein may comprise computer files, variables, programming arrays, programming structures, or any electronic information storage schemes or methods, or any combinations thereof, suitable for the purposes described herein. User interface elements described herein may comprise elements from graphical user interfaces, command line interfaces, and other interfaces suitable for the purposes described herein.
  • [0124]
    Examples of the technology may be stored or distributed on computer-readable media, including magnetically or optically readable computer disks, hard-wired or preprogrammed chips (e.g., EEPROM semiconductor chips), nanotechnology memory, biological memory, or other data storage media. Indeed, computer-implemented instructions, data structures, screen displays, and other data under aspects of the invention may be distributed over the Internet or over other networks (including wireless networks), on a propagated signal on a propagation medium (e.g., an electromagnetic wave(s), a sound wave, etc.) over a period of time, or they may be provided on any analog or digital network (packet switched, circuit switched, or other scheme).
  • [0125]
    Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense; that is to say, in the sense of “including, but not limited to.” As used herein, the terms “connected,” “coupled,” or any variant thereof, means any connection or coupling, either direct or indirect, between two or more elements; the coupling or connection between the elements can be physical, logical, or a combination thereof. Additionally, the words “herein,” “above,” “below,” and words of similar import, when used in this application, refer to this application as a whole and not to any particular portions of this application. Where the context permits, words in the above Detailed Description using the singular or plural number may also include the plural or singular number respectively. The word “or,” in reference to a list of two or more items, covers all of the following interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list.
  • [0126]
    The above Detailed Description is not intended to be exhaustive or to limit the invention to the precise form disclosed above. While specific examples for the invention are described above for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize. For example, while processes or blocks are presented in a given order, alternative implementations may perform routines having steps, or employ systems having blocks, in a different order, and some processes or blocks may be deleted, moved, added, subdivided, combined, and/or modified to provide alternative or subcombinations. Each of these processes or blocks may be implemented in a variety of different ways. Also, while processes or blocks are at times shown as being performed in series, these processes or blocks may instead be performed or implemented in parallel, or may be performed at different times. Further any specific numbers noted herein are only examples: alternative implementations may employ differing values or ranges.
  • [0127]
    The teachings of the invention provided herein can be applied to other systems, not necessarily the system described above. The elements and acts of the various examples described above can be combined to provide further implementations of the invention.
  • [0128]
    Any patents and applications and other references noted above, including any that may be listed in accompanying filing papers, are incorporated herein by reference. Aspects of the invention can be modified, if necessary, to employ the systems, functions, and concepts of the various references described above to provide yet further implementations of the invention.
  • [0129]
    These and other changes can be made to the invention in light of the above Detailed Description. While the above description describes certain examples of the invention and describes the best mode contemplated, no matter how detailed the above appears in text, the invention can be practiced in many ways. Details of the system may vary considerably in its specific implementation, while still being encompassed by the invention disclosed herein. As noted above, particular terminology used when describing certain features or aspects of the invention should not be taken to imply that the terminology is being redefined herein to be restricted to any specific characteristics, features, or aspects of the invention with which that terminology is associated. In general, the terms used in the following claims should not be construed to limit the invention to the specific examples disclosed in the specification, unless the above Detailed Description section explicitly defines such terms. Accordingly, the actual scope of the invention encompasses not only the disclosed examples, but also all equivalent ways of practicing or implementing the invention under the claims.
  • [0130]
    While certain aspects of the invention are presented below in certain claim forms, the applicant contemplates the various aspects of the invention in any number of claim forms. Accordingly, the applicant reserves the right to add additional claims after filing the application to pursue such additional claim forms for other aspects of the invention. For example, while only one aspect of the invention is recited as a means-plus-function claim under 35 U.S.C sec. 112, sixth paragraph, other aspects may likewise be embodied as a means-plus-function claim, or in other forms, such as being embodied in a computer-readable medium. (Any claims intended to be treated under 35 U.S.C. §112, ¶6 will begin with the words “means for”, but use of the term “for” in any other context is not intended to invoke treatment under 35 U.S.C. ¶112, ¶6.

Claims (26)

  1. 1. A method of handling a Network Data Management Protocol (“NDMP”) backup or copy operation, the method comprising:
    receiving an original NDMP backup data stream from a client;
    modifying the received original NDMP backup data stream by performing one or more of the following modification procedures upon the original NDMP backup data stream: deduplication, encryption, and compression,
    wherein the one or more modification procedures creates a modified version of the NDMP backup data stream;
    storing the modified version of the NDMP backup data stream in stored chunks,
    wherein each chunk is addressable by a physical offset and a physical length,
    wherein each chunk represents a modified version of a contiguous subset of the original NDMP backup data stream, and,
    wherein the client associates a logical offset and a logical length with the contiguous subset of the original NDMP backup data stream that is represented by each chunk; and,
    for each stored chunk, associating the stored chunk with:
    the physical offset and physical length that permits addressing of the chunk, and,
    the logical offset and logical length that the client associates with the contiguous subset of the original NDMP backup data stream that is represented by the chunk.
  2. 2. The method of claim 1 further comprising:
    receiving a request for a certain subset of the original NDMP backup data stream,
    wherein the request includes a logical offset and a logical length;
    using the received logical offset and logical length to:
    identify a stored chunk that comprises a modified version of at least a portion of the requested subset of the original NDMP backup data stream, and
    determine a physical offset and a physical length that permits addressing of the identified chunk;
    reading at least a portion of the identified chunk using the determined physical offset and physical length;
    converting the portion of the identified chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream in order to create a converted version of the portion of the identified chunk; and,
    sending data from the converted version of the portion of the identified chunk that is at least a portion of the requested subset of the original NDMP backup data stream.
  3. 3. The method of claim 2, further comprising:
    reading another portion of the identified chunk;
    converting the other portion of the identified chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream to create a converted version of the other portion of the identified chunk;
    buffering data from the converted version of the other portion of the identified chunk, wherein the buffered data does not form a portion of the requested subset of the original NDMP backup data stream; and,
    responding to another request for a different subset of the original NDMP backup data stream with at least some of the buffered data.
  4. 4. The method of claim 2, further comprising:
    reading at least a portion of another chunk that represents a modified version of a contiguous subset of the original NDMP backup data stream that is adjacent to at least a portion of the requested subset of the original NDMP backup data stream;
    converting the portion of the other chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream to create a converted version of the portion of the other chunk;
    buffering data from the converted version of the portion of the other chunk, wherein the buffered data does not form a portion of the requested subset of the original NDMP backup data stream; and,
    responding to another request for a different subset of the original NDMP backup data stream with at least some of the buffered data.
  5. 5. The method of claim 1, wherein modifying the received original NDMP backup data stream further comprises adding metadata to the data stream.
  6. 6. The method of claim 1, wherein a storage policy specifies a configuration of the backup or copy operation.
  7. 7. The method of claim 1, wherein associating a stored chunk with a physical offset, physical length, logical offset, and logical length comprises associating a chunk identifier with the physical offset, physical length, logical offset, and logical length in a chunk mapping table.
  8. 8. The method of claim 1,
    wherein modifying the received original NDMP backup data stream comprises performing deduplication; and
    wherein at least one chunk corresponds to a deduplication data structure that is created by the performance of deduplication.
  9. 9. The method of claim 1,
    wherein modifying the received original NDMP backup data stream comprises performing deduplication; and
    wherein at least one chunk corresponds to a logical subdivision of a deduplication data structure that is created by the performance of deduplication.
  10. 10. The method of claim 1, wherein a storage policy specifies an approximate size of the stored chunks.
  11. 11. The method of claim 1, wherein the size of stored chunks is configurable to tune performance of restore operations.
  12. 12. The method of claim 1, further comprising:
    determining that one or more stored chunks are being moved to a new physical location during a storage operation; and,
    for each of the one or more moved chunks, associating the chunk and its associated logical offset and logical length with a new physical offset and new physical length that permits addressing of the chunk at its new physical location.
  13. 13. A system for handling a Network Data Management Protocol (“NDMP”) backup or copy operation, the system comprising:
    a processor; and,
    a secondary storage computing device, coupled to the processor, configured to:
    receive an original NDMP backup data stream from a client;
    modify the received original NDMP backup data stream by performing one or more of the following modification procedures upon the original NDMP backup data stream: deduplication, encryption, and compression,
    wherein the one or more modification procedures create a modified version of the NDMP backup data stream;
    store the modified version of the NDMP backup data stream in stored chunks,
    wherein each chunk is addressable by a physical offset and a physical length,
    wherein each chunk represents a modified version of a subset of the original NDMP backup data stream, and,
    wherein the client associates a logical offset and a logical length with the subset of the original NDMP backup data stream that is represented by each chunk; and,
    for each stored chunk, associate the stored chunk with:
    the physical offset and physical length that permits addressing of the chunk, and,
    the logical offset and logical length that the client associates with the subset of the original NDMP backup data stream that is represented by the chunk.
  14. 14. The system of claim 13 wherein the secondary storage computing device is further configured to:
    receive a request for a certain subset of the original NDMP backup data stream, wherein the request includes a logical offset and a logical length;
    use the received logical offset and logical length to:
    identify a stored chunk that comprises a modified version of at least a portion of the requested subset of the original NDMP backup data stream, and
    determine a physical offset and a physical length that permits addressing of the identified chunk;
    read at least a portion of the identified chunk using the determined physical offset and physical length;
    convert the portion of the identified chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream in order to create a converted version of the portion of the identified chunk; and,
    send data from the converted version of the portion of the identified chunk that is at least a portion of the requested subset of the original NDMP backup data stream.
  15. 15. The system of claim 14, wherein the secondary storage computing device is further configured to:
    read another portion of the identified chunk;
    convert the other portion of the identified chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream to create a converted version of the other portion of the identified chunk;
    buffer data from the converted version of the other portion of the identified chunk, wherein the buffered data does not form a portion of the requested subset of the original NDMP backup data stream; and,
    respond to another request for a different subset of the original NDMP backup data stream with at least some of the buffered data.
  16. 16. The system of claim 14, wherein the secondary storage computing device is further configured to:
    read at least a portion of another chunk that represents a modified version of a subset of the original NDMP backup data stream that is adjacent to at least a portion of the requested subset of the original NDMP backup data stream;
    convert the portion of the other chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream to create a converted version of the portion of the other chunk;
    buffer data from the converted version of the portion of the other chunk, wherein the buffered data does not form a portion of the requested subset of the original NDMP backup data stream; and,
    respond to another request for a different subset of the original NDMP backup data stream with at least some of the buffered data.
  17. 17. The system of claim 13,
    wherein the secondary storage computing device modifies the received original NDMP backup data stream by performing deduplication; and
    wherein at least one chunk corresponds to a deduplication data structure that is created by the performance of deduplication.
  18. 18. The system of claim 13, wherein a data storage policy specifies an approximate size of the stored chunks.
  19. 19. The system of claim 13, wherein the secondary storage computing device is further configured to:
    determine that one or more stored chunks are being moved to a new physical location during a storage operation; and,
    for each of the one or more moved chunks, associate the chunk and its associated logical offset and logical length with a new physical offset and new physical length that permits addressing of the chunk at its new physical location.
  20. 20. A tangible computer-readable storage medium whose contents cause a data storage system to perform an operation for recovering Network Data Management Protocol (“NDMP”) backup data, the operation comprising:
    receiving a request for an unmodified version of NDMP data that was modified prior to being stored in secondary storage media, the request including a logical offset and a logical length; and,
    returning the requested unmodified version of NDMP data using the received logical offset and logical length.
  21. 21. The computer-readable storage medium of claim 20, wherein the NDMP data that was modified prior to being stored in secondary storage media was modified by one or more of: deduplication, encryption, and compression.
  22. 22. The computer-readable storage medium of claim 20,
    wherein returning the requested unmodified version of NDMP data using the received logical offset and logical length comprises accessing an index using the received logical offset and logical length to identify from the index a physical offset and physical length associated with a physical data storage device, and
    wherein the physical offset and physical length permit addressing and accessing a stored chunk of data that represents a modified form of a portion of the requested NDMP data.
  23. 23. A data storage system for reverting data to a previous state using point-in-time data, the system comprising:
    means for receiving a request from a user to revert data to an earlier state at a previous time using previously obtained point-in-time data;
    means for presenting the user with an option to take a snapshot of a current state of the data;
    means for determining whether the user has requested that a snapshot of the current state of the data be taken;
    means for taking a snapshot of the current state of the data, when the user has requested that a snapshot of the current state of the data be taken; and,
    means for reverting the data using the previously obtained point-in-time data.
  24. 24. The system of claim 23, wherein the means for presenting the user with the option to take a snapshot of the current state of the data includes means for presenting a displayable dialog box that provides the user with:
    an option to take a snapshot of the current state of the data, and,
    an option to always take a precautionary snapshot before reverting data.
  25. 25. A tangible computer-readable storage medium whose contents cause a data storage system to perform an operation for reverting a set of data, the operation comprising:
    performing a precautionary snapshot of a current state of a set of data, and,
    reverting the set of data to a previous state using point-in-time data, wherein the precautionary snapshot is performed prior to reverting the set of data.
  26. 26. The tangible computer-readable storage medium of claim 25, wherein the operation further comprises:
    receiving a user's request to revert the set of data to a previous state using the point-in-time data;
    in response to the request, automatically providing a user interface display that presents the user with the option to perform the precautionary snapshot, and,
    receiving the user's selection of the option to perform the precautionary snapshot via the user interface display.
US13241625 2010-09-30 2011-09-23 Data recovery operations, such as recovery from modified network data management protocol data Active 2034-11-26 US9244779B2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US38855410 true 2010-09-30 2010-09-30
US13241625 US9244779B2 (en) 2010-09-30 2011-09-23 Data recovery operations, such as recovery from modified network data management protocol data

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US13241625 US9244779B2 (en) 2010-09-30 2011-09-23 Data recovery operations, such as recovery from modified network data management protocol data
US15005209 US9557929B2 (en) 2010-09-30 2016-01-25 Data recovery operations, such as recovery from modified network data management protocol data
US15419272 US20170139780A1 (en) 2010-09-30 2017-01-30 Data recovery operations, such as recovery from modified network data management protocol data

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15005209 Division US9557929B2 (en) 2010-09-30 2016-01-25 Data recovery operations, such as recovery from modified network data management protocol data

Publications (2)

Publication Number Publication Date
US20120084523A1 true true US20120084523A1 (en) 2012-04-05
US9244779B2 US9244779B2 (en) 2016-01-26

Family

ID=45890827

Family Applications (3)

Application Number Title Priority Date Filing Date
US13241625 Active 2034-11-26 US9244779B2 (en) 2010-09-30 2011-09-23 Data recovery operations, such as recovery from modified network data management protocol data
US15005209 Active US9557929B2 (en) 2010-09-30 2016-01-25 Data recovery operations, such as recovery from modified network data management protocol data
US15419272 Pending US20170139780A1 (en) 2010-09-30 2017-01-30 Data recovery operations, such as recovery from modified network data management protocol data

Family Applications After (2)

Application Number Title Priority Date Filing Date
US15005209 Active US9557929B2 (en) 2010-09-30 2016-01-25 Data recovery operations, such as recovery from modified network data management protocol data
US15419272 Pending US20170139780A1 (en) 2010-09-30 2017-01-30 Data recovery operations, such as recovery from modified network data management protocol data

Country Status (1)

Country Link
US (3) US9244779B2 (en)

Cited By (48)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080243420A1 (en) * 2006-12-22 2008-10-02 Parag Gokhale Systems and methods of media management, such as management of media to and from a media storage library
US8463994B2 (en) 2005-12-19 2013-06-11 Commvault Systems, Inc. System and method for improved media identification in a storage device
US8463753B2 (en) 2003-04-03 2013-06-11 Commvault Systems, Inc. System and method for extended media retention
US20130173906A1 (en) * 2011-12-29 2013-07-04 Eric T. Obligacion Cloning storage devices through secure communications links
US20130218901A1 (en) * 2012-02-16 2013-08-22 Apple Inc. Correlation filter
US8539118B2 (en) 2006-09-22 2013-09-17 Commvault Systems, Inc. Systems and methods of media management, such as management of media to and from a media storage library, including removable media
US20130282662A1 (en) * 2012-04-23 2013-10-24 Commvault Systems, Inc. Integrated snapshot interface for a data storage system
US8706976B2 (en) 2007-08-30 2014-04-22 Commvault Systems, Inc. Parallel access virtual tape library and drives
US20140129524A1 (en) * 2012-10-05 2014-05-08 Hitachi, Ltd. Restoring method and computer system
US20140201154A1 (en) * 2013-01-11 2014-07-17 Commvault Systems, Inc. Sharing of secondary storage data
US8805849B1 (en) * 2012-06-20 2014-08-12 Symantec Corporation Enabling use of analytic functions for distributed storage system data
US8832031B2 (en) 2006-12-22 2014-09-09 Commvault Systems, Inc. Systems and methods of hierarchical storage management, such as global management of storage operations
US20140317448A1 (en) * 2013-04-23 2014-10-23 Facebook, Inc. Incremental checkpoints
US20140325616A1 (en) * 2013-04-30 2014-10-30 International Business Machines Corporation File system level data protection during potential security breach
US8930648B1 (en) * 2012-05-23 2015-01-06 Netapp, Inc. Distributed deduplication using global chunk data structure and epochs
US8996460B1 (en) * 2013-03-14 2015-03-31 Emc Corporation Accessing an image in a continuous data protection using deduplication-based storage
CN104503904A (en) * 2014-12-12 2015-04-08 北京国双科技有限公司 Method and device for processing data stream
US20150142749A1 (en) * 2013-11-19 2015-05-21 International Business Machines Corporation Method and system for a safe archiving of data
US9069799B2 (en) 2012-12-27 2015-06-30 Commvault Systems, Inc. Restoration of centralized data storage manager, such as data storage manager in a hierarchical data storage system
US9201917B2 (en) 2003-04-03 2015-12-01 Commvault Systems, Inc. Systems and methods for performing storage operations in a computer network
US9280423B1 (en) * 2013-06-27 2016-03-08 Emc Corporation Mounting block level backup images
US9367702B2 (en) 2013-03-12 2016-06-14 Commvault Systems, Inc. Automatic file encryption
US9397833B2 (en) * 2014-08-27 2016-07-19 International Business Machines Corporation Receipt, data reduction, and storage of encrypted data
US9397832B2 (en) * 2014-08-27 2016-07-19 International Business Machines Corporation Shared data encryption and confidentiality
US9405631B2 (en) 2003-11-13 2016-08-02 Commvault Systems, Inc. System and method for performing an image level snapshot and for restoring partial volume data
US9405928B2 (en) 2014-09-17 2016-08-02 Commvault Systems, Inc. Deriving encryption rules based on file content
US9411986B2 (en) 2004-11-15 2016-08-09 Commvault Systems, Inc. System and method for encrypting secondary copies of data
US9448731B2 (en) 2014-11-14 2016-09-20 Commvault Systems, Inc. Unified snapshot storage management
US20160292048A1 (en) * 2015-03-30 2016-10-06 Western Digital Technologies, Inc. Data deduplication using chunk files
US9471578B2 (en) 2012-03-07 2016-10-18 Commvault Systems, Inc. Data storage system utilizing proxy device for storage operations
US9483489B2 (en) 2013-01-14 2016-11-01 Commvault Systems, Inc. Partial sharing of secondary storage files in a data storage system
US9495251B2 (en) 2014-01-24 2016-11-15 Commvault Systems, Inc. Snapshot readiness checking and reporting
US9501486B1 (en) * 2012-07-06 2016-11-22 Veritas Technologies Llc Systems and methods for preventing unintended data loss during data restoration
US9507525B2 (en) 2004-11-05 2016-11-29 Commvault Systems, Inc. Methods and system of pooling storage devices
US9529871B2 (en) 2012-03-30 2016-12-27 Commvault Systems, Inc. Information management of mobile device data
US9563628B1 (en) * 2012-12-11 2017-02-07 EMC IP Holding Company LLC Method and system for deletion handling for incremental file migration
US9600487B1 (en) * 2014-06-30 2017-03-21 EMC IP Holding Company LLC Self healing and restartable multi-steam data backup
WO2017058736A1 (en) * 2015-09-29 2017-04-06 Veritas Technologies Llc Systems and methods for restoring data from opaque data backup streams
US9632874B2 (en) 2014-01-24 2017-04-25 Commvault Systems, Inc. Database application backup in single snapshot for multiple applications
US9639426B2 (en) 2014-01-24 2017-05-02 Commvault Systems, Inc. Single snapshot for multiple applications
US9648105B2 (en) 2014-11-14 2017-05-09 Commvault Systems, Inc. Unified snapshot storage management, using an enhanced storage manager and enhanced media agents
US9710397B2 (en) 2012-02-16 2017-07-18 Apple Inc. Data migration for composite non-volatile storage device
US9753812B2 (en) 2014-01-24 2017-09-05 Commvault Systems, Inc. Generating mapping information for single snapshot for multiple applications
US9774672B2 (en) 2014-09-03 2017-09-26 Commvault Systems, Inc. Consolidated processing of storage-array commands by a snapshot-control media agent
US9804930B2 (en) 2013-01-11 2017-10-31 Commvault Systems, Inc. Partial file restore in a data storage system
US9886346B2 (en) 2013-01-11 2018-02-06 Commvault Systems, Inc. Single snapshot for multiple agents
US9898371B2 (en) 2012-03-07 2018-02-20 Commvault Systems, Inc. Data storage system utilizing proxy device for storage operations
US9928144B2 (en) 2015-03-30 2018-03-27 Commvault Systems, Inc. Storage management of data using an open-archive architecture, including streamlined access to primary data originally stored on network-attached storage and archived to secondary storage

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9244779B2 (en) 2010-09-30 2016-01-26 Commvault Systems, Inc. Data recovery operations, such as recovery from modified network data management protocol data
US9858427B2 (en) * 2016-02-26 2018-01-02 International Business Machines Corporation End-to-end encryption and backup in data protection environments

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6014695A (en) * 1996-11-19 2000-01-11 Hitachi, Ltd. Computer network system for packet-transferring data between computers, network file server in computer network system and data transferring method thereof
US20080177806A1 (en) * 2007-01-22 2008-07-24 David Maxwell Cannon Method and system for transparent backup to a hierarchical storage system
US8200638B1 (en) * 2008-04-30 2012-06-12 Netapp, Inc. Individual file restore from block-level incremental backups by using client-server backup protocol
US8204862B1 (en) * 2009-10-02 2012-06-19 Symantec Corporation Systems and methods for restoring deduplicated data
US8234468B1 (en) * 2009-04-29 2012-07-31 Netapp, Inc. System and method for providing variable length deduplication on a fixed block file system
US8412848B2 (en) * 2009-05-29 2013-04-02 Exagrid Systems, Inc. Method and apparatus for content-aware and adaptive deduplication

Family Cites Families (384)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4686620A (en) 1984-07-26 1987-08-11 American Telephone And Telegraph Company, At&T Bell Laboratories Database backup method
GB8622010D0 (en) 1986-09-12 1986-10-22 Hewlett Packard Ltd File backup facility
US5193154A (en) 1987-07-10 1993-03-09 Hitachi, Ltd. Buffered peripheral system and method for backing up and retrieving data to and from backup memory device
US5005122A (en) 1987-09-08 1991-04-02 Digital Equipment Corporation Arrangement with cooperating management server node and network service node
JPH0743676B2 (en) 1988-03-11 1995-05-15 株式会社日立製作所 -Back up-data dump control method and apparatus
US5455926A (en) 1988-04-05 1995-10-03 Data/Ware Development, Inc. Virtual addressing of optical storage media as magnetic tape equivalents
US4995035A (en) 1988-10-31 1991-02-19 International Business Machines Corporation Centralized management in a computer network
DE3843218A1 (en) 1988-12-22 1990-06-28 Grau Gmbh & Co Holdingges A method and apparatus for operating an automatic datentraegerarchivs
US8040727B1 (en) 1989-04-13 2011-10-18 Sandisk Corporation Flash EEprom system with overhead data stored in user data sectors
DE69033438D1 (en) 1989-04-13 2000-03-02 Sandisk Corp Replacement of faulty memory cells of a EEprommatritze
US5093912A (en) 1989-06-26 1992-03-03 International Business Machines Corporation Dynamic resource pool expansion and contraction in multiprocessing environments
EP0405926B1 (en) 1989-06-30 1996-12-04 Digital Equipment Corporation Method and apparatus for managing a shadow set of storage media
US5133065A (en) 1989-07-27 1992-07-21 Personal Computer Peripherals Corporation Backup computer program for networks
US5321816A (en) 1989-10-10 1994-06-14 Unisys Corporation Local-remote apparatus with specialized image storage modules
US5504873A (en) 1989-11-01 1996-04-02 E-Systems, Inc. Mass data storage and retrieval system
US5276860A (en) 1989-12-19 1994-01-04 Epoch Systems, Inc. Digital data processor with improved backup storage
US5276867A (en) 1989-12-19 1994-01-04 Epoch Systems, Inc. Digital data storage system with improved data migration
US5241164A (en) 1990-01-05 1993-08-31 Symbol Technologies, Inc. Method of decoding bar code symbols from partial scans
DE69124817D1 (en) 1990-04-17 1997-04-10 Sharp Kk Method and apparatus for recording data
GB2246218B (en) 1990-07-18 1994-02-09 Stc Plc Distributed data processing systems
US5239647A (en) 1990-09-07 1993-08-24 International Business Machines Corporation Data storage hierarchy with shared storage level
US5544347A (en) 1990-09-24 1996-08-06 Emc Corporation Data storage system controlled remote data mirroring with respectively maintained data indices
US5212772A (en) 1991-02-11 1993-05-18 Gigatrend Incorporated System for storing data in backup tape device
US5287500A (en) 1991-06-03 1994-02-15 Digital Equipment Corporation System for allocating storage spaces based upon required and optional service attributes having assigned piorities
US5333315A (en) 1991-06-27 1994-07-26 Digital Equipment Corporation System of device independent file directories using a tag between the directories and file descriptors that migrate with the files
US5347653A (en) 1991-06-28 1994-09-13 Digital Equipment Corporation System for reconstructing prior versions of indexes using records indicating changes between successive versions of the indexes
US5410700A (en) 1991-09-04 1995-04-25 International Business Machines Corporation Computer system which supports asynchronous commitment of data
US5367698A (en) 1991-10-31 1994-11-22 Epoch Systems, Inc. Network file migration system
US5241670A (en) 1992-04-20 1993-08-31 International Business Machines Corporation Method and system for automated backup copy ordering in a time zero backup copy session
US5241668A (en) 1992-04-20 1993-08-31 International Business Machines Corporation Method and system for automated termination and resumption in a time zero backup copy process
US5263154A (en) 1992-04-20 1993-11-16 International Business Machines Corporation Method and system for incremental time zero backup copying of data
US5265159A (en) 1992-06-23 1993-11-23 Hughes Aircraft Company Secure file erasure
US5506986A (en) 1992-07-14 1996-04-09 Electronic Data Systems Corporation Media management system using historical data to access data sets from a plurality of data storage devices
CA2153769C (en) 1993-01-21 2001-08-07 Steven E. Kullick Apparatus and method for transferring and storing data from an arbitrarily large number of networked computer storage devices
DE69434311D1 (en) 1993-02-01 2005-04-28 Sun Microsystems Inc Archiving file system for data providers in a distributed network environment
JP2583016B2 (en) 1993-04-16 1997-02-19 インターナショナル・ビジネス・マシーンズ・コーポレイション Recording media library and a control method thereof
US5889935A (en) 1996-05-28 1999-03-30 Emc Corporation Disaster control features for remote data mirroring
JPH0721135A (en) 1993-07-02 1995-01-24 Fujitsu Ltd Data processing system with duplex monitor function
US5499364A (en) 1993-10-14 1996-03-12 Digital Equipment Corporation System and method for optimizing message flows between agents in distributed computations
US5544345A (en) 1993-11-08 1996-08-06 International Business Machines Corporation Coherence controls for store-multiple shared data coordinated by cache directory entries in a shared electronic storage
JPH09509768A (en) 1993-11-09 1997-09-30 シーゲート テクノロジー,インコーポレイテッド Backup and restore system data for the computer network
US5495607A (en) 1993-11-15 1996-02-27 Conner Peripherals, Inc. Network management system having virtual catalog overview of files distributively stored across network domain
US5537585A (en) 1994-02-25 1996-07-16 Avail Systems Corporation Data storage management for network interconnected processors
US5491810A (en) 1994-03-01 1996-02-13 International Business Machines Corporation Method and system for automated data storage system space allocation utilizing prioritized data set parameters
JP3086773B2 (en) 1994-03-16 2000-09-11 株式会社東芝 Optical disk device
US7106843B1 (en) 1994-04-19 2006-09-12 T-Netix, Inc. Computer-based method and apparatus for controlling, monitoring, recording and reporting telephone access
US5673381A (en) 1994-05-27 1997-09-30 Cheyenne Software International Sales Corp. System and parallel streaming and data stripping to back-up a network
US5638509A (en) 1994-06-10 1997-06-10 Exabyte Corporation Data storage and protection system
JPH0877073A (en) 1994-08-31 1996-03-22 Toshiba Corp Collective optical disk device
US5412668A (en) 1994-09-22 1995-05-02 International Business Machines Corporation Parity striping feature for optical disks
US5574906A (en) 1994-10-24 1996-11-12 International Business Machines Corporation System and method for reducing storage requirement in backup subsystems utilizing segmented compression and differencing
US5608865A (en) 1995-03-14 1997-03-04 Network Integrity, Inc. Stand-in Computer file server providing fast recovery from computer file server failures
US5978577A (en) 1995-03-17 1999-11-02 Csg Systems, Inc. Method and apparatus for transaction processing in a distributed database system
US5559957A (en) 1995-05-31 1996-09-24 Lucent Technologies Inc. File system for a data storage device having a power fail recovery mechanism for write/replace operations
US5699361A (en) 1995-07-18 1997-12-16 Industrial Technology Research Institute Multimedia channel formulation mechanism
US5813009A (en) 1995-07-28 1998-09-22 Univirtual Corp. Computer based records management system method
JPH0944381A (en) 1995-07-31 1997-02-14 Toshiba Corp Method and device for data storage
US6023705A (en) 1995-08-11 2000-02-08 Wachovia Corporation Multiple CD index and loading system and method
US5815662A (en) 1995-08-15 1998-09-29 Ong; Lance Predictive memory caching for media-on-demand systems
US5619644A (en) 1995-09-18 1997-04-08 International Business Machines Corporation Software directed microcode state save for distributed storage controller
US5819020A (en) 1995-10-16 1998-10-06 Network Specialists, Inc. Real time backup system
US5778395A (en) 1995-10-23 1998-07-07 Stac, Inc. System for backing up files from disk volumes on multiple nodes of a computer network
US5729743A (en) 1995-11-17 1998-03-17 Deltatech Research, Inc. Computer apparatus and method for merging system deltas
US5761677A (en) 1996-01-03 1998-06-02 Sun Microsystems, Inc. Computer system method and apparatus providing for various versions of a file without requiring data copy or log operations
US6131095A (en) 1996-12-11 2000-10-10 Hewlett-Packard Company Method of accessing a target entity over a communications network
US6457017B2 (en) 1996-05-17 2002-09-24 Softscape, Inc. Computing system for information management
JP3538766B2 (en) 1996-05-23 2004-06-14 インターナショナル・ビジネス・マシーンズ・コーポレーション Apparatus and method for generating a copy of a data file
US5812398A (en) 1996-06-10 1998-09-22 Sun Microsystems, Inc. Method and system for escrowed backup of hotelled world wide web sites
US5758359A (en) 1996-10-24 1998-05-26 Digital Equipment Corporation Method and apparatus for performing retroactive backups in a computer system
US5875478A (en) 1996-12-03 1999-02-23 Emc Corporation Computer backup using a file system, network, disk, tape and remote archiving repository media system
WO1998033113A1 (en) 1997-01-23 1998-07-30 Overland Data, Inc. Virtual media library
US5875481A (en) 1997-01-30 1999-02-23 International Business Machines Corporation Dynamic reconfiguration of data storage devices to balance recycle throughput
US6658526B2 (en) 1997-03-12 2003-12-02 Storage Technology Corporation Network attached virtual data storage subsystem
US5924102A (en) 1997-05-07 1999-07-13 International Business Machines Corporation System and method for managing critical files
US6094416A (en) 1997-05-09 2000-07-25 I/O Control Corporation Multi-tier architecture for control network
US5887134A (en) 1997-06-30 1999-03-23 Sun Microsystems System and method for preserving message order while employing both programmed I/O and DMA operations
US5958005A (en) 1997-07-17 1999-09-28 Bell Atlantic Network Services, Inc. Electronic mail security
US6137864A (en) 1997-07-25 2000-10-24 Lucent Technologies Inc. Specifiable delete times for voice messaging
US6195794B1 (en) 1997-08-12 2001-02-27 International Business Machines Corporation Method and apparatus for distributing templates in a component system
DE69802294T2 (en) 1997-08-29 2002-05-16 Hewlett Packard Co Systems for data backup and recovery
EP0899662A1 (en) 1997-08-29 1999-03-03 Hewlett-Packard Company Backup and restore system for a computer network
US6732293B1 (en) 1998-03-16 2004-05-04 Symantec Corporation Method, software and apparatus for recovering and recycling data in conjunction with an operating system
US5950205A (en) 1997-09-25 1999-09-07 Cisco Technology, Inc. Data transmission over the internet using a cache memory file system
US6275953B1 (en) 1997-09-26 2001-08-14 Emc Corporation Recovery from failure of a data processor in a network server
JPH11110441A (en) 1997-10-02 1999-04-23 Fujitsu Ltd Electronic transaction system
US6026398A (en) 1997-10-16 2000-02-15 Imarket, Incorporated System and methods for searching and matching databases
US6223205B1 (en) 1997-10-20 2001-04-24 Mor Harchol-Balter Method and apparatus for assigning tasks in a distributed server system
US6052735A (en) 1997-10-24 2000-04-18 Microsoft Corporation Electronic mail object synchronization between a desktop computer and mobile device
US6021415A (en) 1997-10-29 2000-02-01 International Business Machines Corporation Storage management system with file aggregation and space reclamation within aggregated files
US5983239A (en) 1997-10-29 1999-11-09 International Business Machines Corporation Storage management system with file aggregation supporting multiple aggregated file counterparts
US6418478B1 (en) 1997-10-30 2002-07-09 Commvault Systems, Inc. Pipelined high speed data transfer mechanism
US6131099A (en) 1997-11-03 2000-10-10 Moore U.S.A. Inc. Print and mail business recovery configuration method and system
JPH11143754A (en) 1997-11-05 1999-05-28 Hitachi Ltd Version information and constitution information display method and device therefor, and computer readable recording medium for recording version information and constitution information display program
US5860068A (en) 1997-12-04 1999-01-12 Petabyte Corporation Method and system for custom manufacture and delivery of a data product
US6304880B1 (en) 1997-12-12 2001-10-16 International Business Machines Corporation Automated reclamation scheduling override in a virtual tape server
US6131190A (en) 1997-12-18 2000-10-10 Sidwell; Leland P. System for modifying JCL parameters to optimize data storage allocations
US6374336B1 (en) 1997-12-24 2002-04-16 Avid Technology, Inc. Computer system and process for transferring multiple high bandwidth streams of data between multiple storage units and multiple applications in a scalable and reliable manner
US6076148A (en) 1997-12-26 2000-06-13 Emc Corporation Mass storage subsystem and backup arrangement for digital data processing system which permits information to be backed up while host computer(s) continue(s) operating in connection with information stored on mass storage subsystem
US6674924B2 (en) 1997-12-30 2004-01-06 Steven F. Wright Apparatus and method for dynamically routing documents using dynamic control documents and data streams
US6154787A (en) 1998-01-21 2000-11-28 Unisys Corporation Grouping shared resources into one or more pools and automatically re-assigning shared resources from where they are not currently needed to where they are needed
US6260069B1 (en) 1998-02-10 2001-07-10 International Business Machines Corporation Direct data retrieval in a distributed computing system
DE69816415T2 (en) 1998-03-02 2004-04-15 Hewlett-Packard Co. (N.D.Ges.D.Staates Delaware), Palo Alto Data Backup System
US6026414A (en) 1998-03-05 2000-02-15 International Business Machines Corporation System including a proxy client to backup files in a distributed computing environment
US7117227B2 (en) 1998-03-27 2006-10-03 Call Charles G Methods and apparatus for using the internet domain name system to disseminate product information
US6154738A (en) 1998-03-27 2000-11-28 Call; Charles Gainor Methods and apparatus for disseminating product information via the internet using universal product codes
US6088694A (en) 1998-03-31 2000-07-11 International Business Machines Corporation Continuous availability and efficient backup for externally referenced objects
US6161111A (en) 1998-03-31 2000-12-12 Emc Corporation System and method for performing file-handling operations in a digital data processing system using an operating system-independent file map
US5898593A (en) 1998-04-02 1999-04-27 International Business Machines Corporation Automated data storage library with a movable column for selective import/export of portable magazines
US6167402A (en) 1998-04-27 2000-12-26 Sun Microsystems, Inc. High performance message store
US7209949B2 (en) 1998-05-29 2007-04-24 Research In Motion Limited System and method for synchronizing information between a host system and a mobile data communication device
US6421711B1 (en) 1998-06-29 2002-07-16 Emc Corporation Virtual ports for data transferring of a data storage system
US6094605A (en) 1998-07-06 2000-07-25 Storage Technology Corporation Virtual automated cartridge system
US6452915B1 (en) 1998-07-10 2002-09-17 Malibu Networks, Inc. IP-flow classification in a wireless point to multi-point (PTMP) transmission system
US6862622B2 (en) 1998-07-10 2005-03-01 Van Drebbel Mariner Llc Transmission control protocol/internet protocol (TCP/IP) packet-centric wireless point to multi-point (PTMP) transmission system architecture
WO2000004483A3 (en) 1998-07-15 2000-06-29 Imation Corp Hierarchical data storage management
US6269431B1 (en) 1998-08-13 2001-07-31 Emc Corporation Virtual storage and block level direct access of secondary storage for recovery of backup data
US6353878B1 (en) 1998-08-13 2002-03-05 Emc Corporation Remote control of backup media in a secondary storage subsystem through access to a primary storage subsystem
GB9817922D0 (en) 1998-08-17 1998-10-14 Connected Place Ltd A method of producing a checkpoint which describes a base file and a method of generating a difference file defining differences between an updated file and a
US6269382B1 (en) 1998-08-31 2001-07-31 Microsoft Corporation Systems and methods for migration and recall of data from local and remote storage
US6266784B1 (en) 1998-09-15 2001-07-24 International Business Machines Corporation Direct storage of recovery plan file on remote server for disaster recovery and storage management thereof
US6356901B1 (en) 1998-12-16 2002-03-12 Microsoft Corporation Method and apparatus for import, transform and export of data
US6246882B1 (en) 1998-12-22 2001-06-12 Telefonaktiebolaget Lm Ericsson (Publ) Wide area item tracking system
US6487561B1 (en) 1998-12-31 2002-11-26 Emc Corporation Apparatus and methods for copying, backing up, and restoring data using a backup segment size larger than the storage block size
US6266678B1 (en) 1998-12-31 2001-07-24 Computer Associates Think, Inc. System and method for dynamically viewing contents of a data file
US7107395B1 (en) 1998-12-31 2006-09-12 Emc Corporation Apparatus and methods for operating a computer storage system
US6212512B1 (en) 1999-01-06 2001-04-03 Hewlett-Packard Company Integration of a database into file management software for protecting, tracking and retrieving data
US6496744B1 (en) 1999-01-11 2002-12-17 David Philip Cook Method and system for custom manufacture and delivery of a data product
US6721334B1 (en) 1999-02-18 2004-04-13 3Com Corporation Method and apparatus for packet aggregation in packet-based network
US6615349B1 (en) 1999-02-23 2003-09-02 Parsec Sight/Sound, Inc. System and method for manipulating a computer file and/or program
US6324581B1 (en) 1999-03-03 2001-11-27 Emc Corporation File server system using file system storage, data movers, and an exchange of meta data among data movers for file locking and direct access to shared file systems
US6389432B1 (en) 1999-04-05 2002-05-14 Auspex Systems, Inc. Intelligent virtual volume access
GB9909436D0 (en) 1999-04-23 1999-06-23 Pact Routing device
US6308245B1 (en) 1999-05-13 2001-10-23 International Business Machines Corporation Adaptive, time-based synchronization mechanism for an integrated posix file system
US6175830B1 (en) 1999-05-20 2001-01-16 Evresearch, Ltd. Information management, retrieval and display system and associated method
US6850946B1 (en) 1999-05-26 2005-02-01 Wireless Valley Communications, Inc. Method and system for a building database manipulator
US6519679B2 (en) 1999-06-11 2003-02-11 Dell Usa, L.P. Policy based storage configuration
EP1194854A1 (en) 1999-06-30 2002-04-10 Microsoft Corporation Methods and systems for reporting and resolving support incidents
US7035880B1 (en) 1999-07-14 2006-04-25 Commvault Systems, Inc. Modular backup and retrieval system used in conjunction with a storage area network
US7395282B1 (en) 1999-07-15 2008-07-01 Commvault Systems, Inc. Hierarchical backup and retrieval system
US6538669B1 (en) 1999-07-15 2003-03-25 Dell Products L.P. Graphical user interface for configuration of a storage system
US7389311B1 (en) 1999-07-15 2008-06-17 Commvault Systems, Inc. Modular backup and retrieval system
US6366900B1 (en) 1999-07-23 2002-04-02 Unisys Corporation Method for analyzing the conditional status of specialized files
US6490666B1 (en) 1999-08-20 2002-12-03 Microsoft Corporation Buffering data in a hierarchical data storage environment
US6550057B1 (en) 1999-08-31 2003-04-15 Accenture Llp Piecemeal retrieval in an information services patterns environment
US6771595B1 (en) 1999-08-31 2004-08-03 Intel Corporation Apparatus and method for dynamic resource allocation in a network environment
US6343324B1 (en) 1999-09-13 2002-01-29 International Business Machines Corporation Method and system for controlling access share storage devices in a network environment by configuring host-to-volume mapping data structures in the controller memory for granting and denying access to the devices
US7702831B2 (en) 2000-01-06 2010-04-20 Super Talent Electronics, Inc. Flash memory controller for electronic data flash card
US20020069324A1 (en) 1999-12-07 2002-06-06 Gerasimov Dennis V. Scalable storage architecture
US6338006B1 (en) 1999-12-11 2002-01-08 International Business Machines Corporation Data storage library with efficient cartridge eject
US7689510B2 (en) 2000-09-07 2010-03-30 Sonic Solutions Methods and system for use in network management of content
US6564228B1 (en) 2000-01-14 2003-05-13 Sun Microsystems, Inc. Method of enabling heterogeneous platforms to utilize a universal file system in a storage area network
US6542972B2 (en) 2000-01-31 2003-04-01 Commvault Systems, Inc. Logical view and access to physical storage in modular data and storage management system
US6658436B2 (en) 2000-01-31 2003-12-02 Commvault Systems, Inc. Logical view and access to data managed by a modular data and storage management system
US6438368B1 (en) 2000-03-30 2002-08-20 Ikadega, Inc. Information distribution system and method
US20020049778A1 (en) 2000-03-31 2002-04-25 Bell Peter W. System and method of information outsourcing
US7447907B2 (en) 2000-04-04 2008-11-04 Ecd Systems, Inc. Method and system for data delivery and reproduction
US20020032613A1 (en) 2000-04-18 2002-03-14 Buettgenbach Thomas H. Methods and systems for the physical delivery of goods ordered through an electronic network
US6356801B1 (en) 2000-05-19 2002-03-12 International Business Machines Corporation High availability work queuing in an automated data storage library
US20020010661A1 (en) 2000-05-31 2002-01-24 Waddington Steffanie G. Distribution system
US6950871B1 (en) 2000-06-29 2005-09-27 Hitachi, Ltd. Computer system having a storage area network and method of handling data in the computer system
US6330642B1 (en) 2000-06-29 2001-12-11 Bull Hn Informatin Systems Inc. Three interconnected raid disk controller data processing system architecture
US7281032B2 (en) 2000-06-30 2007-10-09 Hitachi, Ltd. File sharing system with data mirroring by storage systems
US7082441B1 (en) 2000-08-17 2006-07-25 Emc Corporation Method and storage and manipulation of storage system metrics
GB2366014B (en) 2000-08-19 2004-10-13 Ibm Free space collection in information storage systems
GB2383325B (en) 2000-08-21 2004-06-02 Singapore Technologies Logisti Order-handling inventory management system and method
WO2003107219A1 (en) 2000-09-11 2003-12-24 Zambeel, Inc. Storage system having partitioned migratable metadata
CA2321019A1 (en) 2000-09-27 2002-03-27 Ibm Canada Limited - Ibm Canada Limitee Capturing snapshots of a debuggee's state during a debug session
US6434682B1 (en) 2000-09-28 2002-08-13 International Business Machines Corporation Data management system with shortcut migration via efficient automatic reconnection to previously migrated copy
JP2002108662A (en) 2000-10-02 2002-04-12 Fujitsu Ltd Method for managing information
GB0025226D0 (en) 2000-10-14 2000-11-29 Ibm Data storage system and method of storing data
US6973553B1 (en) 2000-10-20 2005-12-06 International Business Machines Corporation Method and apparatus for using extended disk sector formatting to assist in backup and hierarchical storage management
EP1397765A2 (en) 2000-11-03 2004-03-17 Vistant Corporation Method and apparatus for associating the movement of goods with the identity of an entity moving these goods
US7034683B2 (en) 2000-11-06 2006-04-25 Loran Technologies, Inc. Electronic vehicle product and personnel monitoring
US6941370B2 (en) 2000-12-20 2005-09-06 International Business Machines Corporation Dynamic proxy reconfiguration system and method to support sharing of extra capacity
US7581011B2 (en) 2000-12-22 2009-08-25 Oracle International Corporation Template based workflow definition
US6862692B2 (en) 2001-01-29 2005-03-01 Adaptec, Inc. Dynamic redistribution of parity groups
US6662281B2 (en) 2001-01-31 2003-12-09 Hewlett-Packard Development Company, L.P. Redundant backup device
US7096269B2 (en) 2001-03-30 2006-08-22 Hitachi, Ltd. Path selection methods for storage based remote copy
US6616047B2 (en) 2001-03-31 2003-09-09 Koninklijke Philips Electronics N.V. Machine readable label reader system with robust context generation
US7277953B2 (en) 2001-04-18 2007-10-02 Emc Corporation Integrated procedure for partitioning network data services among multiple subscribers
US7904350B2 (en) 2001-07-20 2011-03-08 International Business Machines Corporation Network-based supply chain management method
US6948038B2 (en) 2001-07-24 2005-09-20 Microsoft Corporation System and method for backing up and restoring data
US6650961B2 (en) 2001-08-07 2003-11-18 Hewlett-Packard Development Company, L.P. Disk cartridge data storage apparatus
WO2003021449A1 (en) 2001-08-29 2003-03-13 Ramzi Nassar Armored data services
US7024517B1 (en) 2001-09-27 2006-04-04 Emc Corporation System and method for configuring data storage in accordance with workload requirements
US7058649B2 (en) 2001-09-28 2006-06-06 Intel Corporation Automated presentation layer content management system
JP2005505045A (en) 2001-09-28 2005-02-17 コムヴォールト・システムズ・インコーポレーテッド Quick create and manage methods and apparatus for recovery volume
EP1442387A4 (en) 2001-09-28 2008-01-23 Commvault Systems Inc System and method for archiving objects in an information store
US20030065759A1 (en) 2001-10-01 2003-04-03 Britt Julie Anne Event driven storage resource metering
JP2005510794A (en) 2001-11-23 2005-04-21 コムヴォールト・システムズ・インコーポレーテッド Selective data replication system and method
US20030101155A1 (en) 2001-11-23 2003-05-29 Parag Gokhale Method and system for scheduling media exports
US6922687B2 (en) 2001-12-21 2005-07-26 Barca, L.L.C. Closed loop asset management process
US7751628B1 (en) 2001-12-26 2010-07-06 Reisman Richard R Method and apparatus for progressively deleting media objects from storage
US7032816B2 (en) 2001-12-28 2006-04-25 Kimberly-Clark Worldwide, Inc. Communication between machines and feed-forward control in event-based product manufacturing
US6785078B2 (en) 2002-01-04 2004-08-31 International Business Machines Corporation Concurrent read and write access to simulated sequential data of a removable random access data storage medium
US7969306B2 (en) 2002-01-11 2011-06-28 Sap Aktiengesellschaft Context-aware and real-time item tracking system architecture and scenarios
US6968479B2 (en) 2002-03-06 2005-11-22 Hewlett-Packard Development Company, L.P. Verifying data in a data storage device
US7107285B2 (en) 2002-03-16 2006-09-12 Questerra Corporation Method, system, and program for an improved enterprise spatial system
CA2510111A1 (en) 2002-12-16 2004-07-15 Questerra Corporation Real-time insurance policy underwriting and risk management
US7467167B2 (en) 2002-03-19 2008-12-16 Network Appliance, Inc. System and method for coalescing a plurality of snapshots
US6934879B2 (en) 2002-03-28 2005-08-23 International Business Machines Corporation Method and apparatus for backing up and restoring data from nonvolatile memory
US6983351B2 (en) 2002-04-11 2006-01-03 International Business Machines Corporation System and method to guarantee overwrite of expired data in a virtual tape server
CA2485660C (en) 2002-05-16 2012-04-17 United Parcel Service Of America, Inc. Systems and methods for package sortation and delivery using radio frequency identification technology
US7085786B2 (en) 2002-05-20 2006-08-01 International Business Machines Corporation Apparatus and method to schedule and perform database maintenance
US20030229884A1 (en) 2002-05-21 2003-12-11 Hewlett-Packard Development Company Interaction manager template
US6972918B2 (en) 2002-06-06 2005-12-06 Renesas Technology Corp. Magnetic disk memory system
WO2004008348A1 (en) 2002-07-16 2004-01-22 Horn Bruce L Computer system for automatic organization, indexing and viewing of information from multiple sources
US8417678B2 (en) 2002-07-30 2013-04-09 Storediq, Inc. System, method and apparatus for enterprise policy management
US7069466B2 (en) 2002-08-14 2006-06-27 Alacritus, Inc. Method and system for copying backup data
US20040107199A1 (en) 2002-08-22 2004-06-03 Mdt Inc. Computer application backup method and system
US7103731B2 (en) 2002-08-29 2006-09-05 International Business Machines Corporation Method, system, and program for moving data among storage units
US6851031B2 (en) 2002-08-30 2005-02-01 Alacritus, Inc. Method of importing data from a physical data storage device into a virtual tape library
US20040083202A1 (en) 2002-08-30 2004-04-29 Arkivio, Inc. Techniques to control recalls in storage management applications
US20050210304A1 (en) 2003-06-26 2005-09-22 Copan Systems Method and apparatus for power-efficient high-capacity scalable storage system
US7130970B2 (en) 2002-09-09 2006-10-31 Commvault Systems, Inc. Dynamic storage device pooling in a computer system
WO2004025470A1 (en) 2002-09-10 2004-03-25 Exagrid Systems, Inc. Primary and remote data backup with nodal failover
US20040054607A1 (en) 2002-09-12 2004-03-18 Waddington Steffanie G. Distribution system
WO2004025423A3 (en) 2002-09-16 2006-06-22 Commvault Systems Inc System and method for blind media support
US7103619B1 (en) 2002-09-26 2006-09-05 Unisys Corporation System and method for automatic audit data archiving within a remote database backup system
US7162604B1 (en) 2002-10-08 2007-01-09 Ultera Systems, Inc. Multi-user virtual tape system
US8230066B2 (en) 2002-11-04 2012-07-24 International Business Machines Corporation Location independent backup of data from mobile and stationary computers in wide regions regarding network and server activities
JP2004157637A (en) 2002-11-05 2004-06-03 Hitachi Ltd Storage management method
US7412433B2 (en) 2002-11-19 2008-08-12 International Business Machines Corporation Hierarchical storage management using dynamic tables of contents and sets of tables of contents
US7584298B2 (en) 2002-12-13 2009-09-01 Internap Network Services Corporation Topology aware route control
US7143010B2 (en) 2003-12-17 2006-11-28 Cinecast, Llc System and method for remotely monitoring, diagnosing, intervening with and reporting problems with cinematic equipment
US7197490B1 (en) 2003-02-10 2007-03-27 Network Appliance, Inc. System and method for lazy-copy sub-volume load balancing in a network attached storage pool
US20040193953A1 (en) 2003-02-21 2004-09-30 Sun Microsystems, Inc. Method, system, and program for maintaining application program configuration settings
US6973369B2 (en) 2003-03-12 2005-12-06 Alacritus, Inc. System and method for virtual vaulting
US7702659B2 (en) 2003-03-27 2010-04-20 Sandisk Il Ltd. Robust, self-maintaining file system
JP2004318828A (en) 2003-03-31 2004-11-11 Seiko Epson Corp Data backup system, data backup method, wearable computer, mail transmission system, image information transmission system and data backup program
US20050039069A1 (en) 2003-04-03 2005-02-17 Anand Prahlad Remote disaster data recovery system and method
WO2004090872A3 (en) 2003-04-03 2005-01-13 Commvault Systems Inc Method and system for controlling a robotic arm in a storage device
US7174433B2 (en) 2003-04-03 2007-02-06 Commvault Systems, Inc. System and method for dynamically sharing media in a computer network
WO2004090789A3 (en) 2003-04-03 2005-02-03 Commvault Systems Inc System and method for extended media retention
US8244841B2 (en) 2003-04-09 2012-08-14 Microsoft Corporation Method and system for implementing group policy operations
US7120823B2 (en) 2003-04-17 2006-10-10 International Business Machines Corporation Method and apparatus for recovering logical partition configuration data
US7155465B2 (en) 2003-04-18 2006-12-26 Lee Howard F Method and apparatus for automatically archiving a file system
US20050021524A1 (en) 2003-05-14 2005-01-27 Oliver Jack K. System and method of managing backup media in a computing environment
GB2391361B (en) 2003-05-23 2005-09-21 Bridgeworks Ltd Library element management
JP4759513B2 (en) 2003-06-02 2011-08-31 リキッド・マシンズ・インコーポレーテッドLiquid Machines,Inc. Dynamic management of data objects in a distributed and collaborative environment
WO2004114086A3 (en) 2003-06-17 2005-11-17 United Security Appl Id Inc Electronic security system for monitoring and recording activity and data relating to cargo
US7454569B2 (en) 2003-06-25 2008-11-18 Commvault Systems, Inc. Hierarchical system and method for performing storage operations in a computer network
US7203944B1 (en) 2003-07-09 2007-04-10 Veritas Operating Corporation Migrating virtual machines among computer systems to balance load caused by virtual machines
US7277246B2 (en) 2003-07-18 2007-10-02 Quantum Corporation Methods and systems for providing predictive maintenance, preventative maintenance, and/or failure isolation in a tape storage subsystem
JP4421230B2 (en) 2003-08-12 2010-02-24 株式会社日立製作所 Performance information analysis method
JP2007502470A (en) 2003-08-14 2007-02-08 コンペレント・テクノロジーズ The system and method of the virtual disk drive
US7093089B2 (en) 2003-08-18 2006-08-15 Hewlett-Packard Development Company, Lp. Systems and methods for storing data on computer systems
US7191283B2 (en) 2003-08-21 2007-03-13 International Business Machines Corporation Grouping of storage media based on parameters associated with the storage media
US7707060B2 (en) 2003-08-21 2010-04-27 International Business Machines Corporation Automatic collection and dissemination of product usage information
US6950723B2 (en) 2003-08-22 2005-09-27 International Business Machines Corporation Method, system, and program for virtualization of data storage library addresses
US7010387B2 (en) 2003-08-28 2006-03-07 Spectra Logic Corporation Robotic data storage library comprising a virtual port
US7577807B2 (en) 2003-09-23 2009-08-18 Symantec Operating Corporation Methods and devices for restoring a portion of a data store
US7213118B2 (en) 2003-09-29 2007-05-01 International Business Machines Corporation Security in an automated data storage library
JP3712071B2 (en) 2003-10-02 2005-11-02 ソニー株式会社 File management apparatus, a file management method, a recording medium recording a program of a program and a file management method of the file management method
JP4267420B2 (en) 2003-10-20 2009-05-27 株式会社日立製作所 Storage devices and backup acquisition method
JP4066932B2 (en) 2003-11-10 2008-03-26 株式会社日立製作所 Computer resource allocation method based on the predicted
US7539707B2 (en) 2003-11-13 2009-05-26 Commvault Systems, Inc. System and method for performing an image level snapshot and for restoring partial volume data
US8959299B2 (en) 2004-11-15 2015-02-17 Commvault Systems, Inc. Using a snapshot as a data source
WO2005050382A3 (en) 2003-11-13 2006-06-22 Commvault Systems Inc System and method for data storage and tracking
WO2005065084A3 (en) 2003-11-13 2007-01-25 Commvault Systems Inc System and method for providing encryption in pipelined storage operations in a storage network
US7440982B2 (en) 2003-11-13 2008-10-21 Commvault Systems, Inc. System and method for stored data archive verification
GB0327522D0 (en) 2003-11-26 2003-12-31 Money Controls Ltd Packaging device and container for sheet objects
US9401838B2 (en) 2003-12-03 2016-07-26 Emc Corporation Network event capture and retention system
US7200621B2 (en) 2003-12-17 2007-04-03 International Business Machines Corporation System to automate schema creation for table restore
US7165059B1 (en) 2003-12-23 2007-01-16 Veritas Operating Corporation Partial file migration mechanism
US7103740B1 (en) 2003-12-31 2006-09-05 Veritas Operating Corporation Backup mechanism for a multi-class file system
US7293133B1 (en) 2003-12-31 2007-11-06 Veritas Operating Corporation Performing operations without requiring split mirrors in a multi-class file system
US8825591B1 (en) 2003-12-31 2014-09-02 Symantec Operating Corporation Dynamic storage mechanism
US7451283B2 (en) 2004-01-09 2008-11-11 International Business Machines Corporation Method, system, and program for copying tracks between a primary storage and secondary storage
US7418464B2 (en) 2004-01-27 2008-08-26 International Business Machines Corporation Method, system, and program for storing data for retrieval and transfer
US7720817B2 (en) 2004-02-04 2010-05-18 Netapp, Inc. Method and system for browsing objects on a protected volume in a continuous data protection system
US20050177828A1 (en) 2004-02-05 2005-08-11 Graham Christoph J. Restore media build automation
US7627617B2 (en) 2004-02-11 2009-12-01 Storage Technology Corporation Clustered hierarchical file services
US7216244B2 (en) 2004-02-25 2007-05-08 Hitachi, Ltd. Data storage system with redundant storage media and method therefor
US8620286B2 (en) 2004-02-27 2013-12-31 Synchronoss Technologies, Inc. Method and system for promoting and transferring licensed content and applications
US7966293B1 (en) 2004-03-09 2011-06-21 Netapp, Inc. System and method for indexing a backup using persistent consistency point images
US7539702B2 (en) 2004-03-12 2009-05-26 Netapp, Inc. Pre-summarization and analysis of results generated by an agent
JP4394493B2 (en) 2004-03-24 2010-01-06 株式会社日立製作所 File management method, file management apparatus, and a file management program
US8327050B2 (en) 2005-04-21 2012-12-04 International Business Machines Corporation Systems and methods for compressing files for storage and operation on compressed files
CN101346883A (en) 2005-10-26 2009-01-14 斯多维兹有限公司 Method and system for compression of data for block mode access storage
US7246258B2 (en) 2004-04-28 2007-07-17 Lenovo (Singapore) Pte. Ltd. Minimizing resynchronization time after backup system failures in an appliance-based business continuance architecture
US8112605B2 (en) 2005-05-02 2012-02-07 Commvault Systems, Inc. System and method for allocation of organizational resources
US7343356B2 (en) 2004-04-30 2008-03-11 Commvault Systems, Inc. Systems and methods for storage modeling and costing
US8266406B2 (en) 2004-04-30 2012-09-11 Commvault Systems, Inc. System and method for allocation of organizational resources
US7536424B2 (en) 2004-05-02 2009-05-19 Yoram Barzilai System and methods for efficiently managing incremental data backup revisions
US7395446B2 (en) 2004-05-03 2008-07-01 Microsoft Corporation Systems and methods for the implementation of a peer-to-peer rule-based pull autonomous synchronization system
US20060004639A1 (en) 2004-06-22 2006-01-05 O'keefe Edward L Jr Interstate sale control process
US9218588B2 (en) 2004-06-29 2015-12-22 United Parcel Service Of America, Inc. Offline processing systems and methods for a carrier management system
JP4576923B2 (en) 2004-08-09 2010-11-10 株式会社日立製作所 Storage capacity management method for a storage system
US20060075007A1 (en) 2004-09-17 2006-04-06 International Business Machines Corporation System and method for optimizing a storage system to support full utilization of storage space
US7428622B2 (en) 2004-09-28 2008-09-23 Akhil Tulyani Managing disk storage media based on access patterns
US7434090B2 (en) 2004-09-30 2008-10-07 Copan System, Inc. Method and apparatus for just in time RAID spare drive pool management
US8335789B2 (en) 2004-10-01 2012-12-18 Ricoh Co., Ltd. Method and system for document fingerprint matching in a mixed media environment
US20060106881A1 (en) 2004-10-25 2006-05-18 Empower Technologies System and method for global data synchronization
EP1810440A2 (en) 2004-10-26 2007-07-25 Kestrel Wireless, Inc. Method, system, and network for selectively controlling the utility a target
CA2587055A1 (en) 2004-11-05 2006-05-18 Commvault Systems, Inc. Method and system of pooling storage devices
US7536291B1 (en) 2004-11-08 2009-05-19 Commvault Systems, Inc. System and method to support simulated storage operations
US7822715B2 (en) 2004-11-16 2010-10-26 Petruzzo Stephen E Data mirroring method
US7809699B2 (en) 2004-11-17 2010-10-05 Iron Mountain Incorporated Systems and methods for automatically categorizing digital assets
US20060161879A1 (en) 2005-01-18 2006-07-20 Microsoft Corporation Methods for managing standards
US8005913B1 (en) 2005-01-20 2011-08-23 Network Protection Sciences, LLC Controlling, filtering, and monitoring of mobile device access to the internet, data, voice, and applications
US8347088B2 (en) 2005-02-01 2013-01-01 Newsilike Media Group, Inc Security systems and methods for use with structured and unstructured data
US20060169769A1 (en) 2005-02-02 2006-08-03 Leap-Up Llc Intelligent manager for automatically handling and managing media
US8161318B2 (en) 2005-02-07 2012-04-17 Mimosa Systems, Inc. Enterprise service availability through identity preservation
US8903949B2 (en) 2005-04-27 2014-12-02 International Business Machines Corporation Systems and methods of specifying service level criteria
US20060282194A1 (en) 2005-06-10 2006-12-14 Bdt Ag Tape library storage bridge
US7302540B1 (en) 2005-07-26 2007-11-27 Storage Technology Corporation Virtual tape storage system having snapshot virtual tape library for disaster recovery testing
US7660812B2 (en) 2005-09-02 2010-02-09 Imation Corp. Tracking physical inventory of data storage media
US8712959B1 (en) 2005-09-28 2014-04-29 Oracle America, Inc. Collaborative data redundancy for configuration tracking systems
US7805416B1 (en) 2005-09-30 2010-09-28 Emc Corporation File system query and method of use
US20070130105A1 (en) 2005-10-28 2007-06-07 Microsoft Corporation Obtaining server usage information
US7613752B2 (en) 2005-11-28 2009-11-03 Commvault Systems, Inc. Systems and methods for using metadata to enhance data management operations
US7603518B2 (en) 2005-12-19 2009-10-13 Commvault Systems, Inc. System and method for improved media identification in a storage device
US7617262B2 (en) 2005-12-19 2009-11-10 Commvault Systems, Inc. Systems and methods for monitoring application data in a data replication system
US7584227B2 (en) 2005-12-19 2009-09-01 Commvault Systems, Inc. System and method for containerized data storage and tracking
US8655850B2 (en) 2005-12-19 2014-02-18 Commvault Systems, Inc. Systems and methods for resynchronizing information
US8572330B2 (en) 2005-12-19 2013-10-29 Commvault Systems, Inc. Systems and methods for granular resource management in a storage network
US7398524B2 (en) 2005-12-22 2008-07-08 Alan Joshua Shapiro Apparatus and method for subtractive installation
US8621549B2 (en) 2005-12-29 2013-12-31 Nextlabs, Inc. Enforcing control policies in an information management system
US7565340B2 (en) 2006-01-09 2009-07-21 The State Of Oregon Acting By And Through The State Board Of Higher Education On Behalf Of Oregon State University Methods for assisting computer users performing multiple tasks
US7818417B2 (en) 2006-01-10 2010-10-19 International Business Machines Corporation Method for predicting performance of distributed stream processing systems
US7650389B2 (en) 2006-02-01 2010-01-19 Subhashis Mohanty Wireless system and method for managing logical documents
US20070185912A1 (en) 2006-02-08 2007-08-09 International Business Machines Corporation Off-loading I/O and computationally intensive operations to secondary systems
US7693832B2 (en) 2006-02-28 2010-04-06 Microsoft Corporation Rich set of synchronization rules across multiple accounts with multiple folder and consent types
US8051043B2 (en) 2006-05-05 2011-11-01 Hybir Inc. Group based complete and incremental computer file backup system, process and apparatus
US7659820B2 (en) 2006-06-23 2010-02-09 Sun Microsystems, Inc. Removable data storage media tracking system
US20080077622A1 (en) 2006-09-22 2008-03-27 Keith Robert O Method of and apparatus for managing data utilizing configurable policies and schedules
US7539783B2 (en) 2006-09-22 2009-05-26 Commvault Systems, Inc. Systems and methods of media management, such as management of media to and from a media storage library, including removable media
US7680843B1 (en) 2006-09-26 2010-03-16 Symantec Operating Corporation Method and system to offload archiving process to a secondary system
US7890796B2 (en) 2006-10-04 2011-02-15 Emc Corporation Automatic media error correction in a file server
US7882077B2 (en) 2006-10-17 2011-02-01 Commvault Systems, Inc. Method and system for offline indexing of content and classifying stored data
US8655914B2 (en) 2006-10-17 2014-02-18 Commvault Systems, Inc. System and method for storage operation access security
US20080147621A1 (en) 2006-10-18 2008-06-19 Aaron Charles Newman Method and system for backup and restoration of content within a blog
WO2008070688A1 (en) 2006-12-04 2008-06-12 Commvault Systems, Inc. Systems and methods for creating copies of data, such as archive copies
US7840537B2 (en) 2006-12-22 2010-11-23 Commvault Systems, Inc. System and method for storing redundant information
US20080228771A1 (en) 2006-12-22 2008-09-18 Commvault Systems, Inc. Method and system for searching stored data
US7734669B2 (en) 2006-12-22 2010-06-08 Commvault Systems, Inc. Managing copies of data
US7831566B2 (en) 2006-12-22 2010-11-09 Commvault Systems, Inc. Systems and methods of hierarchical storage management, such as global management of storage operations
US8346733B2 (en) 2006-12-22 2013-01-01 Commvault Systems, Inc. Systems and methods of media management, such as management of media to and from a media storage library
US8775823B2 (en) 2006-12-29 2014-07-08 Commvault Systems, Inc. System and method for encrypting secondary copies of data
US8850140B2 (en) 2007-01-07 2014-09-30 Apple Inc. Data backup for mobile device
CN101606361A (en) 2007-02-13 2009-12-16 日本电气株式会社 Mobile terminal management system, network device, and mobile terminal operation control method used for them
US20080307347A1 (en) 2007-06-08 2008-12-11 Apple Inc. Application-Based Backup-Restore of Electronic Information
US8032569B2 (en) 2007-06-28 2011-10-04 Seiko Epson Corporation Information management system, display system, management apparatus and program
US8706976B2 (en) 2007-08-30 2014-04-22 Commvault Systems, Inc. Parallel access virtual tape library and drives
US8006111B1 (en) 2007-09-21 2011-08-23 Emc Corporation Intelligent file system based power management for shared storage that migrates groups of files based on inactivity threshold
US8688641B1 (en) 2008-03-31 2014-04-01 Symantec Operating Corporation Per user and per process layer visibility
US9098495B2 (en) 2008-06-24 2015-08-04 Commvault Systems, Inc. Application-aware and remote single instance data management
US8219524B2 (en) 2008-06-24 2012-07-10 Commvault Systems, Inc. Application-aware and remote single instance data management
WO2010009367A1 (en) 2008-07-18 2010-01-21 Geospatial Holdings, Inc. Method, apparatus, and system for determining accurate location data related to underground installations
US9119017B2 (en) 2011-03-18 2015-08-25 Zscaler, Inc. Cloud based mobile device security and policy enforcement
US8307177B2 (en) 2008-09-05 2012-11-06 Commvault Systems, Inc. Systems and methods for management of virtualization data
US20100070474A1 (en) 2008-09-12 2010-03-18 Lad Kamleshkumar K Transferring or migrating portions of data objects, such as block-level data migration or chunk-based data migration
US20100070466A1 (en) 2008-09-15 2010-03-18 Anand Prahlad Data transfer techniques within data storage devices, such as network attached storage performing data migration
US9015181B2 (en) 2008-09-26 2015-04-21 Commvault Systems, Inc. Systems and methods for managing single instancing data
US8478876B2 (en) 2008-09-29 2013-07-02 Infosys Technologies Limited System and method for dynamic management and distribution of data in a data network
US8832044B1 (en) 2009-03-04 2014-09-09 Symantec Corporation Techniques for managing data compression in a data protection system
US20100269164A1 (en) 2009-04-15 2010-10-21 Microsoft Corporation Online service data management
US8695058B2 (en) 2009-05-20 2014-04-08 Mobile Iron, Inc. Selective management of mobile device data in an enterprise environment
US8578120B2 (en) 2009-05-22 2013-11-05 Commvault Systems, Inc. Block-level single instancing
US9594759B2 (en) 2009-06-16 2017-03-14 Microsoft Technology Licensing, Llc Backup and archival of selected items as a composite object
US20100332401A1 (en) 2009-06-30 2010-12-30 Anand Prahlad Performing data storage operations with a cloud storage environment, including automatically selecting among multiple cloud storage sites
US8966017B2 (en) 2009-07-09 2015-02-24 Novell, Inc. Techniques for cloud control and management
US8914342B2 (en) 2009-08-12 2014-12-16 Yahoo! Inc. Personal data platform
US8600998B1 (en) 2010-02-17 2013-12-03 Netapp, Inc. Method and system for managing metadata in a cluster based storage environment
US8527549B2 (en) 2010-02-22 2013-09-03 Sookasa Inc. Cloud based operating and virtual file system
US8572706B2 (en) 2010-04-26 2013-10-29 Vmware, Inc. Policy engine for cloud platform
US9386098B2 (en) 2010-06-11 2016-07-05 Fidelithon Systems, Llc Smartphone management system and method
US8635204B1 (en) 2010-07-30 2014-01-21 Accenture Global Services Limited Mining application repositories
US9244779B2 (en) 2010-09-30 2016-01-26 Commvault Systems, Inc. Data recovery operations, such as recovery from modified network data management protocol data
US8364652B2 (en) 2010-09-30 2013-01-29 Commvault Systems, Inc. Content aligned block-based deduplication
US9020900B2 (en) 2010-12-14 2015-04-28 Commvault Systems, Inc. Distributed deduplicated storage system
US8954446B2 (en) 2010-12-14 2015-02-10 Comm Vault Systems, Inc. Client-side repository in a networked deduplicated storage system
US8626128B2 (en) 2011-04-07 2014-01-07 Microsoft Corporation Enforcing device settings for mobile devices
US20130054533A1 (en) 2011-08-24 2013-02-28 Microsoft Corporation Verifying a data recovery component using a managed interface
US8931107B1 (en) 2011-08-30 2015-01-06 Amazon Technologies, Inc. Techniques for generating block level data captures
US20130318207A1 (en) 2012-02-15 2013-11-28 James Eric Dotter Systems and methods for managing mobile app data
US9027076B2 (en) 2012-03-23 2015-05-05 Lockheed Martin Corporation Method and apparatus for context aware mobile security
US9529871B2 (en) 2012-03-30 2016-12-27 Commvault Systems, Inc. Information management of mobile device data
US9026498B2 (en) 2012-08-13 2015-05-05 Commvault Systems, Inc. Lightweight mounting of a secondary copy of file system data
US8938481B2 (en) 2012-08-13 2015-01-20 Commvault Systems, Inc. Generic file level restore from a block-level secondary copy
US8700578B1 (en) 2012-09-27 2014-04-15 Emc Corporation System and method for determining physical storage space of a deduplicated storage system
US9858157B2 (en) 2012-10-31 2018-01-02 International Business Machines Corporation Intelligent restore-container service offering for backup validation testing and business resiliency
US9069799B2 (en) 2012-12-27 2015-06-30 Commvault Systems, Inc. Restoration of centralized data storage manager, such as data storage manager in a hierarchical data storage system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6014695A (en) * 1996-11-19 2000-01-11 Hitachi, Ltd. Computer network system for packet-transferring data between computers, network file server in computer network system and data transferring method thereof
US20080177806A1 (en) * 2007-01-22 2008-07-24 David Maxwell Cannon Method and system for transparent backup to a hierarchical storage system
US8200638B1 (en) * 2008-04-30 2012-06-12 Netapp, Inc. Individual file restore from block-level incremental backups by using client-server backup protocol
US8234468B1 (en) * 2009-04-29 2012-07-31 Netapp, Inc. System and method for providing variable length deduplication on a fixed block file system
US8412848B2 (en) * 2009-05-29 2013-04-02 Exagrid Systems, Inc. Method and apparatus for content-aware and adaptive deduplication
US8204862B1 (en) * 2009-10-02 2012-06-19 Symantec Corporation Systems and methods for restoring deduplicated data

Cited By (83)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8924428B2 (en) 2001-11-23 2014-12-30 Commvault Systems, Inc. Systems and methods of media management, such as management of media to and from a media storage library
US9251190B2 (en) 2003-04-03 2016-02-02 Commvault Systems, Inc. System and method for sharing media in a computer network
US9940043B2 (en) 2003-04-03 2018-04-10 Commvault Systems, Inc. Systems and methods for performing storage operations in a computer network
US8463753B2 (en) 2003-04-03 2013-06-11 Commvault Systems, Inc. System and method for extended media retention
US9201917B2 (en) 2003-04-03 2015-12-01 Commvault Systems, Inc. Systems and methods for performing storage operations in a computer network
US9405631B2 (en) 2003-11-13 2016-08-02 Commvault Systems, Inc. System and method for performing an image level snapshot and for restoring partial volume data
US9619341B2 (en) 2003-11-13 2017-04-11 Commvault Systems, Inc. System and method for performing an image level snapshot and for restoring partial volume data
US9507525B2 (en) 2004-11-05 2016-11-29 Commvault Systems, Inc. Methods and system of pooling storage devices
US9411986B2 (en) 2004-11-15 2016-08-09 Commvault Systems, Inc. System and method for encrypting secondary copies of data
US9633232B2 (en) 2004-11-15 2017-04-25 Commvault Systems, Inc. System and method for encrypting secondary copies of data
US8463994B2 (en) 2005-12-19 2013-06-11 Commvault Systems, Inc. System and method for improved media identification in a storage device
US8656068B2 (en) 2006-09-22 2014-02-18 Commvault Systems, Inc. Systems and methods of media management, such as management of media to and from a media storage library, including removable media
US8539118B2 (en) 2006-09-22 2013-09-17 Commvault Systems, Inc. Systems and methods of media management, such as management of media to and from a media storage library, including removable media
US8886853B2 (en) 2006-09-22 2014-11-11 Commvault Systems, Inc. Systems and methods for uniquely identifying removable media by its manufacturing defects wherein defects includes bad memory or redundant cells or both
US20080243420A1 (en) * 2006-12-22 2008-10-02 Parag Gokhale Systems and methods of media management, such as management of media to and from a media storage library
US8832031B2 (en) 2006-12-22 2014-09-09 Commvault Systems, Inc. Systems and methods of hierarchical storage management, such as global management of storage operations
US8756203B2 (en) 2006-12-22 2014-06-17 Commvault Systems, Inc. Systems and methods of media management, such as management of media to and from a media storage library
US8484165B2 (en) 2006-12-22 2013-07-09 Commvault Systems, Inc. Systems and methods of media management, such as management of media to and from a media storage library
US8706976B2 (en) 2007-08-30 2014-04-22 Commvault Systems, Inc. Parallel access virtual tape library and drives
US8996823B2 (en) 2007-08-30 2015-03-31 Commvault Systems, Inc. Parallel access virtual tape library and drives
US20130173906A1 (en) * 2011-12-29 2013-07-04 Eric T. Obligacion Cloning storage devices through secure communications links
US9710397B2 (en) 2012-02-16 2017-07-18 Apple Inc. Data migration for composite non-volatile storage device
US8914381B2 (en) * 2012-02-16 2014-12-16 Apple Inc. Correlation filter
US20130218901A1 (en) * 2012-02-16 2013-08-22 Apple Inc. Correlation filter
US9898371B2 (en) 2012-03-07 2018-02-20 Commvault Systems, Inc. Data storage system utilizing proxy device for storage operations
US9471578B2 (en) 2012-03-07 2016-10-18 Commvault Systems, Inc. Data storage system utilizing proxy device for storage operations
US9928146B2 (en) 2012-03-07 2018-03-27 Commvault Systems, Inc. Data storage system utilizing proxy device for storage operations
US9529871B2 (en) 2012-03-30 2016-12-27 Commvault Systems, Inc. Information management of mobile device data
US20160334995A1 (en) * 2012-04-23 2016-11-17 Commvault Systems, Inc Integrated snapshot interface for a data storage system
US20130282662A1 (en) * 2012-04-23 2013-10-24 Commvault Systems, Inc. Integrated snapshot interface for a data storage system
US9928002B2 (en) * 2012-04-23 2018-03-27 Commvault Systems, Inc. Integrated snapshot interface for a data storage system
US9342537B2 (en) * 2012-04-23 2016-05-17 Commvault Systems, Inc. Integrated snapshot interface for a data storage system
US8930648B1 (en) * 2012-05-23 2015-01-06 Netapp, Inc. Distributed deduplication using global chunk data structure and epochs
US8805849B1 (en) * 2012-06-20 2014-08-12 Symantec Corporation Enabling use of analytic functions for distributed storage system data
US9501486B1 (en) * 2012-07-06 2016-11-22 Veritas Technologies Llc Systems and methods for preventing unintended data loss during data restoration
US20140129524A1 (en) * 2012-10-05 2014-05-08 Hitachi, Ltd. Restoring method and computer system
US9015526B2 (en) * 2012-10-05 2015-04-21 Hitachi, Ltd. Restoring method and computer system
US9563628B1 (en) * 2012-12-11 2017-02-07 EMC IP Holding Company LLC Method and system for deletion handling for incremental file migration
US9069799B2 (en) 2012-12-27 2015-06-30 Commvault Systems, Inc. Restoration of centralized data storage manager, such as data storage manager in a hierarchical data storage system
US9760444B2 (en) * 2013-01-11 2017-09-12 Commvault Systems, Inc. Sharing of secondary storage data
US20140201154A1 (en) * 2013-01-11 2014-07-17 Commvault Systems, Inc. Sharing of secondary storage data
US9811423B2 (en) 2013-01-11 2017-11-07 Commvault Systems, Inc. Partial file restore in a data storage system
US9804930B2 (en) 2013-01-11 2017-10-31 Commvault Systems, Inc. Partial file restore in a data storage system
US9886346B2 (en) 2013-01-11 2018-02-06 Commvault Systems, Inc. Single snapshot for multiple agents
US9483489B2 (en) 2013-01-14 2016-11-01 Commvault Systems, Inc. Partial sharing of secondary storage files in a data storage system
US9483655B2 (en) 2013-03-12 2016-11-01 Commvault Systems, Inc. File backup with selective encryption
US9367702B2 (en) 2013-03-12 2016-06-14 Commvault Systems, Inc. Automatic file encryption
US9990512B2 (en) 2013-03-12 2018-06-05 Commvault Systems, Inc. File backup with selective encryption
US9734348B2 (en) 2013-03-12 2017-08-15 Commvault Systems, Inc. Automatic file encryption
US8996460B1 (en) * 2013-03-14 2015-03-31 Emc Corporation Accessing an image in a continuous data protection using deduplication-based storage
US9471436B2 (en) * 2013-04-23 2016-10-18 Facebook, Inc. Use of incremental checkpoints to restore user data stream processes
US20140317448A1 (en) * 2013-04-23 2014-10-23 Facebook, Inc. Incremental checkpoints
US9069955B2 (en) * 2013-04-30 2015-06-30 International Business Machines Corporation File system level data protection during potential security breach
US20140325616A1 (en) * 2013-04-30 2014-10-30 International Business Machines Corporation File system level data protection during potential security breach
US9306956B2 (en) 2013-04-30 2016-04-05 Globalfoundries Inc. File system level data protection during potential security breach
US9280423B1 (en) * 2013-06-27 2016-03-08 Emc Corporation Mounting block level backup images
US20150142749A1 (en) * 2013-11-19 2015-05-21 International Business Machines Corporation Method and system for a safe archiving of data
US9632874B2 (en) 2014-01-24 2017-04-25 Commvault Systems, Inc. Database application backup in single snapshot for multiple applications
US9892123B2 (en) 2014-01-24 2018-02-13 Commvault Systems, Inc. Snapshot readiness checking and reporting
US9495251B2 (en) 2014-01-24 2016-11-15 Commvault Systems, Inc. Snapshot readiness checking and reporting
US9639426B2 (en) 2014-01-24 2017-05-02 Commvault Systems, Inc. Single snapshot for multiple applications
US9753812B2 (en) 2014-01-24 2017-09-05 Commvault Systems, Inc. Generating mapping information for single snapshot for multiple applications
US9600487B1 (en) * 2014-06-30 2017-03-21 EMC IP Holding Company LLC Self healing and restartable multi-steam data backup
US9667422B1 (en) 2014-08-27 2017-05-30 International Business Machines Corporation Receipt, data reduction, and storage of encrypted data
US9397833B2 (en) * 2014-08-27 2016-07-19 International Business Machines Corporation Receipt, data reduction, and storage of encrypted data
US9397832B2 (en) * 2014-08-27 2016-07-19 International Business Machines Corporation Shared data encryption and confidentiality
US9979542B2 (en) 2014-08-27 2018-05-22 International Business Machines Corporation Shared data encryption and confidentiality
US9608816B2 (en) 2014-08-27 2017-03-28 International Business Machines Corporation Shared data encryption and confidentiality
US9774672B2 (en) 2014-09-03 2017-09-26 Commvault Systems, Inc. Consolidated processing of storage-array commands by a snapshot-control media agent
US9984006B2 (en) 2014-09-17 2018-05-29 Commvault Systems, Inc. Data storage systems and methods
US9405928B2 (en) 2014-09-17 2016-08-02 Commvault Systems, Inc. Deriving encryption rules based on file content
US9727491B2 (en) 2014-09-17 2017-08-08 Commvault Systems, Inc. Token-based encryption determination process
US9720849B2 (en) 2014-09-17 2017-08-01 Commvault Systems, Inc. Token-based encryption rule generation process
US9648105B2 (en) 2014-11-14 2017-05-09 Commvault Systems, Inc. Unified snapshot storage management, using an enhanced storage manager and enhanced media agents
US9921920B2 (en) 2014-11-14 2018-03-20 Commvault Systems, Inc. Unified snapshot storage management, using an enhanced storage manager and enhanced media agents
US9996428B2 (en) 2014-11-14 2018-06-12 Commvault Systems, Inc. Unified snapshot storage management
US9448731B2 (en) 2014-11-14 2016-09-20 Commvault Systems, Inc. Unified snapshot storage management
CN104503904A (en) * 2014-12-12 2015-04-08 北京国双科技有限公司 Method and device for processing data stream
US9928144B2 (en) 2015-03-30 2018-03-27 Commvault Systems, Inc. Storage management of data using an open-archive architecture, including streamlined access to primary data originally stored on network-attached storage and archived to secondary storage
US20160292048A1 (en) * 2015-03-30 2016-10-06 Western Digital Technologies, Inc. Data deduplication using chunk files
US9684569B2 (en) * 2015-03-30 2017-06-20 Western Digital Technologies, Inc. Data deduplication using chunk files
US9979785B2 (en) 2015-09-29 2018-05-22 Veritas Technologies Llc Systems and methods for restoring data from opaque data backup streams
WO2017058736A1 (en) * 2015-09-29 2017-04-06 Veritas Technologies Llc Systems and methods for restoring data from opaque data backup streams

Also Published As

Publication number Publication date Type
US9244779B2 (en) 2016-01-26 grant
US20170139780A1 (en) 2017-05-18 application
US9557929B2 (en) 2017-01-31 grant
US20160147472A1 (en) 2016-05-26 application

Similar Documents

Publication Publication Date Title
US7827150B1 (en) Application aware storage appliance archiving
US8307177B2 (en) Systems and methods for management of virtualization data
US20100070726A1 (en) Using a snapshot as a data source
US7668884B2 (en) Systems and methods for classifying and transferring information in a storage network
US20110167221A1 (en) System and method for efficiently creating off-site data volume back-ups
US20100005151A1 (en) Distributed indexing system for data storage
US7490207B2 (en) System and method for performing auxillary storage operations
US20130339298A1 (en) Collaborative backup in a networked storage system
US20070185917A1 (en) Systems and methods for classifying and transferring information in a storage network
US20130238562A1 (en) Data storage system utilizing proxy device for storage operations
US20140188805A1 (en) Backup and restoration for a deduplicated file system
US9436555B2 (en) Efficient live-mount of a backed up virtual machine in a storage management system
US20130262396A1 (en) Data storage recovery automation
US8352422B2 (en) Data restore systems and methods in a replication environment
US8938481B2 (en) Generic file level restore from a block-level secondary copy
US20160019317A1 (en) Volume or virtual machine level backup and generating placeholders for virtual machine files
US20160098323A1 (en) Intelligent protection of off-line mail data
US20160142483A1 (en) Unified snapshot storage management, using an enhanced storage manager and enhanced media agents
US20140201150A1 (en) Single snapshot for multiple agents
US20160062846A1 (en) Consolidated processing of storage-array commands using a forwarder media agent in conjunction with a snapshot-control media agent
US20110246416A1 (en) Stubbing systems and methods in a data replication environment
US20110246429A1 (en) Stub file prioritization in a data replication system
US20140201171A1 (en) High availability distributed deduplicated storage system
US20140201162A1 (en) Systems and methods to restore selected files from block-level backup for virtual machines
US20140108351A1 (en) Data storage system utilizing proxy device for storage operations

Legal Events

Date Code Title Description
AS Assignment

Owner name: COMMVAULT SYSTEMS, INC., NEW JERSEY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LITTLEFIELD, DUNCAN A.;NALLATHAMBI, VIMAL K.;CHANCHLANI,GIRISH;REEL/FRAME:026956/0360

Effective date: 20110920

AS Assignment

Owner name: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT, NO

Free format text: SECURITY INTEREST;ASSIGNOR:COMMVAULT SYSTEMS, INC.;REEL/FRAME:033266/0678

Effective date: 20140630