WO2022248047A1

WO2022248047A1 - Method of continuous data protection (cdp) in a data storage system using delta compression

Info

Publication number: WO2022248047A1
Application number: PCT/EP2021/064191
Authority: WO
Inventors: Daniel Goodman; Michael Hirsch; Itamar OFEK; Eddy DUER; Igor SHAFRAN; Shaul Tamari
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2021-05-27
Filing date: 2021-05-27
Publication date: 2022-12-01
Also published as: CN117355821A

Abstract

Provided is a method for data management in a data storage (202A-N). The method includes, at each of a series of snapshot time points, replicating a plurality of data blocks from the data storage in a backup storage. The method includes, at each of a series of intermediate time points, (i) determining one or more changed data blocks that have been updated in the data storage, and (ii) replicating the changed data blocks in the backup storage. The method includes, in response to a restore request for a selected intermediate time point, generating a recovery plan and executing the recovery plan by restoring the plurality of data blocks replicated at a preceding snapshot time point and restoring each identified data block from the backup storage in time order from the preceding snapshot time point to the selected intermediate time point.

Description

METHOD OF CONTINUOUS DATA PROTECTION (CDP) IN A DATA STORAGE SYSTEM USING DELTA COMPRESSION

TECHNICAL FIELD

The disclosure relates generally to a restoration of continuous data protection (CDP) to a recovery point, and more particularly, the disclosure relates to a computer-implemented method for data management in a data storage. Moreover, the disclosure also relates to a data protection module for the data storage and a data storage system including the data protection module for data management in the data storage.

BACKGROUND

Continuous data protection (CDP) is a form of data replication that copies data from a source system to a target system. In continuous data protection (CDP) replication, sets of data extents that change in the same time period are grouped into datasets and sent to the target system. A dataset (DS) may contain a fully compressed data extent (as before) or differential data extent. The dataset is significant because it is a stepping stone from one crash-recovery point to another. Typically, each dataset may include 5 seconds worth of data. Continuous data protection (CDP) also means that a recovery/restore to a crash consistent point, which is a boundary between the datasets.

Typically, restoring to a traditional CDP restore point involves restoring target data volume to a snapshot and then applying CDP datasets in order until a recovery point dataset is applied. Each CDP dataset includes all the data extent and metadata describing the writes to the source system over a dataset period. Traffic reduction refers to a process where a method of compression is performed by referencing data that already exists at the target system, calculating and sending a representation of a difference of data. For a non-traffic reduction CDP, existing solutions generate an optimised recovery plan by (i) determining which data extents changed on the source system between the snapshot and the recovery point, and (ii) determining in which CDP dataset that the changed data extent is found and then fetching the data extents from the CDP dataset objects and writing their values to the target data volume until all changed data extents are updated.

Known approaches reduce traffic using a differential compression/delta compression method that includes, (i) building a cache at the source system that includes the data extents from previous datasets that is not yet overwritten on the target system and calculating entropy values for these data extents or sub extents in the cache, (ii) for each data extent (i.e. source data extent) in the dataset to be transferred to the target system, (a) calculating entropy values for data in each data extent, (b) searching cache to detect matches where the data extents have 2 or more matching entropy values to the source data extent, (c) if found, choosing a matching record in the cache, (d) calculating a difference between these two data extents, and (e) instead of sending the source data extent contents, sending an address of the source data extent and the compressed difference of the data (i.e. which has significantly less data to transmit), (iii) at the target system, reading the contents of the original data extent from the target system at the specified source system address, applying the difference and writing the generated data extent contents to the target system address, and (iv) updating the source cache as each processing proceeds by adding new data extents and removing data extents at matching addresses, data extents from expired datasets or data extents whose entropy values are eclipsed. The traffic reduction in CDP introduces dependencies between arbitrary data extents, such that the algorithms to generate an optimised recovery plan are no longer viable.

The optimised recovery plan may fail if the data extent at the source address on the target system is not restored to the value that it had when a delta compression is performed and is unable to restore a desired data extent. Thus, the generation of the recovery plan may be adapted to fulfil the following requirements/limitations. For example, the recovery plan should have a dependency between the new data extents and the existing data extents on the target data volume. The traditional optimised recovery plan may not ensure that the source data extent is not overwritten until all depending data extents have been resolved. If a reference data extent is on the snapshot, this value must be read before a new value may be written to the data extent. If the reference data extent is in the dataset, this value must also be fetched in order to restore the data extent. The data extent dependencies are multi-dimensional compared to the traditional optimised recovery plan. A differential data extent may depend on a number of differential data extents in a restore chain across multiple datasets (e.g. DS7:100 -> DS5:220 -> DS2:90 -> Snapshot: 15). For example, in FIG. 1A, in order to restore a CDP dataset 9 which has 2 data extents with the differential compression. The restore plan is to include a dataset 8 with data extent 120, a dataset 7 with data extent 190, a dataset 5 with data extent 100, a dataset 4 with data extent 200, and from the snapshot, read the data extent 15 before applying the new data extent (i.e. the dataset 9) to that data extent. The complexity to restore the CDP dataset 9 is, (i) determining the dependency chains, (ii) determining a restore plan that detects that DS9:120 and DS7:190 both depend on DS5:100 and only fetch once, (ii) in execution, ensure that the data extents that are dependencies are not overwritten before their values are fetched (e.g. read DS8:120 before the final value of DS9:120 is written to data extent 120). The only way to restore the dataset may be to replay each dataset from the snapshot until a point in time to maintain a state. However, this is very inefficient, slow, and expensive to a customer in resources and there is no way to validate the restoration of differential data extents. The traditional recovery plan only restores the data extent to the recovery point, and not the data extent that is dependent on a previous data extent.

Therefore, there arises a need to address the aforementioned technical problem/drawbacks in restoring a dataset/data block in a data storage to a recovery point.

SUMMARY

It is an object of the disclosure to provide a computer-implemented method for data management in a data storage, a data protection module for the data storage, and a data storage system including the data protection module for data management in the data storage while avoiding one or more disadvantages of prior art approaches.

This object is achieved by the features of the independent claims. Further, implementation forms are apparent from the dependent claims, the description, and the figures.

The disclosure provides a computer-implemented method for data management in a data storage, a data protection module for the data storage, and a data storage system including the data protection module for data management in the data storage.

According to a first aspect, there is provided a computer-implemented method for data management in a data storage. The method includes, at each of a series of snapshot time points, replicating a plurality of data blocks from the data storage in a backup storage. The method includes, at each of a series of intermediate time points, (i) determining one or more changed data blocks that have been updated in the data storage, and (ii) replicating the changed data blocks in the backup storage. If one or more of the changed data blocks are similar to a reference block that has previously been replicated, the similar data blocks are represented by an address of the reference block, a time point of the reference block and a delta value. The method includes, in response to a restore request for a selected intermediate time point, generating a recovery plan by, (i) identifying, for each address in the data storage, a most recently changed data block having the same address in the backup storage in a period of time between the selected intermediate time point and a preceding snapshot time point, and (ii) for each identified data block, further identifying any reference data blocks and the corresponding time point in the backup storage. The method includes executing the recovery plan by restoring the plurality of data blocks replicated at the preceding snapshot time point and restoring each identified data block from the backup storage in time order from the preceding snapshot time point to the selected intermediate time point.

The method enables the generation of an extremely fast and an efficient recovery plan for restoring the changed data blocks in the data storage from the backup storage. The method enables the calculation of the recovery plan in a single attempt, and hence the method is extremely fast. The method restores the data blocks in the data storage using a delta compression, thereby reducing the traffic. The method builds a dependency list for each data block that includes the dependency information of the data blocks needed to restore the differential data blocks in the data storage from the backup storage. This can be achieved by tracking the differential data block dependencies when building the recovery plan and building a dependency list for the data block dependencies that is merged into a primary recovery plan when the matching data block is processed for inclusion in the recovery plan. The method is efficient as the dependency list is small, so the method can keep the dependency information of the data blocks in random access memory (RAM). The method does not require any additional resources besides the RAM during the recovery plan.

The reference block addresses and delta values may be compressed. Optionally, a new intermediate time point is made after a predetermined time and/or a predetermined volume of changed data. Optionally, the predetermined time is 5 seconds and the predetermined volume of changed data is 10 megabytes.

Each reference block address may be stored with a checksum value for the reference block. Optionally, restoring each reference block when executing the recovery plan includes generating a checksum value for the reference block and validating the generated checksum value against the stored checksum value. If the validation of the reference block may fail, the method further includes stopping or restarting the execution of the recovery plan. Optionally, executing the recovery plan includes sending the recovery plan to an external device for execution.

Optionally, generating the recovery plan includes processing each intermediate time point in reverse time order from the selected intermediate time point to the preceding snapshot time point, and for each intermediate time point, (i) determining, for each address in the data storage, if a data block having the same address exists in the recovery plan, and if not, (a) adding the data block having the same address from the present intermediate time point to the recovery plan, and (b) adding any corresponding address and time point for a reference block as an entry on a reference list, and (ii) determining if the reference list includes an entry which refers to the present intermediate time point, and if so, (a) adding the data block at the referenced address to the recovery plan, (b) adding any corresponding address and time point for a reference block as an entry on a reference list, and (c) removing the corresponding entry from the reference list.

According to a second aspect, there is provided a computer readable medium configured to store instructions which, when executed by a processor, cause the processor to execute the above method.

According to a third aspect, there is provided a data protection module for a data storage. The data protection module includes one or more processors configured to execute the above method.

The data protection module enables the generation of an extremely fast and an efficient recovery plan for restoring the changed data blocks in the data storage from the backup storage. The data protection module enables the calculation of the recovery plan in a single attempt, and hence the data protection module is extremely fast in restoring the data blocks in the data storage. The data protection module is efficient in restoring the data blocks in the data storage as the dependency list is small so the data protection module can keep the dependency information of the data blocks in random access memory (RAM). The data protection module does not require any additional resources besides the RAM during the recovery plan.

According to a fourth aspect, there is provided a data storage system. The data storage system includes one or more data storages and the data protection module as described above. A technical problem in the prior art is resolved, where the technical problem is that the restoration of the data block in the data storage to a recovery point with traffic reduction/wide area network (WAN) acceleration.

Therefore, in contradistinction to the prior art, the method enables the generation of an extremely fast and an efficient recovery plan for restoring the changed data blocks in the data storage from the backup storage. The method enables the calculation of the recovery plan in a single attempt, and hence the method is extremely fast. The method is efficient as the dependency list is small, so the method can keep the dependency information of the data blocks in random access memory (RAM). The method does not require any additional resources besides the RAM during the recovery plan.

These and other aspects of the disclosure will be apparent from and the implementation(s) described below.

BRIEF DESCRIPTION OF DRAWINGS

Implementations of the disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which:

FIG. 1 A illustrates a process of restoring a dataset which contain two extents with differential compression according to a prior art;

FIG. IB is a block diagram of a data protection module for a data storage in accordance with an implementation of the disclosure;

FIG. 2 is a block diagram of a data storage system in accordance with an implementation of the disclosure;

FIG. 3 illustrates an exemplary view of a data protection module for a data storage that executes a recovery plan in accordance with an implementation of the disclosure;

FIG. 4 illustrates an exemplary view of a data protection module for restoring the contents of a data block at multiple intermediate time points in accordance with an implementation of the disclosure; FIG. 5 illustrates an exemplary view of a data protection module that includes multiple dependency chains in accordance with an implementation of the disclosure;

FIGS. 6A-6B are flow diagrams that illustrate a method for data management in a data storage in accordance with an implementation of the disclosure; and

FIG. 7 is an illustration of an exemplary data protection module, a data storage system, or a computer system in which the various architectures and functionalities of the various previous implementations may be implemented.

DETAILED DESCRIPTION OF THE DRAWINGS

Implementations of the disclosure provide a computer-implemented method for data management in a data storage and the disclosure also relates to a data protection module for the data storage and a data storage system including the data protection module for data management in the data storage.

To make solutions of the disclosure more comprehensible for a person skilled in the art, the following implementations of the disclosure are described with reference to the accompanying drawings.

Terms such as "a first", "a second", "a third", and "a fourth" (if any) in the summary, claims, and foregoing accompanying drawings of the disclosure are used to distinguish between similar objects and are not necessarily used to describe a specific sequence or order. It should be understood that the terms so used are interchangeable under appropriate circumstances, so that the implementations of the disclosure described herein are, for example, capable of being implemented in sequences other than the sequences illustrated or described herein. Furthermore, the terms "include" and "have" and any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, a method, a system, a product, or a device that includes a series of steps or units, is not necessarily limited to expressly listed steps or units but may include other steps or units that are not expressly listed or that are inherent to such process, method, product, or device. Definitions:

Delta compression: Delta compression or data difference is a way of storing or transmitting data in the form of differences (i.e. deltas) between blocks of data rather than the complete blocks. The differences are recorded discretely, and may be called "deltas" or "diffs". By comparing data, a reference block of data can be found such that the differences between a block of data and the reference block are small. The delta compression greatly reduces data redundancy. Collections of unique deltas are substantially more space-efficient than their non- encoded equivalents.

Extent: An extent is a contiguous area of physical storage allocated in a file system. An extent may be referred to as a block of data, or may include a range of one or more data blocks.

Snapshot: The snapshot is a state of a storage system (e.g. a data storage system) captured at a given point in time. Preserving the storage system state not only allows data to be recovered in the event of failure but restored to known working points.

Data Storage: The data storage is essential because it backs up critical data to a central location. Users can then easily access this data. The data storage units are data storage devices/sy stems that allow storage and retrieval of data from a central location for authorized network users.

FIG. IB is a block diagram of a data protection module 100 for a data storage in accordance with an implementation of the disclosure. The data storage is configured to store one or more data blocks. The data blocks may be referred to as extents of the data storage. Each data block or extent may be associated with an address, e.g. an extent may have the form <start_address, length> where the length may be fixed at, for example, 4kB or 1MB.

The data protection module 100 includes one or more processors 102A-N. The one or more processors 102A-N are configured to execute a method for data management in a data storage. The one or more processors 102A-N are configured, at each of a series of snapshot time points, to replicate one or more data blocks from the data storage in a backup storage. The one or more processors 102A-N are configured, at each of a series of intermediate time points, to (i) determine one or more changed data blocks that have been updated in the data storage, and (ii) replicate the changed data blocks in the backup storage. If one or more of the changed data blocks are similar to a reference block that has previously been replicated, the similar data blocks are represented by an address of the reference block, a time point of the reference block, and a delta value. The series of intermediate time points may be referred to as a series of datasets.

The one or more processors 102A-N are configured, in response to a restore request for a selected intermediate time point, to generate a recovery plan by, (i) identifying, for each address in the data storage, a most recently changed data block having the same address in the backup storage in a period of time between the selected intermediate time point and a preceding snapshot time point, and (ii) for each identified data block, further identifying any reference data blocks and the corresponding time point in the backup storage. The one or more processors 102A-N are configured to execute the recovery plan by restoring the one or more data blocks replicated at the preceding snapshot time point and restoring each identified data block from the backup storage in time order from the preceding snapshot time point to the selected intermediate time point. Optionally, a new intermediate time point is made after a predetermined time or a predetermined volume of changed data. Optionally, the predetermined time is 5 seconds and the predetermined volume of changed data is 10 megabytes.

In some implementations, the restore request may specify to restore data to a target disc which is different from a source disc (i.e. the data storage). In some implementations, one or more processors associated with the target disc may be configured to generate and execute the recovery plan.

The data protection module 100 enables the generation of an extremely fast and an efficient recovery plan for restoring the changed data blocks in the data storage from the backup storage. The data protection module 100 enables the calculation of the recovery plan in a single attempt, and hence the data protection module 100 is extremely fast in restoring the data blocks in the data storage. The data protection module 100 is efficient in restoring the data blocks in the data storage as the dependency list is small, so the data protection module 100 can keep the dependency information of the data blocks in random access memory (RAM). The data protection module 100 does not require any additional resources besides the RAM to store the recovery plan and the current dataset being processed during the recovery plan.

The one or more data blocks may be referred to as a set of extents. Each of the one or more data blocks may be a fully compressed data block or the delta value (i.e. a differential data block). The data protection module 100, at each of the series of snapshot time points, builds a snapshot for the data storage, i.e. a full disk image of the data storage. The snapshot may contain each data block in the data storage at the snapshot time point. Each data block in the snapshot may be a fully compressed data block.

The differential data blocks may include the address of the reference block, the time point of the reference block, the delta value, or checksum. The reference block addresses and delta values may be compressed. The reference block may be a data block that has been previously replicated from the data storage in the backup storage. Optionally, to restore the differential data block in the data storage, the data protection module 100 tracks the differential data block dependencies when generating the recovery plan, and identifies, for each address in the data storage, the most recently changed data block having the same address in the backup storage in a period of time between the selected intermediate time point and the preceding snapshot time point. For each identified data block, the data protection module 100 further identifies any reference data blocks, and the corresponding time point in the backup storage, when the matching data block is processed for inclusion in the recovery plan.

Optionally, the data protection module 100 executes the recovery plan in the time order and flushes the backup storage (i.e. a target disk) before moving to the next data block in the recovery plan. The data protection module 100 may read each data block exactly once, thereby saving inputs/outputs (EOs) at a recovery time. Optionally, the data protection module 100 reads all the data blocks required from the back storage at the same time, thereby minimizing a file/an object opens/accesses and maximizing sequential read possibilities.

Optionally, the backup storage (i.e. the target disk) acts as a cache and this may enable each data block to be processed using the data protection module 100 that is used to process the differential data blocks during the regular replication.

The data protection module 100 executes the recovery plan efficiently. The data protection module 100 recovers a minimum number of data blocks from the backup storage that is needed to restore the data blocks or rebuild data blocks dependencies. The data protection module 100 determines the restore time for the recovery plan. The data protection module 100 ensures that the data block is not overwritten until all depending data / data blocks have been resolved. The data protection module 100 reads reference data blocks on the snapshot, before a new value is written to the data blocks and fetches the reference data blocks from the backup storage, in order to restore the data blocks. Each reference block address may be stored with a checksum value for the reference block. Optionally, the data protection module 100 is configured to restore each reference block when executing the recovery plan includes generating a checksum value for the reference block and validating the generated checksum value against the stored checksum value. Optionally, the data protection module 100 adds the checksum value for the reference block to ensure that the differential transform is being applied to the correct reference block contents and to detect errors early and fail out with the correct error. If the validation of the reference block may fail, the data protection module 100 further includes stop or restart the execution of the recovery plan. Optionally, executing the recovery plan includes sending the recovery plan to an external device for execution.

Optionally, the data protection module 100 restores the target disk to a state that is required in order to recover the current data block in the recovery plan rather than the final state at the recovery point.

The data protection module 100 may add metadata in the data blocks to facilitate a recovery plan that supports differential data block dependencies. The metadata may include the address of the reference data block, the time point of the reference block and the delta value, etc. The data protection module 100 may build the recovery plan that restores reference data blocks for the differential data blocks. The data protection module 100 executes the recovery plan, through each intermediate time point, so that the backup storage is in the correct state for each data block to be restored. The data protection module 100 may validate the contents of the reference block to validate the recovery plan execution and ensure that the data (i.e. the source data) for the differential transform is valid.

Optionally, the data protection module 100 generates the recovery plan. The generation of the recovery plan includes processing each intermediate time point in reverse time order from the selected intermediate time point to the preceding snapshot time point. For each intermediate time point, the data protection module 100 may determine, for each address in the data storage, if a data block having the same address exists in the recovery plan. If the data block having the same address does not exist in the recovery plan, the data protection module 100 may add the data block having the same address from the present intermediate time point to the recovery plan, and add any corresponding address and time point for a reference block as an entry on a reference list. The data protection module 100 may determine if the reference list includes an entry which refers to the present intermediate time point. If the reference list includes an entry which refers to the present intermediate time point, the data protection module 100 may add the data block at the referenced address to the recovery plan, add its reference block to the reference list, and remove the corresponding entry from the reference list.

Optionally, when sending the differential data blocks from the data storage (i.e. a source) to the backup storage during a continuous data protection, CDP, the metadata is extended to include the address of the data blocks (i.e. reference data block id) and the checksum of the reference block. For example, when adding a differential data block to the recovery plan, the data protection module 100 adds a reference data block time point, a reference data block checksum value, a reference data block address to a reference list. As each data block is processed, after determining which data block from the backup storage must be restored, if any additional data blocks from that data block exist in the reference list, the data protection module 100 may remove those data blocks from the reference list and if not already in the recovery plan, may add them to the recovery plan.

Optionally, during the execution of the recovery plan, instead of fetching multiple data blocks in parallel, the data protection module 100 executes the recovery plan, through each intermediate time point, starting from the preceding snapshot time point and ending with the selected time point for recovery.

The data protection module 100 may read any reference block from the backup storage or the target disk, calculate the checksum value for the reference block and compare it to the checksum value stored in the recovery plan. If there is no match in the checksum value of the reference block, the recovery plan fails to restore. If there is a match in the checksum value for the reference block, the data protection module 100 may apply the difference to the reference block and write the present data block (i.e. target data blocks).

FIG. 2 is a block diagram of a data storage system 200 in accordance with an implementation of the disclosure. The data storage system 200 includes one or more data storages 202A-N and a data protection module 204. Optionally, the one or more data storages 202A-N are communicatively connected to the data protection module 204. The data protection module 204 is configured to execute a method for data management in a data storage. The data protection module 204 is configured, at each of a series of snapshot time points, to replicate one or more data blocks from the data storage in a backup storage. The data protection module 204 is configured, at each of a series of intermediate time points, to (i) determine one or more changed data blocks that have been updated in the data storage, and (ii) replicate the changed data blocks in the backup storage. If one or more of the changed data blocks are similar to a reference block that has previously been replicated, the similar data blocks are represented by an address of the reference block, a time point of the reference block and a delta value. The data protection module 204 is configured, in response to a restore request for a selected intermediate time point, to generate a recovery plan by, (i) identifying, for each address in the data storage, a most recently changed data block having the same address in the backup storage in a period of time between the selected intermediate time point and a preceding snapshot time point, and (ii) for each identified data block, further identifying any reference data blocks and the corresponding time point in the backup storage. The data protection module 204 is configured to execute the recovery plan by restoring the one or more data blocks replicated at the preceding snapshot time point and restoring each identified data block from the backup storage in time order from the preceding snapshot time point to the selected intermediate time point. Optionally, a new intermediate time point is made after a predetermined time or a predetermined volume of changed data. Optionally, the predetermined time is 5 seconds and the predetermined volume of changed data is 10 megabytes.

In some embodiments, the backup storage may be a designated area of the data storage adjacent to a disk or area that is being backed-up. Alternatively, the backup storage may be located in a second data storage e.g. a cloud storage or backup server. In some implementations, the snapshots may be stored in a different location to the changed data blocks from each intermediate time point. For example, the snapshots may be stored in a designated area of the data storage and the changed data blocks may be stored in a second data storage.

Optionally, the data protection module 204 generates a recovery plan. The generation of the recovery plan includes processing each intermediate time point in reverse time order from a selected intermediate time point to a preceding snapshot time point. For each intermediate time point, the data protection module 204 may determine, for each address in the data storage, if a data block having the same address exists in the recovery plan. If the data block having the same address does not exist in the recovery plan, the data protection module 204 may add the data block having the same address from the present intermediate time point to the recovery plan, and add any corresponding address and time point for a reference block as an entry on a reference list. For each intermediate time point, the data protection module 204 may determine if the reference list includes an entry which refers to the present intermediate time point. If the reference list includes an entry which refers to the present intermediate time point, the data protection module 204 may add the data block at the referenced address, as well as a further reference block if it exists, to the recovery plan, and removes the corresponding entry from the reference list.

The data storage system 200 is a term referred to describe the data storage, or a group of data storages 202A-N, that a network uses store copies of one or more data items across high-speed connections. The data storages 202A-N are configured to back up critical data items/files and other data to a central location. The data storages 202A-N enable users to access these data items/files. The data storages 202A-N are data storage devices/sy stems that are connected to a network that allows storage and retrieval of data from a central location for authorised network users.

FIG. 3 illustrates an exemplary view of a data protection module for a data storage that executes a recovery plan in accordance with an implementation of the disclosure. The exemplary view depicts a restoration of continuous data protection, CDP, for the data storage to a recovery point (e.g. a selected intermediate time point). Optionally, the data protection module restores a snapshot time point (i.e. snap) to a CDP restore point, i.e. a n intermediate time point or dataset (DS) 69. The top row (i.e. snap) may represent the data blocks from the snapshot time point that are needed. Optionally, each row (e.g. 50-69) represents a dataset (DS)/intermediate time point and its relevant raw and traffic reduction data blocks and their dependencies. Optionally, each column (A-J) represents a data block for each intermediate time point (i.e. DS50-DS69) on a backup storage or a target disk. The exemplary view depicts a raw data block 302 and a dependent data block 304. The raw data block 302 may represent the data block that has no dependencies. The dependent data block 304 may represent the data block that is dependent on a reference block. The exemplary view depicts a recovery plan that includes the following data blocks.

1. DS50: D, G

2. DSsi: E

3. DS52: C

4. DS54: J

5. DS57: F

6. DSeo: H

7. DS₆₄: C

8. DS₆₅: E 9. DS₆₆: A, F

10. DSev: B, I

11. DSes: G

12. DS₆₉: A, C

The recovery plan is very efficient as fetching the data blocks from the intermediate time points (DS50 - DS69) includes the metadata to describe data block type, dependencies, etc. The recovery plan may restore a minimum number of data blocks required for restoring. The exemplary view also shows the execution of the recovery plan. The recovery plan is executed as follows:

1. For DS50: D, G, PRE: fetch B_SnaP and EXC: write B_SnaP»D, G

2. For DS₅₁: E, PRE: fetch G₅₀ and EXC: write G o»E

3. For DS₅₂: C, PRE: fetch E₅₁ and EXC: write Esi»C

4. For DS54: J, PRE: fetch I_snaP and EXC: write I_SnaP»J

5. For DS₅₇: F PRE: fetch C₅₂ andEXC: write Cs₂»F

6. For DS60: H, PRE: fetch F57 andEXC: write F 7»H

7. For DS₆₄: C PRE: - andEXC: write C

8. For D S65 : E PRE : fetch Fko and EXC : write H₆o»E

9. For DS₆₆: A, F, PRE: fetch C64 andEXC: write A, C64»F

10. For DS67: B, I PRE: fetch s₆, F₆₆ andEXC: write 56»B, F₆₆»I

11. For DSes: GPRE: - andEXC: write G

12. For DS₆₉: A, C, PRE: fetch C₆₄ andEXC: write C₆₄»A, C

With reference to FIG. 3, FIG. 4 illustrates an exemplary view of a data protection module for restoring the contents of a data block at multiple intermediate time points in accordance with an implementation of the disclosure. This example restores the contents of the data block at multiple intermediate time points/datasets in time to facilitate restoring of other data blocks. Optionally, a data block is updated once the current contents are used to restore the data block. In a non-traffic reduction case, the data block DS₆₉:C is restored in order to fetch its final value. However, with traffic reduction, the contents of the data block DS₅₂ C are restored in order to facilitate the restoration of the data block DS₅₇ C and the contents of the data block DSr_>4:C are restored in order to facilitate the restoration of the data blocks DS₆₉:A and DS₆₆:F. The recovery plan may not restore the contents of the data block DS69:C until the contents of the data block DS₆₉:A have been calculated. With reference to FIG. 3, FIG. 5 illustrates an exemplary view of a data protection module that includes multiple dependency chains in accordance with an implementation of the disclosure. The exemplary view depicts a long dependency chain of data blocks as follows: DSso:G - DS5i:E - DS52:C -> DS57:F - DS₆o:H - DS₆s:E. Optionally, a fan out chain ofthe data blocks as shown in FIG. 5 is as follows: DS64:C - DS₆s>:A and DS64:C - DS₆₆:F - DS67:F The recovery plan that has cache values must determine when to remove unneeded reference data blocks.

FIGS. 6A-6B are flow diagrams that illustrate a method for data management in a data storage in accordance with an implementation of the disclosure. At a step 602, at each of a series of snapshot time points, one or more data blocks is replicated from the data storage in a backup storage. At a step 604, at each of a series of intermediate time points, one or more changed data blocks that have been updated in the data storage are determined, and the changed data blocks are replicated in the backup storage. If one or more of the changed data blocks are similar to a reference block that has previously been replicated, the similar data blocks are represented by an address of the reference block, a time point of the reference block, and a delta value. At a step 606, in response to a restore request for a selected intermediate time point, a recovery plan is generated by, (i) identifying, for each address in the data storage, a most recently changed data block having the same address in the backup storage in a period of time between the selected intermediate time point and a preceding snapshot time point, and (ii) for each identified data block, further identifying any reference data blocks and the corresponding time point in the backup storage. At a step 608, the recovery plan is executed by restoring the one or more data blocks replicated at the preceding snapshot time point and restoring each identified data block from the backup storage in time order from the preceding snapshot time point to the selected intermediate time point.

The method restores the data block in the data storage without multiple data block dependencies. The method enables the generation of an extremely fast and an efficient recovery plan for restoring the changed data blocks in the data storage from the backup storage. The method enables the calculation of the recovery plan in a single attempt, and hence the method is extremely fast. The method restores the one or more data blocks in the data storage using a delta compression, thereby reducing the traffic. The method builds an index for each intermediate time point that includes the dependency information of the one or more data blocks needed to restore the differential data blocks in the data storage from the backup storage. This can be achieved by tracking the differential data block dependencies when building the recovery plan and building a shadow recovery plan for the data block dependencies that is merged into a primary recovery plan when the matching data block is processed for inclusion in the recovery plan. The method is efficient as the dependency list is small, so the method can keep the dependency information of the one or more data blocks in random access memory (RAM). The method does not require any additional resources besides the RAM during the recovery plan. The method validates the contents and integrity of the one or more data blocks before applying a differential transform to ensure that target data from the backup storage is valid.

The reference block addresses and delta values may be compressed. Optionally, a new intermediate time point is made after a predetermined time or a predetermined volume of changed data. Optionally, the predetermined time is 5 seconds and the predetermined volume of changed data is 10 megabytes.

Optionally, generating the recovery plan includes processing each intermediate time point in reverse time order from the selected intermediate time point to the preceding snapshot time point. For each intermediate time point, it is determined, for each address in the data storage, if a data block having the same address exists in the recovery plan. If the data block having the same address does not exists in the recovery plan, the data block having the same address from the present intermediate time point is added to the recovery plan, and any corresponding address and time point for a reference block is added as an entry on a reference list. For each intermediate time point, it is determined if the reference list includes an entry which refers to the present intermediate time point. If the reference list includes an entry which refers to the present intermediate time point, the data block is added at the referenced address to the recovery plan, and the corresponding entry is removed from the reference list.

In an implementation, a computer-readable medium configured to store instructions which, when executed by a processor, causes the processor to execute the above method. FIG. 7 is an illustration of an exemplary data protection module, a data storage system, or a computer system in which the various architectures and functionalities of the various previous implementations may be implemented. As shown, the computer system 700 includes at least one processor 704 that is connected to a bus 702, wherein the computer system 700 may be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect), PCI-Express, AGP (Accelerated Graphics Port), Hyper Transport, or any other bus or point-to- point communication protocol (s). The computer system 700 also includes a memory 706.

Control logic (software) and data are stored in the memory 706 which may take a form of random-access memory (RAM). In the disclosure, a single semiconductor platform may refer to a sole unitary semiconductor-based integrated circuit or chip. It should be noted that the term single semiconductor platform may also refer to multi-chip modules with increased connectivity which simulate on-chip modules with increased connectivity which simulate on- chip operation, and make substantial improvements over utilizing a conventional central processing unit (CPU) and bus implementation. Of course, the various modules may also be situated separately or in various combinations of semiconductor platforms per the desires of the user.

The computer system 700 may also include a secondary storage 710. The secondary storage 710 includes, for example, a hard disk drive and a removable storage drive, representing a floppy disk drive, a magnetic tape drive, a compact disk drive, digital versatile disk (DVD) drive, recording device, universal serial bus (USB) flash memory. The removable storage drive at least one of reads from and writes to a removable storage unit in a well-known manner.

Computer programs, or computer control logic algorithms, may be stored in at least one of the memory 706 and the secondary storage 710. Such computer programs, when executed, enable the computer system 700 to perform various functions as described in the foregoing. The memory 706, the secondary storage 710, and any other storage are possible examples of computer-readable media.

In an implementation, the architectures and functionalities depicted in the various previous figures may be implemented in the context of the processor 704, a graphics processor coupled to a communication interface 712, an integrated circuit (not shown) that is capable of at least a portion of the capabilities of both the processor 704 and a graphics processor, a chipset (namely, a group of integrated circuits designed to work and sold as a unit for performing related functions, and so forth).

Furthermore, the architectures and functionalities depicted in the various previous-described figures may be implemented in a context of a general computer system, a circuit board system, a game console system dedicated for entertainment purposes, an application-specific system. For example, the computer system 700 may take the form of a desktop computer, a laptop computer, a server, a workstation, a game console, an embedded system.

Furthermore, the computer system 700 may take the form of various other devices including, but not limited to a personal digital assistant (PDA) device, a mobile phone device, a smart phone, a television, and so forth. Additionally, although not shown, the computer system 700 may be coupled to a network (for example, a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the Internet, a peer-to-peer network, a cable network, or the like) for communication purposes through an I/O interface 708.

It should be understood that the arrangement of components illustrated in the figures described are exemplary and that other arrangement may be possible. It should also be understood that the various system components (and means) defined by the claims, described below, and illustrated in the various block diagrams represent components in some systems configured according to the subject matter disclosed herein. For example, one or more of these system components (and means) may be realized, in whole or in part, by at least some of the components illustrated in the arrangements illustrated in the described figures.

In addition, while at least one of these components are implemented at least partially as an electronic hardware component, and therefore constitutes a machine, the other components may be implemented in software that when included in an execution environment constitutes a machine, hardware, or a combination of software and hardware.

Although the disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions, and alterations can be made herein without departing from the spirit and scope of the disclosure as defined by the appended claims.

Claims

1. A computer-implemented method for data management in a data storage (202A), the method comprising: at each of a series of snapshot time points, replicating a plurality of data blocks from the data storage (202A) in a backup storage; at each of a series of intermediate time points: determining one or more changed data blocks that have been updated in the data storage (202A); and replicating the changed data blocks in the backup storage, wherein if one or more of the changed data blocks are similar to a reference block that has previously been replicated, the similar data blocks are represented by an address of the reference block, a time point of the reference block and a delta value; and in response to a restore request for a selected intermediate time point, generating a recovery plan by: identifying, for each address in the data storage, a most recently changed data block having the same address in the backup storage in a period of time between the selected intermediate time point and a preceding snapshot time point; and for each identified data block, further identifying any reference data blocks and the corresponding time point in the backup storage; and executing the recovery plan by restoring the plurality of data blocks replicated at the preceding snapshot time point and restoring each identified data block from the backup storage in time order from the preceding snapshot time point to the selected intermediate time point.

2. The method of claim 1, wherein the reference block addresses and delta values are compressed.

3. The method of claim 1 or claim 2, wherein a new intermediate time point is made after a predetermined time and/or a predetermined volume of changed data.

4. The method of claim 3, wherein the predetermined time is 5 seconds and the predetermined volume of changed data is 10 megabytes.

5. The method of any preceding claim, wherein each reference block address is stored with a checksum value for the reference block.

6. The method of claim 5, wherein restoring each reference block when executing the recovery plan comprises generating a checksum value for the reference block and validating the generated checksum value against the stored checksum value.

7. The method of claim 6, wherein if the validation of the reference block fails, the method further comprises stopping or restarting the execution of the recovery plan.

8. The method of any preceding claim, wherein executing the recovery plan comprises sending the recovery plan to an external device for execution.

9. The method of any preceding claim, wherein generating the recovery plan comprises processing each intermediate time point in reverse time order from the selected intermediate time point to the preceding snapshot time point, and for each intermediate time point: determining, for each address in the data storage (202A), if a data block having the same address exists in the recovery plan, and if not: adding the data block having the same address from the present intermediate time point to the recovery plan, and adding any corresponding address and time point for a reference block as an entry on a reference list; and determining if the reference list includes an entry which refers to the present intermediate time point, and if so: adding the data block at the referenced address to the recovery plan, adding any corresponding address and time point for a reference block as an entry on a reference list, and removing the corresponding entry from the reference list.

10. A computer readable medium configured to store instructions which, when executed by a processor (102A), cause the processor (102A) to execute the method of any preceding claim.

11. A data protection module (100, 204) for a data storage (202A), the data protection module (100, 204) comprising one or more processors (102A-N) configured to execute the method of any one of claims 1 to 9.

12. A data storage system (200) comprising: one or more data storages (202A-N); and the data protection module (100, 204) of claim 11.