CN103942011B - A kind of residual quantity fast photographic system and its application method - Google Patents

Abstract

The invention discloses a differential snapshot system and its usage method. The system comprises: a snapshot module, used for maintaining the dependency relationship between source volumes and snapshot volumes, snapshot policy semantics, and managing snapshot metadata; a thin configuration module, the thin configuration The module has a resource management sub-module, which is used to realize the resource management function of the snapshot, which includes resource allocation, resource mapping, and resource recovery; the snapshot module communicates with the thin provisioning module through a common block layer interface, and the snapshot module communicates with the thin provisioning module through the The thin provisioning module communicates by sending data requests or resource management commands. The system decouples snapshot semantics from physical resource management, can allocate storage resources on demand, and can implement three processing logics: copy-on-write, redirect-on-write, and write-anywhere, improving the effective utilization of snapshot resources.

Description

A differential snapshot system and its application method

technical field

The invention relates to the field of information technology, in particular to a differential snapshot technology in the storage field.

Background technique

The Storage Networking Industry Association (SNIA) divides snapshots into two categories according to implementation technologies: full-copy snapshots and differential-copy snapshots. Compared with the full copy snapshot, the differential snapshot technology has outstanding advantages such as low time overhead and high space utilization, so the differential snapshot technology is widely used in the industry. Currently, there are three implementations of differential snapshot technology, namely copy on write (COW, Copy On Write), redirect on write (ROW, Redirect OfWrite), and write anywhere (WA, Write Anywhere), to meet the needs of different instances.

The principle of copy-on-write is shown in Figure 1. After the snapshot is created, before the source volume data is overwritten for the first time, the old data of the source volume is copied to the space reserved for the snapshot, and then the new data is written into the source volume. And build an index table to record the data blocks modified after the snapshot is created. Copy-on-write technology maintains the physical distribution of data on the source volume, so it does not affect the performance of reading data from the source volume. However, the first write after the snapshot volume is created introduces a physical copy of the data, so the write performance of the source volume is poor, and the incremental data sets at each time are scattered in each snapshot volume, and the performance of reading the snapshot volume data will be affected.

Redirection on write, the principle is shown in Figure 2. After the snapshot is created, the write operation to the source volume will be redirected to the space reserved for the snapshot, and an index table will be created to record the data location. The old data in the source volume The data remains in its original location. Redirection-on-write has better write performance on the source volume, and the incremental data sets at each moment are aggregated and stored in a snapshot volume, and the performance of reading snapshot volume data is also better than copy-on-write snapshots. However, since the new data is stored in the snapshot volume, the distribution of the source data set becomes fragmented, so the performance of reading the source data set will be affected. In addition, the source volume data set is stored in both the source volume and the snapshot volume. When deleting a snapshot, the data migration in the snapshot needs to be merged into the source.

Write anywhere, the principle is shown in Figure 3. The basic idea is to virtualize all physical resources, that is, divide the disk into physical blocks of equal size according to a fixed granularity, and establish a mapping table to record the mapping relationship between all logical blocks and physical blocks. Data blocks are managed by mapping pointers. When creating a snapshot, copy the current mapping pointer as a snapshot. When a data block changes, the Write Anywhere technology does not modify the old data, but applies for a new free physical block for the data block, then points the corresponding data block mapping pointer of the source to the new physical block, and writes the new data into the new physical block Block storage space. The mapping pointer of the snapshot still points to the physical block where the old data is stored. The write anywhere technology has better write performance, and random write can also be changed into sequential write through request scheduling to optimize random write performance. When deleting a snapshot, the Write Anywhere technology only needs to reclaim data blocks, and does not need to perform data merge and migration. The Write Anywhere technology mixes the storage space of the source volume and the snapshot volume for unified management, and the read performance of the source volume and the snapshot volume is the same. However, the write-anywhere technology will cause a large amount of data fragmentation, and read requests will cause a large number of merge operations, seriously affecting the performance of reading source data and snapshot data.

Through the analysis of the three implementations of differential snapshots, it is found that they have something in common, that is, maintaining the semantics of the snapshot layer requires allocating storage space for new or old data, that is, snapshots also need to support resource management functions. Combining the aforementioned three implementation methods, the snapshot technology supports two resource management methods, namely the space reservation method and the dynamic resource allocation method. The space reservation method, when creating a snapshot, reserves available resources for the snapshot volume according to the ratio, which belongs to static resource management, and the resource utilization rate of this method is low. This method is applied to copy-on-write and redirect-on-write. The dynamic resource allocation method uniformly manages all available resources in the system. According to the principle of on-demand allocation, a write-time allocation strategy is adopted. This method has a high resource utilization rate, but it needs to dynamically update the resource usage, and the implementation is more complicated. way to apply to write everywhere.

Based on the above considerations, the resource management method and snapshot semantics in the traditional differential snapshot technology are tightly coupled, which brings the following deficiencies:

First, resource mapping information needs to be maintained in the snapshot system. Both resource management and allocation methods require the snapshot system to support the resource mapping function, and the corresponding mapping relationship needs to be modified or established when new data is generated. Resource map management and maintenance add to the complexity of the snapshot system.

Second, snapshot implementation technology is coupled with resource management methods. According to the definition of the Storage Network Industry Association, copy-on-write or redirect-on-write are resource reservation methods, which do not support on-demand allocation. As a result, it is difficult to improve the effective utilization of resources in the two implementation manners.

Contents of the invention

In order to solve the above problems, the object of the present invention is to provide a thin-provisioning-based differential snapshot system and its usage method, which decouples snapshot semantics from physical resource management, allocates storage resources on demand, and enables write-time The processing logic of copy, redirection on write, and write anywhere improves the effective utilization of snapshot resources.

To achieve the above purpose, the present invention proposes a differential snapshot system for allocating storage resources on demand, characterized in that the system includes:

Snapshot module, used to maintain dependencies between source volumes and snapshot volumes, snapshot policy semantics, and manage snapshot metadata;

A thin provisioning module, the thin provisioning module has a resource management sub-module, which is used to realize the resource management function of the snapshot, and the function includes resource allocation, resource mapping, and resource recovery;

The underlying device of the logical volume of the snapshot module is the logical volume of the thin provisioning module, and the resource management of the logical volume by the thin provisioning module realizes on-demand allocation of storage resources, and the snapshot module communicates with the thin provisioning module through a common block layer interface , the snapshot module implements communication by sending a data request or a resource management command to the thin provisioning module.

The differential snapshot system of the present invention is characterized in that, in the snapshot module, the snapshot technology adopted is copy-on-write or redirection-on-write, and in the thin configuration module, allocation-on-write technology is adopted to support writing snapshots anywhere semantics.

The differential snapshot system of the present invention is characterized in that the thin provisioning module also includes:

The resource pool is used for unified virtualized management of physical storage resources and provides available snapshot logic blocks to the outside world;

A physical storage device for storing snapshot data blocks;

Wherein, the snapshot logical block corresponds to the resource mapping table in the logical volume managed by the snapshot data block through the thin provisioning module;

The free resource table records the number and information of available physical resource blocks in the current resource pool.

The present invention also relates to a method for using the above-mentioned differential snapshot system, which is characterized in that the method includes a snapshot module step and a thin provisioning module step, wherein,

The snapshot module steps, including:

The user request processing step is used to perform copy-on-write processing or write-time redirection processing according to the snapshot policy applied to the logical volume accessed by the write request when the snapshot module receives the user's snapshot data block write request, and Send a remapping command or write request to the thin provisioning layer. When the snapshot module receives a read request from the user, it determines the logical volume where the requested data block is located according to the read request address and the snapshot logic, and forwards the read request and write request to the thin provisioning layer. Or return to the user after the read request returns,

The data merging step is used for merging snapshot data blocks corresponding to the same snapshot logical address in different snapshot volumes, and sending a remapping command or a resource recovery command to the thin provisioning layer;

The thin provisioning module steps, including:

The write request step is used to select whether to allocate resources for the snapshot data block and establish a mapping relationship between the snapshot data block and the physical resource block when the thin provisioning module receives the write request of the snapshot data block sent by the snapshot module , and then forward the request to the underlying physical storage module to write the data into the physical storage device, and return upward layer by layer after the writing is completed.

The read request step is used for redirecting the request to the position of the data block in the physical storage device when the thin provisioning module receives the read request of the snapshot data block sent by the snapshot module, and forwarding to the physical storage device, from the physical storage device The read data is returned to the user,

The remapping step is used to perform a remapping operation on the snapshot data block when the thin provisioning module receives a remapping command of the snapshot data block sent by the snapshot module,

The resource reclamation step is used for reclaiming physical resources of the snapshot data block when the thin provisioning module receives a resource reclamation command of the snapshot data block.

The method for using the differential snapshot system of the present invention is characterized in that the copy-on-write processing includes the following steps:

Step 11, the source volume receives the user's write request for the snapshot logical block, queries the metadata in the current snapshot volume dependent on the source volume, and judges whether this write is the first write to the snapshot logical block, if the snapshot volume If there is no valid snapshot data block identified in the metadata of the snapshot logical block, then it is the first write, go to step 12, otherwise, go to step 13,

Step 12, the snapshot source volume sends a remapping command to the underlying device, that is, the logical volume of the thin provisioning system, remaps the old version snapshot data block corresponding to the snapshot logical block from the source volume logical space to the snapshot volume logical space, and updates the metadata of the snapshot volume, and notify the data logical unit of the source volume that the logical copy of the snapshot data block corresponding to the snapshot logical block has been completed,

Step 13, the source volume forwards the snapshot data write request of the user to the underlying device, that is, the thin-provisioned logical volume, and returns the snapshot write request to the user layer after returning;

The redirect-on-write process includes the following steps:

Step 21, the source volume receives the user's write request for the snapshot logical block, queries the current snapshot volume dependent on the source volume, and redirects the write request to the snapshot volume logical space,

Step 22, the snapshot volume receives the user's snapshot logical block write request and forwards it to the underlying device,

Step 23, after the write request is returned, the metadata of the snapshot volume is updated and returned to the user layer.

The method for using the differential snapshot system of the present invention is characterized in that the data merging step is specifically as follows: if the snapshot data block corresponding to the snapshot logical block B that has a valid snapshot data block in the snapshot logical volume LV2 is merged into the snapshot logical volume LV1, where LV1 and LV2 are snapshot volumes of the same source volume at different times, or source volumes and snapshot volumes,

If the data set in the snapshot logical volume LV1 is newer than the data set in the snapshot logical volume LV2, proceed as follows:

Step 41, for the snapshot logical block B in the snapshot logical volume LV2, check the metadata in the snapshot logical volume LV1 to determine whether there is a new version of the snapshot data block of the snapshot logical block in the volume, and if so, execute step 42 , otherwise go to step 43,

Step 42, the snapshot module sends a resource recovery command of the snapshot logical block to the underlying logical volume of the snapshot logical volume LV2, and after the command returns successfully, updates the metadata of the snapshot logical volume LV2, and marks the snapshot logical block as invalid snapshot data block, the version merging of the snapshot data block corresponding to the snapshot logical block is completed,

Step 43, the snapshot module sends a remapping command of the snapshot logical block to the underlying logical volume of the snapshot logical volume LV2, and remaps the snapshot data block corresponding to the snapshot logical block from the underlying logical volume of the snapshot logical volume LV2 to the snapshot logical volume The underlying logical volume of volume LV1, wherein the remapping command includes the following parameters, the logical block number of the snapshot logical block, the source mapping device, and the target mapping device,

Step 44, after the remapping command returns successfully, update the metadata of the snapshot logical volume LV1 and the snapshot logical volume LV2 respectively, identify the metadata in the snapshot logical volume LV1 as valid snapshot data blocks for the snapshot logical block, and set The metadata identification of the snapshot logical volume LV2 is that the snapshot logical block does not have a valid snapshot data block, then the version merging of the snapshot data block corresponding to the snapshot logical block is completed;

If the data set in the snapshot logical volume LV1 is older than the data set in the snapshot logical volume LV2, proceed as follows:

Step 51, for the snapshot logical block B in the snapshot logical volume LV2, send the remapping command of the snapshot logical block to the thin provisioning system, and transfer the snapshot data block corresponding to the snapshot logical block B from the snapshot logical volume LV2 The underlying logical volume is remapped to the underlying logical volume of the snapshot logical volume LV1. The remapping command includes the following parameters, the logical block number of the snapshot logical block, the source mapping device, and the target mapping device,

Step 52, after the remapping command returns successfully, merge the metadata of the snapshot logical block in the snapshot logical volume LV1 and the snapshot logical volume LV2, and update the metadata of the snapshot logical volume LV1 accordingly, and update the snapshot The metadata of the logical volume LV2 identifies the snapshot logical block as having no valid snapshot data block, and the merging of the snapshot data block is completed at this time,

Wherein, the source mapping device and the target mapping device in the parameters of the remapping command are the underlying devices of the snapshot logical volumes LV2 and LV1 respectively, and when the valid snapshot data blocks corresponding to all snapshot logical blocks in the snapshot logical volume LV2 are merged into the The data merging operation of the snapshot logical volume LV1, the snapshot logical volume LV2 and the snapshot logical volume LV1 is completed.

The method for using the differential snapshot system of the present invention is characterized in that, in the thin provisioning module step, the write request step is specifically:

Step 61, the logical volume of the thin provisioning module receives a write request from the upper layer, searches the mapping item of the current snapshot logical block in the logical volume resource mapping table, if the snapshot logical block in the mapping information is not associated with the corresponding physical resource block, then It is the first write to the snapshot logic block, go to step 62; otherwise go to step 64;

Step 62, allocate resources for the snapshot logic block from the resource pool of the resource management module, and update the idle resource record in the resource pool;

Step 63, establishing the mapping relationship between the snapshot logical block and the physical resource block, adding the information of the physical resource block to the mapping item of the snapshot logical block, the information including the physical storage device and address where the physical resource block is located;

Step 64, forwarding the write request to the physical storage module;

Step 65, after the physical storage module writes back, it goes back up layer by layer.

The method for using the differential snapshot system of the present invention is characterized in that, in the thin provisioning module step, the read request step is specifically:

Step 71, the logical volume of the thin provisioning module receives the upper layer read request, searches the resource mapping table of the logical volume for the resource mapping item corresponding to the snapshot logical block accessed by the read request, and obtains the snapshot corresponding to the snapshot logical block The address of the data block in the physical storage device;

Step 72, redirecting the read request to the storage location of the snapshot data block in the physical storage device, and forwarding it to the physical storage module;

In step 73, after the bottom device returns, it returns upward layer by layer.

The method for using the differential snapshot system of the present invention is characterized in that, in the thin provisioning module step, the remapping step is specifically:

Step 81, the thin provisioning module receives the remapping command and parses the remapping operation parameters, that is, obtains the logical block number required for the remapping operation, the source mapping logical volume, and the target mapping logical volume. If the source mapping logical volume is LV3, The target mapped logical volume is LV4, and the logical block of the snapshot to be remapped is B;

Step 82, look up the mapping item corresponding to the snapshot logical block B in the resource mapping table of the source mapping logical volume LV3, if the snapshot logical block B in the mapping item is associated with a physical resource block, obtain its corresponding physical resource block P , execute step 83, otherwise execute step 87;

Step 83, look up the mapping entry corresponding to the snapshot logical block B in the resource mapping table of the target mapping logical volume LV4, if the corresponding mapping entry is found, obtain the corresponding physical resource block Q, and execute step 84, otherwise execute step 85 ;

Step 84, cancel the association between the snapshot logical block and the physical resource block Q, release the physical resource block Q occupied by the snapshot logical block B in the target mapping logical volume LV4, and update the idle resource record in the shared resource pool;

Step 85, associate the snapshot logical block B with the physical resource block P in the resource mapping table of the target mapping logical volume LV4, modify the mapping item corresponding to the snapshot logical block B, and write the physical resource associated with the snapshot logical block Block P information, which includes the physical device where the physical resource block is located and the address in the physical device;

Step 86, cancel the association between the snapshot logical block B and the physical resource block P in the resource mapping table of the source mapping logical volume LV3, and the remapping operation is completed;

Step 87, return to the upper layer.

The method for using the differential snapshot system of the present invention is characterized in that, in the thin provisioning module step, the resource recovery step is specifically:

Step 91, the thin provisioning module receives the resource recovery command, parses the parameters, that is, obtains logical volume and logical block information;

Step 92, look up the mapping item of the corresponding logical block in the resource mapping table of the logical volume specified by the parameter, if the corresponding physical resource block is associated in the mapping item, then perform step 93, otherwise perform step 95;

Step 93, cancel the mapping relationship between the logical block and the physical resource block, and release the mapped physical resource;

Step 94, update the free resource record of the resource pool;

Step 95, return to the upper layer.

The positive effects of the present invention are:

First, the combination of thin provisioning system and snapshot technology, on the one hand, separates physical resource management from the snapshot system, and the snapshot layer only needs to maintain the snapshot semantics without complex resource management; on the other hand, it realizes the dynamic allocation of global resources , the snapshot implementation method is decoupled from the resource management method, that is, the copy-on-write and redirect-on-write semantics do not need to pre-allocate snapshot space.

Second, the thin provisioning system implements the write-anywhere mechanism, and the snapshot layer maintains the copy-on-write and redirect-on-write semantics, realizing the nesting of copy-on-write, redirect-on-write, and write-anywhere technologies.

Description of drawings

FIG. 1 is a schematic diagram of the principle of copy-on-write in a differential snapshot implementation;

FIG. 2 is a schematic diagram of the principle of redirection-on-write in a differential snapshot implementation;

FIG. 3 is a schematic diagram of the principle of writing anywhere in the differential snapshot implementation;

Fig. 4 is a schematic diagram of the logical structure of the differential snapshot system of the present invention;

FIG. 5 is a schematic diagram of the functional structure of the differential snapshot system of the present invention;

Fig. 6 is a schematic diagram of snapshot semantic nesting of the differential snapshot system of the present invention;

FIG. 7 is a flow chart of the copy-on-write processing steps of the differential snapshot system of the present invention.

detailed description

In order to make the purpose, technical solution and advantages of the present invention clearer, the differential snapshot system and its usage method of the present invention will be further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention, and are not intended to limit the present invention.

The present invention provides users with two types of logical volumes, the source volume and the snapshot volume, wherein the snapshot volume depends on the source volume. The underlying device of the logical volume of the snapshot module is the logical volume exported by the thin provisioning module. The present invention synthesizes the processing logic of the three implementation modes of copy-on-write, redirection-on-write, and write-anywhere, and abstracts the logical functions of snapshots. Its hierarchical structure is shown in Figure 2, and its functional structure is shown in Figure 3. The thin provisioning system manages the logic volume of this module to realize on-demand configuration of resources, and the snapshot module only needs to maintain the snapshot semantics to implement copy-on-write or redirection-on-write logic.

The snapshot module communicates with the thin provisioning module through the interface module, and the snapshot module realizes communication by sending requests or resource management commands to the thin provisioning module, involving two processes of user request processing and data merging. The overall steps are divided into snapshot module steps and thin provisioning module steps according to the system structure. This step will be described below taking a snapshot data block of the snapshot module as an object.

1. Snapshot module steps

This step specifically includes a user write request processing step, a user read request processing step, and a data merging step, and these steps will be further described in turn below.

1. User write request processing steps

When a user write request is received, if copy-on-write is performed, the following steps are performed:

Step 11, the source volume receives the user's write request for the snapshot logical block, queries the metadata in the current snapshot volume dependent on the source volume, and judges whether this write is the first write of the snapshot data block, if the snapshot volume If there is no valid snapshot data block identified in the metadata of the snapshot logic block, then it is the first write, then perform step 12, otherwise, perform step 13,

Step 12, the snapshot source volume sends a remapping command to the underlying device, that is, the logical volume of the thin provisioning system, remaps old data blocks from the source volume logical space to the snapshot volume logical space, updates the snapshot volume metadata, and notifies the The data logical unit of the source volume has completed the logical copy of the snapshot data block,

Step 13, the source volume forwards the snapshot data write request of the user to the underlying device, and returns to the user layer after the write request is returned;

For redirect-on-write processing, perform the following steps:

Step 21, the source volume receives the user's write request for the snapshot logical block, queries the current snapshot volume dependent on the source volume, and redirects the write request to the snapshot volume logical space,

Step 22, the snapshot volume receives the user's snapshot logical block write request and forwards it to the underlying device,

Step 23, after the write request is returned, the metadata of the snapshot volume is updated and returned to the user layer.

2. User read request processing steps

The user's read request is the same as the processing logic defined by SNIA, and is divided into read snapshot source volume and read snapshot volume.

Step 31, the user reads the snapshot source volume, the specific steps are as follows:

Step 311, if it is a copy-on-write mode, then perform the following steps:

(a) redirecting the read request to the underlying logical volume of the snapshot source volume, and forwarding to the underlying thin provisioning module logical volume of the source volume;

(b) The logical volume of the thin provisioning module receives the read request and executes the read process, and returns the read data to the upper layer;

(c) The snapshot source volume receives the return from the underlying logical volume and returns it directly to the user.

Step 312, if it is the redirection-on-write mode, perform the following steps:

(a) The snapshot source volume receives a user read request, and searches for the current snapshot volume that depends on the snapshot source volume;

(b) Check the metadata corresponding to the requested logical block in the current snapshot volume, determine whether there is valid data of the logical block, if not, perform step (c); otherwise perform step (d);

(c) Obtain the precursor snapshot logical volume of the current snapshot volume, and use the precursor snapshot logical volume as the current snapshot volume, and execute (b);

(d) The read request is redirected to the current snapshot volume, and the snapshot volume receives the read request and forwards it directly to the logical volume of the underlying thin provisioning module;

(e) After the logical volume of the thin provisioning module receives the read request, it executes the read process of this layer, forwards the request to the underlying physical storage device, and returns the read data layer by layer.

Step 32, the user reads the snapshot volume, then perform the following steps:

Step 321, if it is the copy-on-write mode, then perform the following steps:

(a) The current snapshot volume receives a user read request, check whether the metadata of this volume has the requested data, if not, execute (b); otherwise execute (c);

(b) obtaining the predecessor logical volume of the current snapshot volume, and performing (a) with the logical volume as the current snapshot volume;

(c) The current snapshot volume forwards the request to the underlying thin provisioning module logical volume;

(d) The underlying logical volume receives a read request and executes the read process of this layer;

(e) Return the read data layer by layer.

Step 322, if it is the redirection-on-write mode, perform the following steps:

(a) The snapshot volume receives a read request, obtains the precursor logical volume of the snapshot volume, and redirects the read request to the precursor logical volume of the snapshot volume;

(b) The precursor logical volume receives a read request, checks the metadata in this volume, and judges whether there is requested data, if the requested data does not exist in the logical volume, then execute (c), otherwise execute (d );

(c) taking the precursor logical volume as the current request logical volume, obtaining the precursor logical volume of the current request logical volume, redirecting the read request to the precursor logical volume, and executing (b);

(d) forward the request to the logical volume of the underlying thin provisioning module;

(e) The logical volume of the thin provisioning module receives a read request, and executes the read process of this layer;

(f) Return the read data layer by layer.

3. Data Merging Steps

When it is necessary to merge the snapshot data in two snapshot volumes, perform the following steps,

Data merging performs version merging of the valid data in the LV2 volume with the data in the LV1 volume one by one. The following takes the valid data version merging operation of the snapshot logical block B as an example to illustrate.

If the valid snapshot data block corresponding to snapshot logical block B in logical volume LV2 is merged into logical volume LV1, where LV1 and LV2 are snapshot volumes of the same source volume at different times, if the data set in logical volume LV1 is newer than the For the data set in the logical volume LV2, perform the following steps:

Step 41, for the snapshot logical block B that has valid snapshot data blocks in the logical volume LV2, check the metadata in the logical volume LV1, and determine whether there is a new version snapshot data block of the snapshot logical block in the logical volume LV1, If so, execute step 42, otherwise execute step 43,

Step 42, the logical volume LV2 sends the resource reclamation command of the snapshot logical block B to the underlying thin-provisioned logical volume, after the command returns successfully, update the metadata of the logical volume LV2, and mark the snapshot logical block as having no valid snapshot data block, the version merging of the snapshot data block corresponding to the snapshot logical block B is completed,

Step 43, the logical volume LV2 sends a remapping command of the snapshot logical block B to the underlying thin-provisioned logical volume, and remaps the snapshot data block corresponding to the snapshot logical block from the underlying logical volume of the logical volume LV2 to the logical volume LV1 underlying logical volume, wherein the remapping command includes the following parameters, the logical block number of the snapshot logical block, source mapping logical volume, target mapping logical volume,

Step 44, after the remapping command returns successfully, update the metadata of the logical volume LV1 and the logical volume LV2 respectively, identify the metadata in the logical volume LV1 as valid snapshot data blocks for the snapshot logical block, and update the logical volume LV1 The metadata of LV2 is identified as the snapshot logical block does not have a valid snapshot data block, then the version merging of the snapshot data block corresponding to the snapshot logical block B is completed;

If the data set in the logical volume LV1 is older than the data set in the logical volume LV2, proceed as follows:

Step 51, for the snapshot logical block B in the logical volume LV2, send the remapping command of the snapshot logical block B to the thin provisioning system, and transfer the snapshot data block corresponding to the snapshot logical block B from the underlying logic of the logical volume LV2 The volume is remapped to the underlying logical volume of the logical volume LV1. The remapping command includes the following parameters, the logical block number of the snapshot logical block, the source mapped logical volume, and the target mapped logical volume.

Step 52, after the remapping command returns successfully, merge the metadata of the snapshot logical block B in LV1 and LV2, and accordingly update the metadata of the LV1, and update the metadata of the LV2, and identify the snapshot logical block as There is no valid snapshot data block. At this time, the version merging of the snapshot data block of the snapshot logical block B is completed.

Wherein, the source mapping logical volume and the target mapping logical volume in the remapping command parameters are the underlying logical volumes of logical volumes LV2 and LV1 respectively, when the valid snapshot data blocks corresponding to all snapshot logical blocks in the logical volume LV2 are merged into the The logical volume LV1, the logical volume LV2 and the logical volume LV1 have completed the data merge operation.

2. Simplify the configuration module steps

The steps of the thin provisioning module include a write request step, a read request step, a remapping step, a resource recovery step, and the like.

1. Write request steps

The steps are specifically:

Step 61, the logical volume of the thin provisioning module receives the upper-layer write request, and searches the mapping information of the current snapshot logical block in the resource mapping table of the current volume. If the mapping information associated with the snapshot logical block and the physical resource block is not found, It is the first write to the snapshot logical block, and execute step 62; otherwise, execute step 64;

Step 62, allocate resources for the snapshot logic block from the resource pool of the resource management module, and update the idle resource record in the resource pool;

Step 63: Establish a mapping relationship between the snapshot logical block and the physical resource block, and add the information of the physical resource block in the mapping corresponding to the snapshot logical block in the resource mapping table of the logical volume, the information includes the physical resource block where the physical resource block is located. Physical storage device and address;

Step 64, forwarding the write request to the physical storage module;

Step 65, after the physical storage module writes back, it returns to the upper layer.

2. Read request steps

The steps are specifically:

Step 71, the logical volume of the thin provisioning module receives the upper-layer read request, searches the corresponding resource mapping item in the resource mapping table, and obtains the physical device and the physical address of the data block required for storing the read request;

Step 72, redirecting the read request to the storage location of the snapshot data block in the physical storage device, and forwarding it to the physical storage module;

Step 73, after the bottom equipment returns to this module, it returns upward layer by layer.

3. Remapping steps

The semantics of the source volume or snapshot volume of the snapshot layer are transparent to the thin provisioning module. The thin provisioning module does not distinguish the source volume or the snapshot volume of the snapshot module, but only distinguishes the source device and the target device of the remapping operation, that is, the source logical volume and the target logical volume. roll. The steps of the remapping operation are described below taking a snapshot logical block as an object.

The steps are specifically:

Step 81, the thin provisioning module receives the remapping command, and parses the remapping operation parameters, namely the source mapping logical volume, logical block number, target mapping logical volume information, assuming that the source mapping logical volume is LV3, and the target mapping logical volume is LV4, The snapshot logical block to be remapped is B;

Step 82, look up the mapping item corresponding to the snapshot logical block B in the resource mapping table of the source mapping logical volume LV3, if the snapshot logical block B in the mapping item is associated with a physical resource block, obtain its corresponding physical resource block P , execute step 83, otherwise execute step 87;

Step 83, look up the mapping entry corresponding to the snapshot logical block B in the resource mapping table of the target mapping logical volume LV4, if the mapping entry of the snapshot logical block B is associated with a corresponding physical resource block, obtain the physical resource block Q , and execute step 84, otherwise execute step 85;

Step 84, cancel the association between the snapshot logical block and the physical resource block Q, release the physical resource block Q occupied by the snapshot logical block B in the target logical volume LV4, and update the idle resource record in the shared resource pool;

Step 85, associate the snapshot logical block B with the physical resource block P in the resource mapping table of the target logical volume LV4, modify the mapping item of the snapshot logical block B, and write the information of the physical resource block P, the information includes The physical storage device and the physical address where the physical resource block is located;

Step 86, cancel the association between the snapshot logical block B and the physical resource block P in the resource mapping table of the source mapping logical volume LV3, and the remapping operation is completed;

Step 87, return to the upper layer.

4. Resource recovery steps

The steps are specifically:

Step 91, the thin provisioning module receives the resource recovery command, parses the parameters, that is, obtains logical volume and logical block information;

Step 92, look up the mapping item of the corresponding logical block in the resource mapping table of the logical volume specified by the parameter, if the corresponding mapping item is found, then perform step 93, otherwise perform step 95;

Step 93, cancel the mapping relationship between the logical block and the physical resource block, and release the mapped physical resource;

Step 94, update the free resource record of the resource pool;

Step 95, return to the upper layer.

The present invention has the following advantages:

First, snapshot semantics are decoupled from physical resource management, and resource management is transparent to the snapshot layer. Thin provisioning system is used as the underlying support technology to implement resource management functions. Its core functions include resource allocation, resource mapping, resource recycling, etc.; the core functions of the snapshot layer include maintaining snapshot semantics, that is, maintaining dependencies between snapshot volumes, and managing snapshot layer metadata ( Identifies whether the data block has valid data in the logical space of the snapshot volume), and implements copy-on-write or redirection-on-write logic.

Second, the thin provisioning system implements the allocation-on-write strategy in accordance with the principle of allocation on demand, realizes write-anywhere semantics, and can optimize random writes to sequential writes to improve write performance.

Third, the separation of resource management and snapshot semantics not only solves the problem of combining copy-on-write, redirect-on-write, and on-demand resource allocation, but also realizes the nesting of copy-on-write, redirection-on-write, and write-anywhere mechanisms, as shown in the figure 4.

Claims (10)

1. a kind of residual quantity fast photographic system, for storage resource of distributing according to need, it is characterised in that the system includes：

Snapshot module, dependence, snapshot policy semanteme, management snapshot metadata for safeguarding source book and snapped volume；

Simplify configuration module, the simplify configuration module has resource management submodule, for realizing the resource management work(to snapshot Can, the function includes resource allocation, resource impact, resource reclaim；

The logical volume underlying device of the snapshot module is the logical volume of the simplify configuration module, by the simplify configuration module to this Distribution according to need storage resource is realized in the management of logical volume, and the snapshot module is logical by generic block layer interface with the simplify configuration module Letter, the snapshot module sends request of data or resource management commands realization communication by the simplify configuration module；

Wherein the snapshot module, during for the snapshot data block write request that user is received when the snapshot module, writes according to this and asks The snapshot policy for asking accessed logical volume to be applied, re-orientation processes when carrying out Copy on write treatment or writing, and simplified to this Configuration module sends and remaps order or write request, when the snapshot module receives the read request of user, according to read request ground Location and snapshot logic determine logical volume where requested data block, to simplify configuration module forwards read request, write request or read request After return, returned to user；It is additionally operable to same snapshot logical address that corresponding snapshot data block is carried out in different snapped volumes Merge, and order or resource reclaim order are remapped to simplify configuration module transmission.

2. residual quantity fast photographic system as claimed in claim 1, it is characterised in that in the snapshot module, the snapshot skill for being used Art is Copy on write or redirects when writing, and in the simplify configuration module, using distribution technique when writing, snapshot language is write in support everywhere Justice.

3. residual quantity fast photographic system as claimed in claim 1, it is characterised in that the simplify configuration module also includes：

Resource pool, for carrying out unified virtual management to physical memory resources, externally providing can use snapshot logical block；

Physical storage device, for storing snapshot data block；

Wherein, the resource impact in the logical volume that the snapshot logical block passes through the simplify configuration module management with the snapshot data block Table is corresponding；

Idling-resource table, available physical resource number of blocks and information in record Current resource pond.

4. a kind of application method of residual quantity fast photographic system to described in claim 1-3 any one, it is characterised in that the method Including snapshot module step and simplify configuration module step, wherein,

The snapshot module step, including：

User's request process step, during for the snapshot data block write request that user is received when the snapshot module, writes according to this The snapshot policy that the accessed logical volume of request is applied, re-orientation processes when carrying out Copy on write treatment or writing, and to the essence Simple configuration module sends and remaps order or write request, when the snapshot module receives the read request of user, according to read request Address and snapshot logic determine logical volume where requested data block, and to simplify configuration module forwards read request, write request or reading please After asking return, returned to user；

Data combining step, for corresponding snapshot data block to be closed in different snapped volumes to same snapshot logical address And, and remap order or resource reclaim order to simplify configuration module transmission；

The simplify configuration module step, including：

Write request step, for when the simplify configuration module receives the write request of the snapshot data block that snapshot module sends, Whether needed to distribute resource for the snapshot data block and set up reflecting for the snapshot data block and Physical Resource Block according to policy selection Relation is penetrated, then physical storage device is write data into the forwarding request of bottom physical memory module, it is successively upward after writing complete Return,

Read request step, for when the simplify configuration module receives the read request of the snapshot data block that snapshot module sends, Position of the data block in physical storage device is redirected requests to, physical storage device is forwarded to, from physical storage device Middle reading data return to user,

Step is remapped, order is remapped for receive the snapshot data block that snapshot module sends when the simplify configuration module When, remapping operation is carried out to the snapshot data block,

Resource reclaim step, it is fast to this during for the resource reclaim order that snapshot data block is received when the simplify configuration module Physical resource recovery is carried out according to data block.

5. the application method of residual quantity fast photographic system as claimed in claim 4, it is characterised in that the Copy on write treatment includes The following steps：

Step 11, source book receives user to snapshot logical block write request, and inquiry depends on the unit in the recent snapshot volume of the source book Data, whether be first writing to the snapshot logical block, if it is fast that this is identified in the metadata of the snapped volume if judging that this is write according to this According to logical block without effective snapshot data block, then to write first, step 12 is performed, otherwise, performs step 13,

Step 12, the source book is sent to the logical volume that underlying device is the simplify configuration module and remaps order, and the snapshot is patrolled Collect the corresponding legacy version snapshot data block of block and be remapped to snapped volume logical space from source book logical space, update snapped volume unit Data, and notify that the mathematical logic unit of the source book has completed the logical copy to snapshot logical block correspondence snapshot data block,

The snapshot data write request of user is transmitted to underlying device, i.e. simplify configuration logical volume by step 13, the source book, and this is fast Rear line layer is returned to according to data write request to return；

Re-orientation processes comprise the following steps when this is write：

Step 21, source book receives user to snapshot logical block write request, and inquiry depends on the recent snapshot of the source book to roll up, and will write please Ask and be redirected to the snapped volume logical space,

Step 22, the snapped volume receives the snapshot logical block write request of user, and is transmitted to underlying device,

Step 23, the write request updates snapped volume metadata after returning, and returns to client layer.

6. the application method of residual quantity fast photographic system as claimed in claim 4, it is characterised in that the data combining step is specific For：

If by the corresponding snapshot data merged blocks of snapshot logical block B that there is effective snapshot data block in snapshot logical volume LV2 To snapshot logical volume LV1, wherein LV1, LV2 be same source book in snapped volume not in the same time, or source book and snapped volume,

If the new data set in snapshot logical volume LV2 of data set in snapshot logical volume LV1, carries out following steps：

Step 41, for the snapshot logical block B in snapshot logical volume LV2, checks the first number in snapshot logical volume LV1 According to, judge whether there is the redaction snapshot data block of snapshot logical block in the volume, step 42 is performed if having, otherwise perform step 43,

Step 42, the snapshot module gives the resource reclaim of the snapshot logical block to order to snapshot logical volume LV2 Lower level logicals curly hair Order, after the order is successfully returned, updates the metadata of snapshot logical volume LV2, is without effectively fast by the snapshot logical block identification According to data block, then the versions merging to the corresponding snapshot data block of snapshot logical block is completed,

Step 43, the snapshot module remaps order to what snapshot logical volume LV2 Lower level logicals curly hair gave the snapshot logical block, The corresponding snapshot data block of snapshot logical block is remapped to the snapshot logical volume from snapshot logical volume LV2 Lower level logicals volume LV1 Lower level logicals are rolled up, wherein, this remaps order includes following parameters, and the logical block number (LBN) of the snapshot logical block, source mapping sets It is standby, target mapped device,

Step 44, this is remapped after order successfully returns, and updates the snapshot logical volume LV1 and snapshot logical volume LV2's respectively Metadata, is designated metadata in the snapshot logical volume LV1 snapshot logical block and there is effective snapshot data block, by the snapshot The metadata of logical volume LV2 is designated the snapshot logical block in the absence of effective snapshot data block, then complete to the snapshot logical block The versions merging of corresponding snapshot data block；

If the data set in snapshot logical volume LV1 carries out following steps older than the data set in snapshot logical volume LV2：

Step 51, for the snapshot logical block B in snapshot logical volume LV2, sends the snapshot and patrols to the simplify configuration module That collects block remaps order, and the corresponding snapshot data blocks of snapshot logical block B are rolled up from the Lower level logical of snapshot logical volume LV2 The Lower level logical volume of snapshot logical volume LV1 is remapped to, this remaps order including following parameters, the snapshot logical block is patrolled Volume block number, source mapped device, target mapped device,

Step 52, this remaps after order successfully returns, merges the snapshot logical block and patrolled in the snapshot logical volume LV1 and the snapshot The metadata in volume LV2 is collected, and updates the metadata of snapshot logical volume LV1 accordingly, and update the unit of snapshot logical volume LV2 Data, are that, without effective snapshot data block, the now merging to the snapshot data block is completed by the snapshot logical block identification,

Wherein, this remaps source mapped device and target the mapped device respectively snapshot logical volume LV2 and LV1 in command parameter Underlying device, and when in snapshot logical volume LV2 the corresponding effective snapshot data block of all snapshot logical blocks be merged into this Snapshot logical volume LV1, the snapshot logical volume LV2 are completed with the data union operation of snapshot logical volume LV1.

7. the application method of residual quantity fast photographic system as claimed in claim 4, it is characterised in that in simplify configuration module step In, the write request step specifically,

Step 61, the logical volume of the simplify configuration module receives upper strata write request, is searched in the logical volume resource mapping table The mapping of recent snapshot logical block, if the snapshot logical block does not associate corresponding Physical Resource Block in map information, be To writing first for the snapshot logical block, step 62 is performed；Otherwise perform step 64；

Step 62, for the snapshot logical block distributes resource from the resource pool of the resource management submodule, in renewal resource pool Idling-resource is recorded；

Step 63, sets up the mapping relations of the snapshot logical block and Physical Resource Block, in the mapping Xiang Zhongtian of the snapshot logical block Plus the information of the Physical Resource Block, the information include the Physical Resource Block where physical storage device and address；

Step 64, physical memory module is transmitted to by write request；

Step 65, after the physical memory module writes return, successively returns up.

8. the application method of residual quantity fast photographic system as claimed in claim 4, it is characterised in that in simplify configuration module step In, the read request step specifically,

Step 71, the logical volume of the simplify configuration module receives upper strata read request, is looked into the resource mapping table of the logical volume Look for the read request to access the corresponding resource impact of snapshot logical block, obtain snapshot data block corresponding with the snapshot logical block The address in physical storage device；

Step 72, storage location of the snapshot data block in physical storage device is redirected to by read request, is forwarded to physics Memory module；

Step 73, after underlying device is returned to this module, successively returns up.

9. the application method of residual quantity fast photographic system as claimed in claim 4, it is characterised in that in simplify configuration module step In, it is described remap step specifically,

Step 81, the simplify configuration module is received and remaps order, parses remapping operation parameter, that is, obtain this and remap behaviour Logical block number (LBN) needed for making, source mapping logic volume, target mapping logic volume, if mapping logic volume in source is LV3, target mapping logic It is LV4 to roll up, and snapshot logical block to be remapped is B；

Step 82, the corresponding mapping items of snapshot logical block B are searched in the resource mapping table that the source mapping logic rolls up LV3, if Snapshot logical block B is associated with Physical Resource Block in the mapping, obtains its corresponding Physical Resource Block P, performs step 83, no Then perform step 87；

Step 83, searched in the resource mapping table of target mapping logic volume LV4 to should snapshot logical block B mapping, If snapshot logical block B is associated with Physical Resource Block in the mapping, Physical Resource Block Q is obtained, and perform step 84, otherwise Perform step 85；

Step 84, cancellation snapshot logical block is associated with Physical Resource Block Q's, discharges the snapshot in target mapping logic volume LV4 The Physical Resource Block Q that logical block B takes, and update idle resource record in shared resource pond；

Step 85, closes the snapshot logical block B and Physical Resource Block P in the resource mapping table of target mapping logic volume LV4 Connection, changes the corresponding mapping items of snapshot logical block B, writes the Physical Resource Block P information of snapshot logical block association, should Information at least includes physical equipment where Physical Resource Block, and the address in the physical equipment；

Step 86, the snapshot logical block B and Physical Resource Block P is cancelled in the resource mapping table that the source mapping logic rolls up LV3 Association, remapping operation complete；

Step 87, returns to upper strata.

10. the application method of residual quantity fast photographic system as claimed in claim 4, it is characterised in that in simplify configuration module step In, the resource reclaim step specifically,

Step 91, the simplify configuration module receives resource reclaim order, and analytic parameter obtains logical volume, logical block information；

Step 92, searches the mapping of counterlogic block in the parameter specifies the resource mapping table of logical volume, if the mapping In be associated with corresponding Physical Resource Block, then perform step 93, otherwise perform step 95；

Step 93, cancels the logical block and Physical Resource Block mapping relations, and discharge the physical resource being mapped；

Step 94, updates resource pool idling-resource record；

Step 95, returns to upper strata.