WO2015089798A1

WO2015089798A1 - Data migration method and device

Info

Publication number: WO2015089798A1
Application number: PCT/CN2013/089982
Authority: WO
Inventors: 任仕飞; 薛强
Original assignee: 华为技术有限公司
Priority date: 2013-12-19
Filing date: 2013-12-19
Publication date: 2015-06-25
Also published as: CN103858092A; CN103858092B

Abstract

A data migration technology, comprising: acquiring load information of a first disk group; reading data in at least one emigration Extent of a target logic unit number (LUN) in the first disk group to a cache of a controller when the load information reaches an emigration threshold; acquiring at least one immigration Extent in a second disk group, wherein the number of the immigration Extents is the same as that of the emigration Extents, the RAID (Redundant Arrays of Independent Disks) level of each emigration Extent corresponds to that of an immigration Extent, and the immigration Extent is an idle Extent; and updating the mapping relation between the emigration LUN and the immigration Extent into a mapping relation between the emigration LUN and the immigration Extent.

Description

Data migration method and device

The present invention relates to storage technologies, and in particular, to a data migration technique. Background technique

In the storage field, in order to improve the read and write speed of data or enhance the security of data, a technology called Redundant Arrays of Independent Disks (RAID) is widely used. In this technology, multiple disks are linked together to form a RAID group according to the requirements of a certain RIAD level.

SCSI-based protocols, such as Small Computer System Interface (SCSI), or Fibre Channel (FC), Internet Small Computer System Interface (SCSI), or the like According to the capacity of the RAID group, the RAID group can be logically cut horizontally, and the RAID group is divided into a plurality of logical units (Logic Units). In the field of storage, it is customary to refer to a logical unit as a LU (Logic Unit Number), and the present invention follows this convention unless otherwise specified.

Each disk in the RAID group provides the same amount of storage for this LU, and each cut unit is called an LU. Each LUN can be accessed by the host separately. In this way, the original RAID group can be logically used as multiple RAID groups.

A disk group consists of at least one RAID group. Because the number of disks in a disk group is difficult to expand directly. When the disk group is established, it is difficult to determine the actual amount of resources required by the user. Therefore, it is often difficult for the disk group to provide resources to meet user requirements. Happening. Summary of the invention

The present invention provides a data migration technology that can implement data migration across a disk group.

In a first aspect, the present invention provides a data migration method, which is applied to a controller, where the controller is respectively connected to a first disk group and a second disk group, the first disk group DG and the second disk group. Each includes a disk, each of the disks includes a plurality of chunked Chunks, wherein at least two are from the same A Chunk of a different DG and a different disk constitutes a block group CKG according to a redundant array of inexpensive disk arrays, each CKG is divided into a plurality of lengths Extent, and each of the Chunks in the CKG provides the same size for the Extent The storage space, the method includes: acquiring load information of the first disk group; and reading, when the load information reaches the migration threshold, reading data of at least one of the target logical unit numbers LU in the first disk group In the cache of the controller, obtaining at least one migration into the Extent of the second disk group, the number of the moved intotent is the same as the number of the outbound Extent, and the RAID level of each of the outbound Extents and one The RAID level of the move into the Extent—correspondingly, the move-in Extent is an idle Extent; updating the mapping relationship between the move-out LU and the move-out Extent to the move-out LUN and the move-in Extent Mapping relationship.

In a first possible implementation manner of the first aspect, the load information includes: at least one of a bandwidth BPS per second being higher than a first threshold, and a free storage space being lower than a second threshold.

In a second aspect, the present invention provides a data migration apparatus, configured to migrate data of a first disk group to a second disk group, wherein the first disk group DG and the second disk group each include a disk, each of the The disk includes a plurality of chunked Chunks, wherein at least two Chunks from the same DG and different disks form a chunk group CKG according to a cheap disk redundant array RAID algorithm, each CKG is divided into a plurality of lengths Extent, and the CKG Each of the Chunks provides the same size storage space for the Extent, and the device includes: a parameter obtaining unit, configured to acquire load information of the first disk group; and a data reading unit, connected to the parameter obtaining unit 41, When the load information reaches the migration threshold, the data in the at least one of the target logical unit number LUNs in the first disk group is read out into the cache of the controller; the data writing unit, and the a data reading unit connection, configured to acquire at least one migration into the second disk group, the number of the migration extensions is the same as the number of the outbound extensions, each Corresponding to the RAID level of the outbound extension and the RAID level of the move into the Extent, the move-in Extent is an idle Extent; and the LU update unit is configured to move the moveout LU and the moveout Extent The mapping relationship is updated to the mapping relationship between the migration LU and the migration into the extension.

In a first possible implementation manner of the second aspect, the load information includes: at least one of a bandwidth BPS per second being higher than a first threshold, and an idle storage space being lower than a second threshold. DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings to be used in the embodiments will be briefly described below. The drawings in the following description are only some embodiments of the present invention, and can also be obtained according to the drawings. Other drawings.

1 is a flowchart of a data migration method according to an embodiment of the present invention;

2 is a schematic diagram of a controller according to an embodiment of the present invention;

3 is a structural diagram of a data migration apparatus according to an embodiment of the present invention. detailed description

The technical solutions of the present invention will be clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained based on the embodiments of the present invention are within the scope of the present invention.

In the traditional RAID technology, multiple disks are combined to form a RAID group according to the selected RAID level. Then, each disk in the RAID group is used as a minimum unit, and each disk takes a part of the same size to form a LUN. The LUN is provided to the host as a unit. Because the LU spans all the disks, when the host writes data, the data is written to the entire RAID group. When the data is read, the entire RAID group provides data to the host.

Most RAID levels provide data refactoring. For example, in RAID 3, one disk stores parity data, and the remaining disks store business data. When a disk fails, the data in the failed disk can be recovered by the remaining disks through a verification algorithm. However, because the storage space of a single disk is larger and larger, the number of disks in a RAID group is often small. For example, a disk group consists of 6 disks. Therefore, if one disk fails, it can only be in a RAID group. A small number of remaining disks participate in the refactoring, which can be lengthy, such as rebuilding 1 terabyte (TB) of data blocks, which takes 10 hours. If a new disk fails during refactoring, it may cause unrecoverable data loss to the storage system.

Different from the prior art, the RAID group is directly created by using the disk as the granularity. In the solution of the present invention, each of the plurality of disks is first divided into equal-sized blocks (CK, Chunk), for example, six Disks are grouped into a disk group (Disk Group, DG). Each disk in the DG is divided into Chunk of 64MB (megabytes). Then, the Chunk is used as the granularity to establish a RAID group. For example, three Chunks form a RAID group. These Chunk RAID groups, we call them block groups.

(Chunk Group, CKG). There will be no multiple Chunks belonging to the same CKG on the same disk. In other words, a disk provides at most one Chunk for a CKG. Not every disk in the DG provides Chunk for CKG, that is, the number of Chunks in CKG can be less than the number of disks in the DG.

It should be noted that the technology involved in the present invention is not limited to the storage medium itself, and therefore is applicable to other media such as a solid state hard disk, a rewritable optical disk, a floppy disk, a magnetic tape, and the like in addition to the magnetic disk, and the present invention is implemented for convenience of description. An example is a disk. The data referred to in the embodiments of the present invention, such as pictures, videos, database data, audio, text, and the like.

At this time, the disks do not form a traditional RAID relationship. Instead, they can form a larger disk group (for example, 96 disks). Chunk on the disk can form a CKG with Chunk of different disks in the disk group. Each CKG can have its own RAID level so that Chunk on one disk can belong to multiple tile groups at multiple RAID levels.

Each Chunk in CKG is divided into small blocks of storage space. Each Chunk provides a space of the same size, which can be composed of an Extent, and the size of an Extent is, for example, 256 KB (kilobytes). In this way, a CKG can be split into multiple Extents to form an Extent Pool. Extent is the basic unit that constitutes the LUN of the present invention. One or more Extents are selected from the Extent Pool to form a LUN for host access. When a LUN consists of multiple executs, the contigs that make up the LUN are from the same CKG, and can also come from different CKGs. The RAID levels of the Extents that make up the same LUN can be the same or different. The Raid level of the Extent and the CKG from it. The RAID level is the same. Different Extents belonging to the same LUN have the same storage space, which facilitates mutual exclusion during data migration.

In addition, due to the conventional RAID technology, the way to address data in the LU is the LU ID + Logical Block Address (LBA) address. If every 6 Chunks form a RAID group, then the storage space of each Extent is also from 6 disks. But since the LUN is made up of different Extents, each Extent can span different disks, so in this case, LUs can span more disks than traditional RAID groups. Therefore, when a disk fails, the number of disks participating in the reconstruction is also larger, and the reconstruction time is greatly shortened. The establishment of Chunk, CKG, Extent and LU in the embodiments of the present invention can be completed by a controller. The storage system is composed of a controller and a storage medium. The storage medium provides storage space, and the controller manages the storage space and is responsible for reading and writing data in the storage space. The disk that Extent spans refers to the disk from which the storage space of the Extent is derived.

Embodiments of the present invention can implement data migration across DGs. There may be multiple DGs in a storage system. Each DG consists of multiple disks of the same type. In the DG, CKG is selected according to the number required by the RAID algorithm. CKG is composed of blocks on different hard disks in the DG. The space is composed of several CKG spaces, so the data of the LUN can be distributed to multiple hard disks in the DG.

In the embodiment of the present invention, the same disk group belongs to the same fault domain, and different disk groups belong to different fault domains. That is to say, if any disk in a disk group fails to send, when restoring data, it depends on the disks of other disk groups and relies on other disks inside the disk group for recovery.

With CKG partitioning, disks of the same disk group have direct or indirect relationships with other disks. For example, if a total of five disks of disks VIII, B, C, D, and E form a DG, then Chunk that forms a CKG can come from any two or more disks. For example, Chunk of CKG1 can come to three disks of A, B and C, Chunk of CKG2 comes from four disks of A, B, C and D, and Chunk of CKG3 comes from three disks of C, D and E. · Since Disk A and Disk C provide Chunk to form CKG1, CKG1 and CKG2, Disk A and C establish direct contact; through CKG3, Disk C establishes direct contact with E; The direct connection between the contact and the disks C and E forms an indirect connection between the disks A and E.

As shown in Figure 1, the data migration across DG is described in detail below.

In step 11, the controller acquires the bandwidth of the first DG (BandPer Second, BPS) and the remaining disk space. The first DG has an LU that can be migrated out, which is called an outbound LU. The migrated LU map is composed of multiple Extents and has a mapping relationship with the Extent that constitutes it. Where the controller The first DG and the second DG are managed. The data in any Extent that moves out of any LUN can reduce the load of the first DG. The first DG and the second DG may be composed of storage media of the same specification, such as the same read/write speed storage medium.

Each time an input operation or an output operation is performed, the controller can record and count the total number of input/output operations in a time period. By averaging, BPS can be obtained every second. The controller can query the total storage space of the DG and the disk space occupied by each write operation to obtain the remaining space of the DG.

Step 12: When the BPS is higher than the first threshold, and the remaining disk space is lower than the second threshold, read out data of at least one Extent in the LUN to the controller.

This step describes the trigger conditions for data migration. BPS is the average bandwidth consumed per second for read/write operations. For example, according to the threshold setting, when the BPS is 100MB/S, and the remaining storage space is 10%, the migration is triggered, and then at least one Extent data in the LUN is migrated to the controller's cache. The number of Extent reading data can be set by the user, or the threshold can be set separately and then calculated according to the algorithm. For example, it can be set to not migrate when the BPS 50MB/S is satisfied and the remaining storage space is 50%. Then you can calculate or roughly estimate the number of Extents that need to be migrated based on the threshold settings for stopping the migration.

Step 13. Obtain the Extent information of the second DG, and read the data buffered to the controller and write it into the Extent of the second DG. The Extent that reads data from the first DG has the same RAID level and storage capacity as the Extent that writes data in the second DG. The Extent information of the second DG and the step 12 can also be executed first or in parallel. The Extent that is written does not store valid data and can be called an idle Extent. The RAID level of Extent is the RAID level of CKG that Extent belongs to.

If there are more than one DG managed by the controller, one of the multiple DGs may be selected as the second DG, or may be selected according to certain conditions, for example, setting the immigration threshold and moving into the threshold. The judgment parameters and the judgment parameters of the migration threshold are the same. When the eviction threshold considers whether the BPS is higher than the first threshold, the migrating threshold considers whether the BPS is lower than the third threshold. When the migration threshold considers whether the BPS is higher than the first threshold, and whether the free space is less than the second threshold; then the migration threshold considers BPS Whether it is lower than the third threshold, and whether the free space is greater than the fourth threshold. For example, the DG with the BPS and the remaining storage space satisfying certain conditions can be set as the second DG, for example, the BPS 20 MB/S and the remaining storage space is 80% to become the DG to be moved into the data. It is statistically advantageous to achieve the present invention by selecting any one of the DGs other than the first DG as the second DG.

Step 14. Update the Extent mapped by the outbound LU. Specifically, the mapping with the Extent of the data read out in the first DG is changed to the mapping with the Extent of the data written in the second DG. The memory of the Extend out of the data read out in the first DG can be released, making it an idle Extent for use by other LUs.

In the embodiment of the present invention, the first disk group and the second disk group each include a disk, and each disk includes a plurality of chunks Chunk. The CKG consists of at least two Chunks, which are composed of RAID algorithms. The Chunks that make up the same CKG are from different disks of the same DG. Each CKG is divided into multiple lengths Extent, and each Extent's storage space is provided by all Chunk of CKG. For the same Extent, each Chunk provides the same storage space.

The LU is composed of Extent, and records the Extent contained in the LUN through the mapping relationship. By updating the mapping relationship, a portion of the Extent in the LU comes from the second DG, completing the migration of data in the LU.

In this embodiment, the two parameters of BPS and remaining disk space are referred to as load information, and the two parameters satisfy the threshold at the same time, that is, when BPS 100MB/S, and the remaining storage space 10 %, as a trigger data migration operation. In other embodiments, only one of the parameters may be obtained, and the data migration is triggered when the one parameter satisfies the threshold. Or consider two parameters at the same time, set the weights of the two parameters, weight the two parameters to get a total value, and then compare the total value with the total threshold to determine whether to punish the data migration operation.

In the embodiment of the present invention, the LU-Extent mapping relationship is used to address the content of the LUN, and the data to be migrated is copied to the destination DG, and then the mapping relationship is changed to implement data migration across the DG. In traditional RAID, addressing with LU ID+LAB address cannot complete data migration across DG.

As shown in FIG. 2, from the hardware, the above data migration step can be performed by the processor 21 of the controller 2. Executing, a program instruction is stored in the memory 22 of the controller 2, and the processor 21 executes the program instruction to implement the method. The controller 2 is connected to the disk group 32 of the disk group 31, and specifically, the memory 22 can be connected to the disk group 31 and the disk group 32.

As shown in FIG. 3, the present invention further provides a data migration apparatus 4, which can apply the above migration method. The data migration device 4 includes a parameter acquisition unit 41, a data reading unit 42, a data writing unit 43, and an LU updating unit 44. The data migration appliance 4 is integrated into the controller and can migrate data between disk groups.

The parameter obtaining unit 41 is configured to obtain an input/output operation per second (BPS) of the first DG and a remaining disk space.

The first DG has LUNs that can be migrated out. It is called an migrating LUN. It can also be called an migrating LUN. The migrating LU map has multiple Extents. The controller manages the first DG and the second DG. Data in any Extent that moves out of any LUN can reduce the load on the first DG. The first DG and the second DG may be composed of storage media of the same specification, such as the same read/write speed storage medium.

Each time an input operation or an output operation is performed, it can be recorded in the buffer of the controller, and the total number of input/output operations in a time period is counted, and the BPSo per second can be obtained by averaging, and the parameter acquisition unit 41 queries. The total storage space of the DG, and the disk space occupied by each write operation, can obtain the remaining space of the DG.

The data reading unit 42 is connected to the parameter obtaining unit 41, and is configured to read out data of at least one Extent of the migrated LUN to the controller cache when the BPS is higher than the first threshold and the remaining disk space is lower than the first threshold. .

Thresholds can be set as triggers for data migration. For example, according to the threshold setting, when BPS ^IOOMB/S, and the remaining storage space is 10%, the migration is triggered, and then at least one Extent data in the LU is migrated to the controller's cache. The number of Extent reading data can be set by the user, or the threshold can be set separately and then calculated according to the algorithm. For example, it can be set to not migrate when the BPS 50MB/S is satisfied and the remaining storage space is 50%. Then you can calculate or roughly estimate the number of Extents that need to be migrated based on the threshold settings for these two stop migrations.

The data writing unit 43 is connected to the data reading unit 42 and configured to acquire the Extent of the second DG. Information, the data that is read into the controller cache is written into the Extence of the second DG.

The Extent that reads data from the first DG has the same RAID level and storage capacity as the Extent that writes data in the second DG. The Extent that is written does not store valid data and can be called an idle Extent. The RAID level of Extent is the RAID level of CKG to which Extent belongs.

If the DG has three or more, the data writing unit 43 can arbitrarily select one of them as the second DG, and can also be selected according to certain conditions, for example, setting the immigration threshold, and the item moving into the threshold matches the migration threshold. When the eviction threshold considers whether the BPS is higher than the first threshold, the migrating threshold considers whether the BPS is lower than the third threshold. When the eviction threshold considers whether the BPS is higher than the first threshold and the free space is less than the second threshold; then the migrating threshold considers whether the BPS is lower than the third threshold, and whether the free space is greater than the fourth threshold.

The LU update unit 44 is connected to the data writing unit 43 and is configured to update the Extent mapped by the migrated LU. Specifically, the mapping with the Extent of the read data in the first DG is changed to the second DG. The mapping of the Extent to the data. The LUN update unit 44 can further release the storage space of the Extend data read out from the first DG to make it an idle Extent.

In the embodiment of the present invention, the first disk group DG and the second disk group each include a disk, and each disk includes a plurality of blocks Chunk. CKG consists of at least 2 Chunks in accordance with the RAID algorithm. The Chunks that form the same CKG are from different disks of the same DG. Each CKG is divided into multiple lengths Extent, and each Extent's storage space is provided by all Chunk of CKG. For the same Extent, each Chunk provides the same storage space.

In this embodiment, the two parameters of BPS and remaining disk space are referred to as load information, and the two parameters satisfy the threshold at the same time, that is, when BPS 100MB/S, and the remaining storage space 10 %, as a trigger data migration operation. In other embodiments, only one of the parameters may be acquired, and the data migration is triggered when the one parameter satisfies the threshold. Or the same Consider two parameters, set the weights for the two items, weight them to get a total value, and then compare the total value with the total threshold to determine whether to penalize the data migration operation.

In the above embodiment, in addition to BPS, Input/Output Operations Per Second (IOPS) can be used as the threshold, BPS=IOPS XI/O size, and the I/O size is requested by a single read/write request. The amount of data, such as 512B, 4KB, 1MB, etc.

Those of ordinary skill in the art will appreciate that various aspects of the present invention, or possible implementations of various aspects, can be embodied as a system, method, or computer program product. Thus, aspects of the invention, or possible implementations of various aspects, may be in the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, etc.), or a combination of software and hardware aspects, They are collectively referred to herein as "circuits," "modules," or "systems." Furthermore, aspects of the invention, or possible implementations of various aspects, may take the form of a computer program product, which is a computer readable program code stored in a computer readable medium.

The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium includes, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing, such as random access memory (RAM), read only memory (ROM), Erase programmable read-only memory (EPROM or flash memory), optical fiber, portable read-only memory (CD-ROM).

The processor in the computer reads the computer readable program code stored in the computer readable medium, such that the processor can perform the functional actions specified in each of the steps, or combinations of steps in the flowchart; A means of functional action specified in each block of the block diagram or in a combination of blocks.

Claims

claims

1. A data migration method, applied in a controller, the controller is connected to a first disk group and a second disk group respectively, characterized in that: the first disk group and the second disk group both include Disk, each of the disks includes multiple chunks, among which at least 2 chunks are from the same disk group but different disks, forming a chunk group CKG according to the cheap disk redundant array RAID algorithm, and each CKG is divided into multiple The length of the Extent, and each Chunk in the CKG provides the same size storage space for the Extent. The method includes:

Get the load information of the first disk group;

When the load information reaches the migration threshold, read the data in at least one migration Extent of the target logical unit number LU in the first disk group into the cache of the controller;

Obtain at least one moving-in Extent in the second disk group, the number of the moving-in Extents is the same as the number of the moving-out Extents, the RAID level of each moving-out Extent and the RAID level of one of the moving-in Extents ——Correspondingly, the imported Extent is an idle Extent;

The mapping relationship between the moving-out LU and the moving-out Extent is updated to the mapping relationship between the moving-out LU and the moving-in Extent.

2. The method according to claim 1, characterized in that the load information reaches a migration threshold, including:

Bandwidth per second BPS is above the first threshold; and

Free storage space is below the second threshold.

3. The method according to claim 1, characterized in that the load information reaches a migration threshold, including:

Bandwidth per second BPS is higher than the first threshold.

4. The method according to claim 1, characterized in that the load information reaches a migration threshold, including:

Free storage space is below the second threshold.

5. The method according to claim 1, 2, 3 or 4, characterized in that: the mapping relationship between the moving out LU and the moving out Extent is updated to the mapping relationship between the moving out LU and the moving in Extent. Mapping level After that, further steps include:

Release the storage space of the moved Extent.

6. The method according to claim 1, 2, 3 or 4, characterized in that:

The load information of the second disk group meets the migration threshold.

7. The method according to claim 1, 2, 3 or 4, characterized in that:

The storage spaces of different extents belonging to the same LUN have the same size.

8. A data migration device for migrating data from a first disk group to a second disk group, characterized in that: the first disk group and the second disk group both include disks, and each disk includes Multiple block Chunks, among which at least 2 Chunks from the same disk group but different disks, form a block group CKG according to the cheap disk redundant array RAID algorithm, each CKG is divided into multiple lengths of Extent, and the CKG contains Each Chunk provides the same size storage space for the Extent. The device includes:

The parameter acquisition unit is used to acquire the load information of the first disk group;

The data reading unit is connected to the parameter acquisition unit 41. When the load information reaches the migration threshold, the data in at least one migration Extent of the target logical unit number LUN in the first disk group is read into the migration Extent. in the controller’s cache;

A data writing unit, connected to the data reading unit, is used to obtain at least one moving-in Extent in the second disk group. The number of the moving-in Extents is the same as the number of the moving-out Extents. Each of the The RAID level of the moving-out Extent corresponds to the RAID level of the moving-in Extent, and the moving-in Extent is an idle Extent;

LU update unit, used to update the mapping relationship between the moving-out LU and the moving-out Extent to the mapping relationship between the moving-out LU and the moving-in Extent.

9. The device according to claim 8, wherein the load information reaches a migration threshold, including:

Bandwidth per second BPS is above the first threshold; and

Free storage space is below the second threshold.

10. The device according to claim 8, characterized in that: the load information reaches a migration threshold Values, including:

Bandwidth per second BPS is higher than the first threshold.

11. The device according to claim 8, wherein the load information reaches a migration threshold, including:

Free storage space is below the second threshold.

12. The device according to claim 8, 9, 10 or 11, characterized in that: