WO2018120939A1 - Storage system expanded data migration method and storage system - Google Patents

Abstract

Embodiments of the present invention relate to an expanded data migration method of a storage system, and a storage system, the storage system performing data migration after an expansion numbered i by means of using an auxiliary balanced incomplete block design; given that the number of blocks including any element in the auxiliary balanced incomplete block design is identical, and each migration unit includes an identical number of verification blocks, expanded data migration volume is minimized; in this way, time required for expanded data migration decreases significantly, and response delays to user requests caused by a data migration operation being necessary after expansion are reduced.

Description

Method and storage system for data migration after storage system expansion

This application is required to be submitted to the China Patent Office on December 29, 2016, the application number is 201611251370.8, the application name is “a method for data migration after storage system expansion, storage system”, and the Chinese patent was submitted on February 23, 2017. The priority of the Chinese Patent Application No. 201710100458.8, the entire disclosure of which is hereby incorporated by reference.

Technical field

The present invention relates to the field of storage technologies, and in particular, to a method and a storage system for data migration after a storage system is expanded.

Background technique

In the prior art, a storage system is used to store a large amount of data of an enterprise and a user. As the information grows explosively, more and more data needs to be stored in the storage system, so that an existing storage system often occurs. Demand for expansion. In the prior art, after the capacity expansion, a new data layout manner needs to be designed for the expanded storage system, and all the saved data in the storage system before the expansion is migrated to the expanded capacity according to the new data layout manner. According to this method, all the data saved in the storage system before the capacity expansion will be migrated. As a result, the data migration workload after the expansion is too large, and the data migration takes much time.

Summary of the invention

The present invention provides a method and a storage system for data migration after the storage system is expanded. After the storage system is expanded, only part of the data in the saved data before the expansion is migrated, thereby reducing data migration after capacity expansion. The amount of work.

In a first aspect, an embodiment of the present application provides a method for data migration after a storage system is expanded. The storage system includes a controller. Before the i-th expansion, the storage system includes v _i-1 disks, the i-th time. After the expansion, the storage system includes v _i disks, i is greater than or equal to 1; the controller is communicable with all disks; and the data is distributed on the disk based on a check block dispersion technology in the storage system. include:

The controller confirms the number of blocks included in the migration unit of the i-th expansion;

The storage space of each of the v _i-1 disks included in the storage system before the expansion is divided into a plurality of migration units, and each migration unit includes the foregoing number of blocks;

Based on the number of disks v _i included in the storage system after expansion, and the number of disks included in the storage system before the capacity expansion, v _i-1 confirms the auxiliary balance incomplete block design required for data migration after the i-th expansion.

Based on the auxiliary balance incomplete block design

Selecting at least one migration unit from the plurality of migration units of the v _i-1 disks before the expansion, and migrating the blocks included in the selected at least one migration unit to the expanded v _i disks On one of the disks.

With reference to the first aspect, in a first implementation of the first aspect, the data is distributed on the disk based on a check block decentralization technique in the storage system, specifically:

The controller determines, according to the complete block design table, a save location of the data in the storage system, the complete block design table is generated based on a balanced incomplete block design, and the parameter of the balanced incomplete block design is (b, v, k, r, λ), where:

v indicates that the storage system contains v disks;

b indicates that the storage system includes b strips corresponding to b blocks of the balanced incomplete block design, and each element in each of the b blocks represents 1 block at The disk number of the disk in the storage system;

k means that each stripe contains k blocks;

r means that each disk contains r blocks;

Any two of the v disks respectively contain λ identical strips;

The block is a data block or a check block.

In conjunction with the first implementation of the first aspect, in a second implementation of the first aspect, the method further includes: generating a complete block design table T[i] for each of the expanded storage systems, the T[ i] is a complete block design table used to save new data after the ith expansion, where b _i , v _i , k _i , r _i , λ _i are BIBD parameters constituting T[i].

With reference to the first aspect, in a third implementation of the first aspect, the number of disks v _i included in the storage system after the capacity expansion, and the number of disks included in the storage system before the capacity expansion v _{i-1 are} confirmed Auxiliary balance incomplete block design required for data migration after the i-th expansion

Specifically:

Confirm the said

Parameter

The value of the value of v _i ,

The value of the value is the value of the v _i-1 ;

Based on the stated

with

Query balance incomplete block design database to confirm

Value

Based on the stated

And said

The value confirmation confirms the query of the incomplete block design database to confirm the composition

Zone group

Confirm the said

Parameter

with

The value.

In combination with the first aspect and any of the above implementations, in a fourth implementation of the first aspect,

The controller confirms that the number of blocks included in the migration unit is specifically determined by the following formula: the number of blocks included in the migration unit:

Where r _i-1 is a parameter designed to form a balanced incomplete block of T[i-1];

For the parameters designed to balance the incomplete block of B[i-1], gcd is the operation that takes the greatest common divisor

In combination with the first aspect and any one of the foregoing implementations, in a fifth implementation of the first aspect, the design is based on the auxiliary balance incomplete block

Selecting at least one migration unit from the plurality of migration units of the v _i-1 disks before the expansion, and migrating the blocks included in the selected at least one migration unit to the expanded v _i disks The specific one of the disks is:

The selected at least one migration unit is Rm,n, where m represents a migration unit number of the selected at least one migration unit, and n represents a disk number of the selected at least one migration unit;

Obtaining a set S0, the set S0 being the

The difference between the block m and the set {0,1,···, v _i-1 -1};

Obtaining a set S1, the set S1 being a set {0, 1, . . . , v _i-1 -1}

The difference set of the block m;

If the value of n is the kth small element in the set S1, it is confirmed that the target disk number to which the migration unit Rm, n is to be migrated is an element whose value is the kth in the set S0;

The migration unit Rm,n is migrated to a disk corresponding to the target disk number.

Optionally, the offset of the migration unit Rm,n remains unchanged during the migration process.

A second aspect of the embodiment of the present application provides a storage system, where the storage system includes a controller, and the storage system includes v _i-1 disks before the i-th expansion, and the storage system after the i-th expansion Include v _i disks, the i being greater than or equal to 1; the controller is communicable with all disks;

The disk is configured to store data, where the data is distributed on the disk based on a check block dispersion technology;

The controller is configured to confirm the number of blocks included in the migration unit of the i-th expansion; and divide the storage space of each of the v _-1 disks included in the storage system before the expansion into multiple migrations a unit, each of the migration units includes the above-mentioned number of blocks; based on the number of disks v _i included in the storage system after expansion, and the number of disks included in the storage system before the capacity expansion v _i-1 , the data migration after the i-th expansion is confirmed. Auxiliary balance incomplete block design required

Based on the auxiliary balance incomplete block design

Selecting at least one migration unit from the plurality of migration units of the v _i-1 disks before the expansion, and migrating the blocks included in the selected at least one migration unit to the expanded v _i disks On one of the disks.

The first to fifth implementations of the second aspect are the same as the first to fifth implementations of the first aspect described above, and are not described herein again.

In a third aspect, an embodiment of the present application provides a controller, where the controller is applied to a storage system, where the storage system includes the controller, and the storage system includes v _i-1 before the i-th expansion a disk, after the i-th expansion, the storage system includes v _i disks, i is greater than or equal to 1; the controller is communicable with all disks; and the data is distributed in the storage system based on a check block dispersion technique The controller includes a processor and a memory, the memory stores program instructions, and the processor is configured to execute the program instructions to perform the following actions:

The controller confirms the number of blocks included in the migration unit of the i-th expansion;

The storage space of each of the v _i-1 disks included in the storage system before the expansion is divided into a plurality of migration units, and each migration unit includes the foregoing number of blocks;

Based on the number of disks v _i included in the storage system after expansion, and the number of disks included in the storage system before the capacity expansion, v _i-1 confirms the auxiliary balance incomplete block design required for data migration after the i-th expansion.

Based on the auxiliary balance incomplete block design

Selecting at least one migration unit from the plurality of migration units of the v _i-1 disks before the expansion, and migrating the blocks included in the selected at least one migration unit to the expanded v _i disks On one of the disks.

The first to fifth implementations of the third aspect are the same as the first to fifth implementations of the first aspect described above, and are not described herein again.

With reference to the third aspect, and the second implementation of the third aspect, in a seventh implementation of the third aspect, the controller further includes a first communication interface, and the first communication interface is further configured to receive from the host Write request The request carries a new data to be written, the memory is further configured to buffer the new data to be written, the processor determines a check block of the new data to be written, and the processor further For writing the new data to be written and its check block to the disk through the second communication interface according to the T[i].

In a fourth aspect, an embodiment of the present application provides a storage medium, where the program is stored, and when the program is run by the computing device, the computing device performs the foregoing first aspect or any implementation manner of the first aspect. Methods. The storage medium includes, but is not limited to, a flash memory, an HDD, or an SSD.

A fifth aspect of the present application provides a computer program product, comprising: program instructions, when the computer program product is executed by a computer, the computer performs the method provided by any of the foregoing first aspect or the first aspect . The computer program product can be a software installation package.

In the embodiment of the present invention, the storage system completes the ith expansion and uses the auxiliary balance incomplete block design.

Guide data migration due to the incompletely balanced block design of the auxiliary balance

Any element in the middle contains the same number of blocks, and each migration unit contains the same number of check blocks. After the expansion, the data migration amount is minimized, thereby significantly reducing the time required for data migration after capacity expansion. It also reduces the delay in responding to user requests caused by data migration operations after capacity expansion.

DRAWINGS

FIG. 1 is a schematic structural diagram of a storage system according to an embodiment of the present invention;

2 is a block diagram of a (4, 4, 3, 3, 2)-BIBD according to an embodiment of the present invention;

3 is a diagram showing an example of a complete block design based on the BIBD shown in FIG. 2 according to an embodiment of the present invention;

FIG. 4 is a schematic flowchart of a method for data migration after a storage system is expanded according to an embodiment of the present disclosure;

FIG. 5 is a block diagram of an auxiliary BIBD according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of data migration according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of data layout after data migration is completed after capacity expansion according to an embodiment of the present disclosure;

FIG. 8 is a flowchart of a method for writing new data by the storage system after capacity expansion according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of writing new data by the storage system after capacity expansion according to an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of a controller according to an embodiment of the present invention.

detailed description

FIG. 1 is a schematic diagram of a storage system provided by an embodiment of the present invention. The storage system includes a controller and a disk. In an actual networking, the disk may have multiple disks.

The controller can communicate with a host (not shown), such as through a storage area network (SAN), or through other networks, such as an Ethernet, a local area network, a wide area network, and the like. Host communication. The controller can be a computing device, such as a server, desktop computer, etc., on which an operating system and applications can be installed. The controller can receive input and output (I/O) requests from the host. The controller can also store data (if any) carried in the I/O request and write the data to disk.

FIG. 1 is merely an exemplary illustration. In practical applications, a storage system may include multiple controllers, each having a physical structure and function similar to that of a controller, between each controller, and between controllers and disks. Phase Intercommunication. This embodiment does not limit the number of controllers, the connection mode between the controllers, and the connection mode between any controller and the disk.

The controller is a system controller of the storage system, and the system controller is generally an independent device. Unless otherwise specified, the controller in this embodiment refers to a system controller.

Each block (Chunk) to be saved in the storage system determines the location of the block in the storage system in a manner that satisfies the Balanced Incomplete Block Design (BIBD), each of which A block can be either a data block or a check block. This data distribution is also commonly referred to as Parity Declustering. Among them, BIBD is the “balanced incomplete block design” in combinatorial mathematics, which is defined as follows:

Let the set X={0,1...,v-1}, B={T0,T1...,Tb-1} be a block design on the set X. If the following conditions are met, then B is called (b) ,v,k,r,λ)-BIBD:

|T0|=|T1|=...=|Tb-1|=k;

Any element in X belongs to r blocks;

3) Any pair of elements in X are included in the λ block.

As shown in Fig. 2, it is a (4,4,3,3,2)-BIBD, where b=4, v=4, k=3, r=3, λ=2; where X={0,1 , 2, 3}, B = {T0, T1, T2, T3}, T0 is the block 0, T1 is the block 1, T2 is the block 2, T3 is the block 3, and satisfies |T0 |=|T1|=|T2|=|T3|=k=3; any element in X belongs to r=3 blocks, for example, element 0 in X belongs to block T0, T1, T2; Any pair of elements is included in λ=2 blocks, such as element 0 and element 1, which are included in block T0, T1.

In the embodiment of the present invention, the controller generates a Full Block Design Table by using (b, v, k, r, λ)-BIBD, and determines each block based on the complete block design table. The location on the disk in the storage system. In the storage system corresponding to the complete block design table: a disk array composed of v disks, a total of b strips, each stripe containing k blocks, each disk containing r blocks, and Any two disks respectively contain λ identical strips. For example, the strips to which each block held by the disk 1 belongs are identical to the λ strips in the strip to which each block held by the disk 2 belongs. Each of the b blocks of the BIBD corresponds to the b strips, and each element in each block represents a disk number of a disk that should be saved in the storage system, respectively. Some elements can be marked in the block to represent the parity block (Parity Chunk), and this mark makes the check block evenly distributed. In the embodiment of the present invention, the verification block decentralization technique uses the complete block design table to guide the disk number that each block (such as a data block or a check block) should be placed, and each of the In a position where the offset is as small as possible, in the embodiment, the offset refers to dividing the storage space of each disk by the length of the block, and the storage space of each block length corresponds to one. Offset, the offset can be numbered starting from the starting storage location of each disk, generally starting from 0.

FIG. 3 is a diagram showing an example of a complete block design table constructed by (4, 4, 3, 3, 2)-BIBD shown in FIG. 2 according to an embodiment of the present invention, and based on the complete block design table. The position distribution of the completed block in its corresponding storage system. In the complete block design table on the left side of FIG. 3, the strip numbers 0-3 correspond to the block 0-3 in FIG. 2, respectively, as described above. Each element within the corresponding block of each strip represents the disk number of the disk of the corresponding block in the storage system, and the box in the figure is used to mark the check block in the strip, each strip The element in the band box indicates the disk number of the disk in which the check block in the strip is stored in the storage system. As shown in the figure, the block T0 corresponding to the strip 0 takes the value {0, 1, 2}, that is, the three blocks included in the strip 0 should be saved in the disk number respectively. On the three disks of D0, D1, and D2, the first element of strip 0 is a check block, and the check block should be on disk D0, so P0, D0, 0, D0 in the disk array on the right side of FIG. 1 is on the disks D0, D1, D2, respectively, where Pi represents the parity block of the i-th slice, Di, j represents the data block of the i-th slice, and j takes values from 0 to k-1, respectively k data blocks in the i-th strip.

In the embodiment of the present invention, it is assumed that T[0], T[1], ..., T[t] is a complete block design table used in the expanded history, where T[i] is after the ith expansion. The complete block design table used to populate the new data, T[0] is the complete block design table used before the storage system has not undergone any expansion, where b _i , v _i , k _i , r _i λ _i is a BIBD parameter constituting T[i]. The number of disks added during the i-th expansion process is v _i -v _i-1 .

Auxiliary balance incomplete block design used in the history of capacity expansion, where

The secondary BIBD required for data migration after the ith expansion.

for

BIBD parameters and need to be met

Based on the above description, those skilled in the art can understand that if the storage system has not undergone any expansion, the controller determines the storage location of the newly written data on the disk based on the T[0]. The saved data is also queried based on the T[0]; when the first expansion is performed, the generation is performed.

And based on the

Complete data migration, and then in the storage system after the first expansion, the controller is based on the T[0] and the

Completing the query of the saved data before the first expansion, and further generating T[1] for determining the preservation of the data written in the storage system after the first expansion based on the T[1] Position; further, when the second expansion is required, the controller generates

And based on the

Performing data migration, and then in the storage system after the second expansion, the controller is based on the T[0], the

Said T[1] and said

Completing the query of the saved data before the second expansion, and further generating T[2] for determining the preservation of the data written in the storage system after the second expansion based on the T[2] Position, and so on, after the ith expansion, the controller is based on the T[0], T[1], ..., T[i-1], and based on the

Completing the query of the data saved before the ith expansion, and further generating T[i], and determining the data written in the storage system after the ith expansion based on the T[i] The location of the save.

FIG. 4 is a schematic flowchart diagram of a method for data migration after a storage system is expanded according to an embodiment of the present invention.

Step 400: The controller confirms the number of blocks included in the migration unit.

Before capacity expansion, the storage space of each disk is divided into several areas, each area contains the same number of blocks, and the amount of data read in the same offset range area of each disk in the storage system is the same, As a migration unit, each migration unit contains the same number of parity blocks. During data migration, the migration unit is the smallest unit of data migration. If a migration unit needs to be migrated, all the blocks within it will migrate. In the embodiment of the present invention, the definition of len _{i is} as follows, and is used to indicate the number of blocks included in one migration unit in the ith expansion:

The meaning of the above formula is as follows:

When i=1, that is, when the storage system is expanded for the first time, the number of blocks that the migration unit should include is r ₀ ;

When i>1,

Where gcd is the operation taking the greatest common divisor, where r _i-1 is a parameter designed to form a balanced incomplete block of T[i-1];

A parameter designed to form a balanced incomplete block of B[i-1].

Assuming that the storage system composed of four disks as shown in FIG. 3 has not been expanded yet, the BIBD parameters of T[0] constituting the storage system are (b ₀ , v ₀ , k ₀ , r ₀ , λ ₀ ) = (4, 4, 3, 3, 2), taking the first expansion as an example, the number of blocks included in the migration unit is len ₁ = r ₀ = 3, that is, the block included in the migration unit of the first expansion The number is 3.

After the controller determines the number of blocks len _i included in the migration unit required for data migration after the i-th expansion based on the foregoing method, each of the storage systems is included before the ith expansion based on the len _i The storage space of the disk is divided into one or more areas. If an area contains a valid block, that is, the area stores valid data before the expansion, the area is a migration unit. A person skilled in the art can understand that if an area does not contain a valid block, it indicates that the area does not store data before the capacity expansion, or the stored data has been deleted, indicating that the area does not need to be migrated after the expansion. . In the embodiment of the present invention, a migration unit may be represented by Rm, n, where m represents the migration unit number of the migration unit, and n represents the disk number of the migration unit before the ith expansion of the storage system. In the embodiment of the present invention, the m can be obtained by the following formula:

That is, the offset of each block on the disk n is divided by the len _i and rounded down to know the migration unit number of the migration unit to which the block belongs. According to the foregoing example, it is assumed that the number of blocks included in the first expansion and migration unit of the storage system is three, and according to the above formula, three offsets are 0, 1, and 2 on each disk. The block belongs to the migration unit 0, the three blocks with the offsets of 3, 4, and 5 belong to the migration unit 1, and so on.

Step 401: The controller confirms that the auxiliary balance incomplete block design of the migration is performed.

As mentioned above, where

For the auxiliary BIBD used in the i-th expansion process,

for

BIBD parameters and need to be met

That is, the auxiliary BIBD required to migrate data during the ith expansion.

In the value of the parameter,

The value of the number of disks v _i included in the storage system after the i-th expansion is

The value is the value of the number of disks v _i-1 included in the storage system before the i-th expansion. Further, it is clear

with

After taking the values of the two parameters, the controller can query the BIBD database, from the

with

Select one of the multiple λ values to match

The value of the need to emphasize is when

The larger the value, the higher the time required for subsequent data queries. Usually,

It is recommended to take a smaller value. Confirmed by the controller

with

After the value is obtained, it can be queried and described in the BIBD database.

with

Corresponding block to get the secondary BIBD required for the i-th migration. Further, the controller further confirms the auxiliary BIBD

in

with

In the embodiment of the present invention, the method can be obtained in two ways.

with

Value, method 1: can be confirmed according to the following formula

with

Value:

Method 2: The controller can directly query and query the foregoing in the BIBD database by counting

with

The corresponding block can also be obtained

with

The value.

For the storage system composed of four disks shown in FIG. 3, its BIBD is shown in FIG. 2 before the first expansion, and its corresponding T[0] is shown in FIG. 3. Assume that three disks are added during the first expansion, that is, the number of disks included in the array after expansion is v ₁ =4+3=7,

The parameter is

Parameters need to be met

As shown in Figure 5, for a condition that satisfies this condition

Then

The parameters are (7, 7, 4, 4, 2).

The execution of the above steps 400 and 401 is not limited by the chronological order. The two steps may be performed simultaneously, or the step 400 may be performed first and then the step 401 may be performed, or vice versa.

Step 402: The controller migrates data

The controller is based on the auxiliary balance incomplete block design

Selecting at least one migration unit from the plurality of migration units of the v _i-1 disks before the expansion, and migrating the blocks included in the selected at least one migration unit to the expanded v _i disks On one of the disks. Wherein the auxiliary balance incomplete block design

Determining the disk number of each migration unit after the i-th expansion, for any migration unit Rm,n, according to the

The block m determines whether the Rm, n needs to be migrated. In the embodiment of the present invention, the following criteria may be used to determine whether a migration unit Rm,n needs to be migrated: if Rm,n is already on the disk indicated by the secondary BIBD, the migration unit Rm,n does not need to be migrated, otherwise it migrates to On the newly added disk indicated by the secondary BIBD. Specifically, when the value of n is already in the

In the block m, it indicates that Rm,n is already on the disk indicated by the secondary BIBD, then the migration unit Rm,n does not need to be migrated, otherwise according to the

The indication of the zone group m migrates the migration unit Rm,n onto the newly added disk. In the embodiment of the present invention, the according to the

The indication of the block m is to migrate the migration unit Rm, n to the newly added disk. For details, refer to the following method:

Assuming that the disk number to which the migration unit Rm,n should be migrated is represented as X, then X can be calculated as follows:

Obtaining a set S0, the set S0 being the

The difference between the block m and the set {0,1,···, v _i-1 -1};

Obtaining a set S1, the set S1 being a set {0, 1, . . . , v _i-1 -1}

The difference set of the block m;

If the value of n is the kth small element in the set S1, the value of the disk number X is the kth small element in the set S0.

For example, as shown in FIG. 6 , it is a schematic diagram of data migration, wherein each dot represents 1 migration unit. As shown in the figure, after expansion, the dots located on the disks D4 to D6 are migrated units. After the expansion, the data layout is as shown in FIG. 7. The migration process of the data block D4, 1 is as follows: before the expansion, D4, 1 is on the disk D2, and the offset is 3, and each migration unit calculated in step 400 includes For the three blocks, the migration unit number L to which D4,1 belongs is 3/3=1, and the migration unit to which the D4,1 belongs can be marked as R1,2, that is, the value of m is 1, and the value of n is Is 2. Therefore, the migration is performed by the block 1:0, 1, 4, 5} of the auxiliary BIBD, because the value 2 of n is not in the block 1, indicating that the migration unit R1, 2 needs to be migrated. It can be understood by those skilled in the art that, if the migrated unit corresponding to the saved block of the same offset range of the four disks (D0-D3) before migration is corresponding to the group of {0, 1, 2, 3}, then The disk number X to which the migration unit R1, 2 needs to be migrated is specifically: the set S1 is {0, 1, 2, 3} - {0, 1, 4, 5} = {2, 3}, the set S0 Is {0,1,4,5}–{0,1,2,3}={4,5}, because the value 2 of n is the first small element in the set S1, that is, k is taken The value is 1, so the disk number X to which the migration unit R1,2 needs to be migrated should be the first small element in the set S0, that is, 4, that is, the migration unit R1, 2 needs to be The disk D2 before the capacity expansion is migrated to the disk D4 after the expansion. The migration of the migration unit R1, 2 remains unchanged during the migration. Similarly, for the migration unit R1, 3, it can be determined that the migration needs to be performed according to the above method. Disk D5. It can be understood by those skilled in the art that the above example is only described by using the data block D4, 1 as an example. In the actual migration process, the migration unit is migrated as a whole, for the migration unit R1, 2, for example. The D3, 1, P6, and D7, 1 are migrated to the disk D4 as a whole. After the first expansion, if the user needs to query the data that has been saved in the storage system before the expansion, the controller Can refer to the T[0] and the

The data query can be completed. So far, in the embodiment of the present invention, the storage system completes the data migration after the first expansion. In the embodiment of the present invention, after the expansion, the data migration is guided by using the auxiliary BIBD. Since the number of the blocks included in any element in the auxiliary BIBD is the same, and each migration unit contains the same number of check blocks, Therefore, the data block and the check block are evenly distributed after the expansion, and the data migration amount is minimized during the capacity expansion process. As shown in Figure 7, less than 50% of the data is migrated, thereby significantly reducing data migration after capacity expansion. The time required also reduces the delay in responding to user requests due to the need for data migration operations after capacity expansion.

Further, because the new data source is continuously written after the storage system is expanded, the subsequent new data should be saved in the expanded storage system, as shown in FIG. 8 , which is provided by the embodiment of the present invention. A flow chart of a method for writing new data by the storage system after capacity expansion.

Step 800: The controller generates a new complete block design table T[i]

Determining, by the controller, the remaining available storage space of each disk in the expanded storage system, where the remaining available space of the disk already existing in the storage system before the foregoing expansion is generally after the data is migrated The space that is released may also include the space that is not occupied before the expansion. For the newly added disk in the expansion, the remaining available space is generally the unoccupied storage space except the space occupied by the migrated data. Logically, the controller regards the remaining free space of each disk as a logically independent disk, and then uses the storage system composed of the remaining space of each disk as a logically new storage system, as shown in FIG. 7 above. In the storage system after the first expansion, there are a total of 7 disks, and each disk has 9 offsets of available storage space. The controller generates a new complete block design table T[i] for the expanded storage system, where the T[i] is used for storing the new data after the i-th expansion is written into the storage system. Find. The five parameters (b _i , v _i , k _i , r _i , λ _i ) of the BIBD constituting the T[i], the controller can flexibly be v _i , k _i according to the needs of the storage strategy. λ _i takes a value, and obtains the value of the BIBD constituting the T[i] and the corresponding b _i , r _i by the foregoing method, and then obtains the T according to the obtained new BIBD. i].

Step 801: The controller writes new data to the storage system according to the T[i]

As shown in FIG. 9 , it is a schematic diagram of an embodiment of the present invention. The schematic diagram shows T[1] confirmed by the storage system after the first expansion and the new data is written based on the T[1]. Example.

The right side of FIG. 9 is the T[1], indicating that 7 strips are included in the newly generated T[1], each strip contains 3 blocks, one of which is a check block, and each strip The distribution with the check block is shown in the figure. After the expansion is completed, if the controller receives a write request from the host, the write request carries new data to be written, if the The controller determines that the new data to be written is the data block D28, 0, the data block D28, 1, and further confirms its corresponding check block P28 based on the check algorithm, according to the T[1] on the right side of FIG. It is determined that the above data should be written to strip 0, that is, it needs to be written to the disks D1, D2, D4, and the writing order is to write the check block P28 and the data blocks D28, 0, D28, 1 to the disk D1 in turn, D2, D4, in the write operation is the default position to write each block to the disk with the smallest offset of the current offset, so the check block will write the offset of the disk D1 to the position of 6, the data block D28, 0 will write the position where the offset of the disk D2 is 3, and the data block D28, 1 will write the position of the offset written to the disk D4 to 0. Then the controller sequentially puts the data block D28,0. The data block D28, 1 and the check block P28 are respectively written to the corresponding positions of the disks D1, D2, and D4.

As shown in FIG. 10, it is a schematic structural diagram of a controller according to an embodiment of the present invention. The controller includes a first communication interface 1001, a processor 1002, a memory 1003, and a second communication interface 1004.

The first communication interface 1001 is configured to communicate with a host, and the controller may receive an operation instruction of the host, such as a read request, a write request, etc., through the first communication interface 1001, and hand it to the processor 1002. The first communication interface 1001 is further configured to send a message, such as a write success message, a write failure message, a read failure message, or a read data, to the host; the first communication interface 1001 may be a host bus. Adapter (Host Bus Adapter, HBA) card.

The processor 1002 may be a central processing unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated systems configured to implement the embodiments of the present invention. Circuit.

The controller may also include a memory 1003 that may be used to cache data carried in a write request received by the first communication interface 1001 from a host, or to cache data read from the disk. The memory 1003 can be a volatile memory, a non-volatile memory, or a combination thereof. The volatile memory is, for example, a random access memory (RAM). Non-volatile memory such as floppy disks, magnetic disks, solid state disks (SSDs), optical disks, and the like, and various machine readable media that can store program code or data. The memory 1003 may have a power-saving function, and the power-preserving function means that data stored in the memory 1003 is not lost when the system is powered off and re-powered. The program may be stored in the memory 1003, and the processor 1002 is configured to execute the program instructions to complete various processing actions of the controller. For details, refer to FIG. 2 to FIG. 9 and the controller in the corresponding embodiment. All processing actions will not be described in detail in the embodiment of the device. Optional. Further, the processor 1002 generates T[0], T[1], ..., T[t], and

Both may be stored in the memory 1003.

The second communication interface 1004 is configured to communicate with the disk, and the processor 1002 of the controller may send an operation command, such as a write request or a read request, to the second communication interface 1004 to The disk receives various types of messages from the disk.

Those skilled in the art will appreciate that various aspects of the embodiments of the invention, or possible implementations of various aspects, may be embodied as a system, method, or computer program product. Thus, aspects of the invention, or possible implementations of various aspects, may be in the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, etc.), or a combination of software and hardware aspects, They are collectively referred to herein as "circuits," "modules," or "systems." Furthermore, aspects of the embodiments of the invention, or possible implementations of various aspects, may take the form of a computer program product, which is a computer readable program code stored in a computer readable medium.

Computer readable media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing, such as random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM), optical disc.

The processor in the computer reads the computer readable program code stored in the computer readable medium such that the processor can perform the functional actions specified in each step or combination of steps in the flowchart.

The computer readable program code can execute entirely on the user's computer, partly on the user's computer, as a separate software package, partly on the user's computer and partly on the remote computer, or entirely on the remote computer or server. . It should also be noted that in some alternative implementations, the functions noted in the various steps in the flowcharts or in the blocks in the block diagrams may not occur in the order noted. For example, two steps, or two blocks, shown in succession may be executed substantially concurrently or the blocks may be executed in the reverse order.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. Different methods may be used to implement the described functionality for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of the invention should be determined by the scope of the appended claims.

Claims (12)

1. A method for data migration after a storage system is expanded, wherein the storage system includes a controller, and the storage system includes v1 to ₁ disk before the i-th expansion, and the storage system includes the i-th expansion. v _i disks, i is greater than or equal to 1; the controller is communicable with all disks; the data is distributed on the disk in the storage system based on a check block decentralization technique; the method includes:

   The controller confirms the number of blocks included in the migration unit of the i-th expansion;

   The storage space of each of the v _i-1 disks included in the storage system before the expansion is divided into a plurality of migration units, and each migration unit includes the foregoing number of blocks;

   Based on the number of disks v _i included in the storage system after expansion, and the number of disks included in the storage system before the capacity expansion, v _i-1 confirms the auxiliary balance incomplete block design required for data migration after the i-th expansion.

   

   Based on the auxiliary balance incomplete block design

   

   Selecting at least one migration unit from the plurality of migration units of the v _i-1 disks before the expansion, and migrating the blocks included in the selected at least one migration unit to the expanded v _i disks On one of the disks.
2. The method according to claim 1, wherein the data is distributed on the disk based on a check block dispersion technique in the storage system, specifically:

   The controller determines, according to the complete block design table, a save location of the data in the storage system, the complete block design table is generated based on a balanced incomplete block design, and the parameter of the balanced incomplete block design is (b, v, k, r, λ), where:

   v indicates that the storage system contains v disks;

   b indicates that the storage system includes b strips corresponding to b blocks of the balanced incomplete block design, and each element in each of the b blocks represents 1 block at The disk number of the disk in the storage system;

   k means that each stripe contains k blocks;

   r means that each disk contains r blocks;

   Any two of the v disks respectively contain λ identical strips;

   The block is a data block or a check block.
3. The method according to claim 2, wherein the method further comprises: generating a complete block design table T[i] for each of the expanded storage systems, wherein the T[i] is expanded at the ith time The complete block design table used to save the new data, where b _i , v _i , k _i , r _i , λ _i are the BIBD parameters constituting T[i].
4. The method according to claim 1, characterized in that, based on the number of disks v expansion after the storage system comprises a disk number _i v, and the expansion of the former storage system comprising a _i-1 of the i-th confirmation Auxiliary balance incomplete block design required for data migration after capacity expansion

   

   Specifically:

   Confirm the said

   

   Parameter

   

   The value of the value of v _i ,

   

   The value of the value is the value of the v _i-1 ;

   Based on the stated

   

   with

   

   Query balance incomplete block design database to confirm

   

   Value

   Based on the stated

   

   And said

   

   The value confirmation confirms the query of the incomplete block design database to confirm the composition

   

   Zone group

   Confirm the said

   

   Parameter

   

   with

   

   The value.
5. The method according to claim 4, wherein the controller confirms that the number of blocks included in the migration unit is specifically determined by the following formula: the number of blocks included in the migration unit:

   

   Where r _i-1 is a parameter designed to form a balanced incomplete block of T[i-1];

   

   For the parameters designed to form the equilibrium incomplete block of B[i-1], gcd is the operation taking the greatest common divisor.
6. The method of claim 1 wherein said designing an incomplete block based on said auxiliary balance

   

   Selecting at least one migration unit from the plurality of migration units of the v _i-1 disks before the expansion, and migrating the blocks included in the selected at least one migration unit to the expanded v _i disks The specific one of the disks is:

   The selected at least one migration unit is Rm,n, where m represents a migration unit number of the selected at least one migration unit, and n represents a disk number of the selected at least one migration unit;

   Obtaining a set S0, the set S0 being the

   

   The difference between the block m and the set {0,1,···, v _i-1 -1};

   Obtaining a set S1, the set S1 being a set {0, 1, . . . , v _i-1 -1}

   

   The difference set of the block m;

   If the value of n is the kth small element in the set S1, it is confirmed that the target disk number to which the migration unit Rm, n is to be migrated is an element whose value is the kth in the set S0;

   The migration unit Rm,n is migrated to a disk corresponding to the target disk number.
7. A storage system, wherein the storage system includes a controller, and the storage system includes v _i-1 disks before the i-th expansion, and the storage system includes v _i disks after the i-th expansion. Said i is greater than or equal to 1; the controller is communicable with all disks;

   The disk is configured to store data, where the data is distributed on the disk based on a check block dispersion technology;

   The controller is configured to confirm the number of blocks included in the migration unit of the i-th expansion; and divide the storage space of each of the v _-1 disks included in the storage system before the expansion into multiple migrations a unit, each of the migration units includes the above-mentioned number of blocks; based on the number of disks v _i included in the storage system after expansion, and the number of disks included in the storage system before the capacity expansion v _i-1 , the data migration after the i-th expansion is confirmed. Auxiliary balance incomplete block design required

   

   Based on the auxiliary balance incomplete block design

   

   Selecting at least one migration unit from the plurality of migration units of the v _i-1 disks before the expansion, and migrating the blocks included in the selected at least one migration unit to the expanded v _i disks On one of the disks.
8. The storage system according to claim 7, wherein the data is distributed on the disk based on a check block dispersion technique:

   The controller determines, according to the complete block design table, a save location of the data in the storage system, the complete block design table is generated based on a balanced incomplete block design, and the parameter of the balanced incomplete block design is (b, v, k, r, λ), where:

   v indicates that the storage system contains v disks;

   b indicates that the storage system includes b strips corresponding to b blocks of the balanced incomplete block design, and each element in each of the b blocks represents 1 block at The disk number of the disk in the storage system;

   k means that each stripe contains k blocks;

   r means that each disk contains r blocks;

   Any two of the v disks respectively contain λ identical strips;

   The block is a data block or a check block.
9. A storage system according to claim 8 wherein:

   The controller is further configured to generate a complete block design table T[i] for each of the expanded storage systems, where the T[i] is a complete block used to save new data after the ith expansion. A design table in which b _i , v _i , k _i , r _i , λ _i are BIBD parameters constituting T[i].
10. The storage system according to claim 7, wherein said disk controller for expansion after the number v the storage system comprises a number of disk _I v, and the expansion of the former storage system comprising a _i-1 based on the confirmation The auxiliary balance incomplete block design required for data migration after the i-th expansion

    

    Specifically:

    The controller is further configured to confirm the

    

    Parameter

    

    The value of the value of v _i ,

    

    The value of the value of v _i-1 ;

    

    Query balance incomplete block design database to confirm

    

    Value based on

    

    And said

    

    The value confirmation confirms the query of the incomplete block design database to confirm the composition

    

    Zone group; confirm the said

    

    Parameter

    

    with

    

    The value.
11. The storage system according to claim 10, wherein the controller is configured to confirm that the number of blocks included in the migration unit is specifically,

    The controller is further configured to confirm the number of blocks included in the migration unit by using the following formula:

    

    Where r _i-1 is a parameter designed to form a balanced incomplete block of T[i-1];

    

    For the parameters designed to form the equilibrium incomplete block of B[i-1], gcd is the operation taking the greatest common divisor.
12. The storage system according to claim 7, wherein said controller is configured to design an incomplete block based on said auxiliary balance

    

    Selecting at least one migration unit from the plurality of migration units of the v _i-1 disks before the expansion, and migrating the blocks included in the selected at least one migration unit to the expanded v _i disks The specific one of the disks is:

    The selected at least one migration unit is Rm,n, where m represents a migration unit number of the selected at least one migration unit, and n represents a disk number of the selected at least one migration unit;

    Obtaining a set S0, the set S0 being the

    

    The difference between the block m and the set {0,1,···, v _i-1 -1};

    Obtaining a set S1, the set S1 being a set {0, 1, . . . , v _i-1 -1}

    

    The difference set of the block m;

    If the value of n is the kth small element in the set S1, it is confirmed that the target disk number to which the migration unit Rm, n is to be migrated is an element whose value is the kth in the set S0;

    The migration unit Rm,n is migrated to a disk corresponding to the target disk number.

Images