WO2018028107A1

WO2018028107A1 - Coding fault-tolerant method for array-type storage system

Info

Publication number: WO2018028107A1
Application number: PCT/CN2016/110614
Authority: WO
Inventors: 唐聃; 舒红平; 王亚强
Original assignee: 成都信息工程大学
Priority date: 2016-08-07
Filing date: 2016-12-18
Publication date: 2018-02-15
Also published as: CN106254033B; CN106254033A

Abstract

The present invention belongs to the technical field of computer information storage, and specifically relates to a coding fault-tolerant method for any multi-node array-type storage system. The fault type targeted by the method is a node fault, namely, once any fault occurs at a certain node, it is considered that all data of the node is no longer reliable or is lost. The method is applicable to any multi-node array-type storage, can improve the reliability of a storage system, is applicable to the case where a company or an institution, etc. has a large volume of data and has high requirements regarding data stability, and can be widely applied to server systems.

Description

Code error tolerance method for array storage system

Technical field

The invention belongs to the technical field of computer information storage, in particular to a coding fault tolerance method for an array storage system of any multi-node.

Background technique

With the rapid growth of networks and servers, the capacity of data is increasing, and the importance and security of data is also being taken more seriously. In order to cope with the problem of data storage reliability caused by the rapid growth of data volume, and also to improve the concurrency efficiency and reduce the cost of data access, it is usually effective to use a plurality of storage nodes to jointly construct a storage system. It is usually a network-based distributed storage system, and its prototype can be traced back to a centralized RAID (Redundant Array of Inexpensive Disk) system. The introduction of RAID technology is to combine multiple small disks into a large-capacity logical hard disk according to certain conditions. In multiple hard disks, data and check are performed according to certain rules, such as striping, blocking, and interleaving. The data is processed in various combinations, and the reliability is improved by means of redundant data, so that the failure of a single hard disk does not affect the normal access of data, thereby ensuring the security of important data; and it is also desirable to use such a plurality of small disks to be fixed. The conditions are combined into a large-capacity logical disk to replace the expensive single-block large-capacity hard disk, thereby reducing data storage costs.

Although the stability of a single hardware device is already high, node failure events still occur frequently for distributed storage systems composed of many nodes. Especially in the case of a large amount of data, when the average failure probability of a single node is constant, the total number of nodes increases. It means that the number of nodes that a system may fail at the same time in any time period also increases; sudden behaviors such as hacking or operational errors may also cause multiple nodes in a storage system to fail at the same time. The common image-based backup-based reliability enhancement strategy improves the reliability of the storage system. However, it is obviously necessary to waste huge storage space. As the fault-tolerant capability increases, the storage efficiency decreases and the update cost increases.

Summary of the invention

The purpose of the present invention is to provide a coding and fault tolerance method for an array storage system of any multi-node, which can effectively improve fault tolerance.

To achieve the above objective, the specific technical solution of the coding and fault tolerance method of the array storage system of the present invention is that the array storage system is composed of n data nodes and k check nodes, and each node is in the same strip. Each bar contains m elements arranged vertically, m is a positive integer greater than 2, each element is represented by one bit, forming an array of m*(n+k), and the elements stored by the data node are arrays of data elements, The elements stored in the check node are arrays of check elements; the specific steps of the method are:

S1, the maximum number of fault tolerances required for the storage system is err, then err, m, n need to meet the following conditions: n>=m*err-err+1;

S2, two-dimensionally numbering each element in the data element array, that is, using D(i, j) to represent the i-th row and the j-th element of the data element array; for each element in the verification element array, the column priority is performed. One-dimensional numbering, that is, C(h) is used to represent the hth element in the array of check elements; based on the known data elements, the check element calculation of the err round is performed, n calculations are performed each round, and n checks are generated. The calculation formula for the element and the check element is:

In the above formula, x and y represent the yth code chain deployment of the xth round, 1<=x<=err,1<=y<=n, where j can be calculated by formula (2):

In the above formula (2), % is the modulo operation.

Take an integer for

All data elements and check elements involved in each calculation constitute a check chain;

S3. When there is an error of no more than err nodes, all elements on the node are called invalid elements, traverse all check chains, and look for a check chain with only one failed element, and different non-failed elements on the chain Or the operation, the result is the value of the invalid element, repeating the traversal check chain to calculate the invalid element until all the failed elements are retrieved.

The nodes include, but are not limited to, a PC, a server, or a disk.

The above method can be seen in step S3. The error type for the method is a node error, that is, once a node has an arbitrary error, the data of the node is considered to be no longer reliable or all lost. The method is applicable to any multi-node array. storage.

The invention provides a coding fault tolerance method of an array storage system, which can improve the reliability of the storage system, and is suitable for a large amount of data such as a company or an organization and has high requirements on data stability, and can be widely applied. In the server system.

DRAWINGS

The drawings are intended to provide a further understanding of the invention and are not intended to limit the invention.

FIG. 1 is a schematic diagram showing the logical structure of a multi-node storage system to which the present invention is applicable.

detailed description

The implementation of the present invention will be further described below in conjunction with the embodiments.

As shown in FIG. 1, an array storage system is composed of n data nodes and k check nodes, and each node may be, but not limited to, a PC, a server, or a disk; each node is divided into an equal number of The storage areas are called strips, and the corresponding strips on each node form strips; each strip can adopt different storage methods.

Let a stripe contain m elements, m is 3, each element is represented by one bit; and 3 elements are arranged vertically to form a 3*(n+k) array.

S1, the maximum number of fault tolerances required for the storage system is err, then err, m, n need to meet the following conditions: n>=m*err-err+1=3*3-3+1=7, taking n is 7; The size of the data element array is 3*7; assume that the values of the elements in the data element array are as follows:

11	11	00	00	11	00	11
00	11	00	11	00	00	00
11	11	11	00	11	11	11

S2, two-dimensionally numbering each element in the strip of the data node, that is, using D(i, j) to represent the i-th row and the j-th element of the data element array; for each element in the check element array, according to the column Priority is given to one-dimensional numbering, that is, C (h) is used to represent the h-th element in the check element array; 3 rounds of check element calculation is performed based on known data elements, and 7 calculations are performed each round, resulting in 7 The check element is calculated using the formula (1):

In the above formula, x and y represent the yth code chain deployment of the xth round, 1<=x<=err,1<=y<=n,

Is the XOR operation of D(i,j), where j can be calculated by formula (2):

In the above formula (2), % is the modulo operation.

Take an integer for

In detail, the calculation process of each check element value is as follows:

Round x=1: y = 1 calculation,

That is, C(1)=1, this calculation involves

All data elements and check elements form a check chain, that is, D(1,1), D(2,2), D(3,3), C(1) constitute a check chain;

The first y=2 calculations,

That is, C(2)=1, this calculation D(1,2), D(2,3), D(3,4), C(2) constitute a check chain;

The check elements C(3)=0+1+1=0 are successively obtained; C(4)=0+0+1=1; C(5)=1+0+1=0; C(6)= 0+0+1=1; C(7)=1+0+1=0 and the check chain.

Round x=2: y = 1 calculation

That is, C(8)=0, D(1,1), D(2,7), D(3,6), and C(8) involved in this calculation constitute a check chain;

The first y=2 calculations,

That is, C(9)=0, this calculation D(1,2), D(2,1), D(3,7), C(9) constitute a check chain;

The check element C(10)=0+1+1=0 is successively obtained; C(11)=0+0+1=1; C(12)=1+1+1=1; C(13)= 0+0+0=0; C(14)=1+0+1=0 and check chain;

Round x=3: y = 1 calculation

That is, C(15)=0, D(1,1), D(2,3), D(3,5), and C(15) involved in this calculation constitute a check chain;

The first y=2 calculations,

That is, C(16)=1, D(1,2), D(2,4), D(3,6), and C(16) involved in this calculation constitute a check chain;

The check elements C(17)=0+0+1=1 are successively obtained; C(18)=0+0+1=1; C(19)=1+0+1=0; C(20)= 0+0+1=1 and check chain;

Until the xth err=3 rounds y=n=7 calculations,

That is, C(21)=0, D(1,7), D(2,2), D(3,4), and C(21) involved in this calculation constitute one check chain.

The check element C(h) array is arranged in the first dimension by column first as follows:

C(1)C(1)	C(4)C(4)	C(7)C(7)	C(10)C(10)	C(13)C(13)	C(16)C(16)	C(19)C(19)
C(2)C(2)	C(5)C(5)	C(8)C(8)	C(11)C(11)	C(14)C(14)	C(17)C(17)	C(20)C(20)
C(3)C(3)	C(6)C(6)	C(9)C(9)	C(12)C(12)	C(15)C(15)	C(18)C(18)	C(21)C(21)

which is:

11	11	00	11	00
11	00	11	11	11
00	11	11	11	00

The final storage array is as follows:

11	11	00	00	11	00	11	11	11	00	11	00
00	11	00	11	00	00	00	11	00	11	11	11
11	11	11	00	11	11	11	00	11	11	11	00

S3. When there is an error of not more than err nodes, all the elements on the node are called invalid elements; in this embodiment, it is assumed that three nodes have errors, such as errors occur in the first, third, and ninth nodes, The data on columns 3 and 9 are all lost, that is, the elements D(1,1), D(2,1), D(3,1), the elements D(1,3), D(2,3), D( 3, 3), elements C (4), C (5), C (6) are invalid, as shown in the following table:

xx	11	xx	00	11	00	11	11	xx	00	11	00
xx	11	xx	11	00	00	00	11	xx	11	11	11
xx	11	xx	00	11	11	11	00	xx	11	11	00

Data recovery: traverse all check chains, look for a check chain with only one failed element, and XOR the non-failed elements on the chain. The result is the value of the failed element. Repeat the traversal check chain to calculate the invalid element. Until all the data is retrieved, the process is as follows:

First find the check chain D(1,3), D(2,4), D(3,5), C(3) with only one failed element, then D(1,3)=1^1^0 =0; Assignment is shown in the following table:

xx	11	00	00	11	00	11	11	xx	00	11	00
xx	11	xx	11	00	00	00	11	xx	11	11	11
xx	11	xx	00	11	11	11	00	xx	11	11	00

Repeat the traversal check chain to find the check chain D(1,2), D(2,3), D(3,4), C(2) with only one failed element, then D(2,3)=1 ^0^1=1;

Continue to repeat the traversal check chain to find the check chain D(1,1), D(2,7), D(3,6), C(8) with only one failed element, then D(1,1)= 0^1^0=1;

Continue to repeat the traversal check chain to calculate the failed elements D(3,3)=1, D(2,1)=0, D(3,1)=1, C(4)=1, C(5)=0. , C(6)=1, and finally all the failed data is completely recovered.

The invention can recover all the failed elements when there is an error of not more than err nodes, thereby effectively improving the reliability of the storage system.

The present invention has been described in connection with the embodiments of the present invention, and it is obvious that the invention is not limited by the above-described manner, as long as various insubstantial improvements made by the method concept and technical solution of the present invention are adopted, or the present invention is not improved. The concept and technical solution of the invention are directly applicable to other occasions, and are all within the scope of the invention.

Claims

An encoding fault tolerance method for an array storage system, wherein the array storage system is composed of n data nodes and k check nodes; each of the same strips in each node includes m elements arranged longitudinally, m For a positive integer greater than 2, each element is represented by one bit, forming an array of m*(n+k), the elements stored by the data node are arrays of data elements, and the elements stored by the check node are arrays of check elements; The feature is that the specific steps of the method are:

S1, the maximum number of fault tolerances required for the storage system is err, then err, m, n need to meet the following conditions: n>=m*err-err+1;

S2, two-dimensionally numbering each element in the data element array, that is, using D(i, j) to represent the i-th row and the j-th element of the data element array; for each element in the verification element array, the column priority is performed. One-dimensional numbering, that is, C(h) is used to represent the hth element in the array of check elements; based on the known data elements, the check element calculation of the err round is performed, n calculations are performed each round, and n checks are generated. The calculation formula for the element and the check element is:

In the above formula, x and y represent the yth code chain deployment of the xth round, 1<=x<=err,1<=y<=n,
Is the XOR operation of D(i,j), where j can be calculated by formula (2):

In the above formula (2), % is the modulo operation.
Take an integer for

All data elements and check elements involved in each calculation constitute a check chain;

S3. When there is an error of no more than err nodes, all elements on the node are called invalid elements, traverse all check chains, and look for a check chain with only one failed element, and different non-failed elements on the chain Or operation, the result is the value of the invalid element, repeated Traverse the check chain to calculate the invalidation element until all data is retrieved.
The method of encoding fault tolerance of an arrayed storage system according to claim 1, wherein the nodes include, but are not limited to, a PC, a server, or a disk.