WO2018166078A1

WO2018166078A1 - Mds array code encoding and decoding method for repairing failure of multiple nodes

Info

Publication number: WO2018166078A1
Application number: PCT/CN2017/087770
Authority: WO
Inventors: 侯韩旭; 韩永祥; 周清峰
Original assignee: 东莞理工学院
Priority date: 2017-03-16
Filing date: 2017-06-09
Publication date: 2018-09-20
Also published as: CN107086870B; CN107086870A

Abstract

The encoding of an MDS array code for repairing a failure of multiple nodes, a component thereof being a C(k,r,p) code. An original information data block and a redundant block are stored by building a (p - 1) * (k + r) matrix, wherein p is a prime number, and p is greater than k and r, and k and r are any integer less than p and greater than 0; column k is called an information column, and corresponds to k data blocks; and column r is a redundant column, and corresponds to r redundant blocks, and an addition and subtraction operation in the C(k,r,p) code are both XOR operations. The beneficial effects of the coding are: being able to repair a new Cauchy array code with any n to k nodes having failed and a low encoding and decoding calculation complexity, thereby improving the fault tolerance of a system. The encoding and decoding of a new Cauchy array code is realised by binary XOR operation, and compared with a CRS code, the calculation complexity of the encoding and decoding process is lower.

Description

MDS array code encoding and decoding method for repairing multi-node failure

[Technical Field]

The present invention relates to the field of data processing, and in particular, to an MDS array code encoding and decoding method for repairing multi-node failure.

【Background technique】

With the rapid development of computer network applications, the amount of network information data has become larger and larger, and mass information storage has become particularly important. The continuous growth of data storage pressure has driven the rapid development of the entire storage market. Distributed storage has become the mainstream technology of today's big data storage with its superior features such as high cost performance, low initial investment, and pay-as-you-go.

Currently, storage node failure of distributed storage systems has become a normal state. When the storage nodes deployed by the system become unreliable, redundancy must be introduced to improve the reliability of the node failure. The easiest way to introduce redundancy is to directly back up the original data. Although the direct backup is simple, its storage efficiency and system reliability are not high. The method of introducing redundancy through coding can improve its storage efficiency and enhance the reliability of the system. Therefore, the high probability of availability, reliability, and security of distributed storage are key technical issues for distributed storage systems.

In the current storage system, the encoding method generally adopts the MDS code, and the MDS code can achieve the optimization of the storage space efficiency. One (n, k) MDS code needs to divide a raw data file into k equal-sized data blocks through linear The encoding generates n mutually uncorrelated coding blocks, and n coding blocks are stored by n nodes. If n coding blocks contain the original k data blocks, the other n-k coding blocks are referred to as redundant blocks, and the MDS code including k data blocks is referred to as a system MDS code. The MDS code satisfies the MDS attribute: that is, taking any k of n code blocks can decode the original k data blocks. This coding technology plays an important role in providing effective network storage coding, and is particularly suitable for storing large files and archival data backup.

In a distributed storage system, data is encoded in some way and the result of the encoding is stored in n storage nodes. This process is called an encoding process. The data receiver only needs to connect and download the data of any k storage nodes of the n storage nodes to recover the original data. This process is called a data reconstruction process or a decoding process.

Different MDS codes have different coding and decoding computational complexity. The higher the complexity, the larger the amount of calculation, and the longer the calculation takes. Design a good MDS code, which can reduce the amount of calculation, shorten the working time, reduce the consumption of resources, save the cost of the system operation, and make the operation and storage More flexible. The MDS array code is a kind of MDS code, which is characterized in that only a simple binary exclusive OR operation is used in the encoding and decoding process, so it can be easily and efficiently implemented in the system. The invention is a binary system MDS array code capable of accommodating loss, encoding and decoding computational complexity of any n-k coding blocks.

EVENODD code, cited in the paper [M.Blaum, J. Brady, J. Bruck, and J. Menon, "EVENODD: An efficient scheme for tolerating double disk failures in RAID architectures," IEEE Transactions on Computers, vol.44, no .2, pp. 192–202, 1995].

The EVENODD code is an MDS array code that can accommodate 2 coded block losses. In the EVENODD code, it is required that the number of data blocks must be a prime number and the number of redundant blocks is two. The encoding process of the EVENODD code requires only a simple binary exclusive OR operation, and each redundant bit of the two check blocks is an information bit XOR result that passes through a straight line with a slope of 0 and 1. However, the EVENODD code has the drawback of being non-expandable: the EVENODD code has only two check blocks, so it can only recover the failure of two storage nodes at most, which is not easy to expand. Literature [M. Blaum, J. Bruck, and A. Vardy. MDS array codes with independent parity symbols. In IEEE Transactions on Information Theory, vol. 42, no. 2, pp. 529-542, 1996] gives EVENODD The code is extended, but the conditions for satisfying the MDS attribute are very severe, and the number of redundant blocks (nk) satisfying the MDS attribute is at most 8.

The CRS code, the full name is the Cauchy Reed-Solomon code, cited in the paper [Plank, JS, Xu, L. Optimizing Cauchy Reed-Solomon codes for fault-tolerant network storage applications. In IEEE International Symposium on Network Computing and Applications, pp. 173 –180, 2006]. It is an MDS system array code based on finite field and Cauchy matrix construction, which can accommodate the loss of any n-k code blocks. In the encoding and decoding process, it converts a finite field multiplication operation into multiple binary exclusive OR operations. However, the CRS code still has the disadvantage of high computational and decoding computational complexity: in the process of encoding and decoding, the CRS code converts the finite field multiplication operation into multiple binary exclusive OR operations, because the number of binary exclusive OR operations is reflected. Its codec computational complexity, however, the number of its corresponding binary XOR operations is uncontrollable and very high during the encoding and decoding process. For the general parameters k and n, how to design its codec calculation method and reduce the number of binary XOR, that is, the complexity of codec calculation, is a major defect that CRS code urgently needs to solve.

[Summary of the Invention]

In order to solve the problems in the prior art, the present invention provides a method for repairing and decoding a multi-node failure MDS array code, which solves the problem of high computational complexity of coding and decoding in the prior art.

The invention is implemented by the following technical solutions: designing and manufacturing a mds array code code for repairing multi-node failure, the component of which is a C(k, r, p) code, by constructing a (p-1)×( a matrix of k + r) to store the original information data block and the redundant block, where p is a prime number, and p is greater than k and r, k and r are smaller than any integer less than p greater than 0; k columns are called information columns, Corresponding to k data blocks; r columns are redundant columns, which correspond to r redundant blocks, and the addition and subtraction operations in the C(k, r, p) code are XOR operations.

As a further improvement of the present invention, the C (k, r, p) code is constructed by: k (p-1) information bits are given, k check bits are calculated and added, and k data polynomials are obtained, which are calculated. The coding polynomial is stored, and the number of times the polynomial is stored is a coefficient of a polynomial from 0 to p-2.

As a further improvement of the present invention, the p-1 information bits stored in the jth information column and the first data polynomial represented by the check bits thereof are s _j (x)=s _0,j +s _1,j x+...+ s _p-2,j x ^p-2 +s _p-1,j x ^p-1 , wherein s _i,j represents the i-th bit of the j-th information column of the array code C(k,r,p); The third coding polynomial is

As a further improvement of the present invention, the division in the C(k, r, p) code is performed by calculating a division operation for each data polynomial and then adding the results of the corresponding data polynomial division operations.

As a further improvement of the present invention, the C(k, r, p) code is a binary Cauchy array code whose generating matrix is G = [I C], wherein the matrix I represents a unit matrix of k × k, and the matrix C is k ×r Cauchy matrix,

The invention also provides a decoding method for repairing multi-node failure mds array code coding, comprising the following steps: when ρ≤r columns are lost in the array code C(k, r, p), γ information columns are set a ₁ , a ₂ ,..., a _γ and δ redundant columns b ₁ , b ₂ , . . . , b _{δ are} lost, where 1≤a ₁ <a ₂ <...<a _γ ≤k, ₁ ≤ b ₁ <b ₂ <...<b _γ ≤ r, k ≥ γ ≥ 0, r ≥ δ ≥ 0 and γ + δ = ρ ≤ r, and the column of information that is not lost is represented as M = { 1,2,...,k}\{a ₁ ,a ₂ ,...,a _γ }, denote redundant columns without loss as P={1,2,...,r}\{ b ₁ , b ₂ ,...,b _δ }, first download the surviving k-γ information columns i ₁ , i ₂ , . . . , i _k-γ ∈M, and γ redundant columns l ₁ , l ₂ ,...,l _γ ∈P, recovers the lost γ information columns, and then multiplies the coded vector of the missing redundant column by k data polynomials to calculate the missing redundant polynomial.

As a further improvement of the invention: adding a check bit to the information column i _τ and forming a data polynomial

Where τ=1,2,...,k-γ; adding a bit zero to the redundant column l _h and forming a coding polynomial

Where h = 1, 2, ..., γ; the polynomial obtained by subtracting k-γ data polynomials from γ coding polynomials is

Solving missing γ data polynomials by solving linear equations, where the linear equations are

; then get γ missing columns of information.

As a further improvement of the invention: the missing δ redundant columns are obtained by multiplying the corresponding coding vectors by k information polynomials.

The invention has the beneficial effects that the new Cauchy array code with any n-k node failures and low coding and decoding computation complexity can be repaired, and the fault tolerance of the system is improved. The encoding and decoding of the new Cauchy array code is realized by binary exclusive OR operation. Compared with the CRS code, the coding and decoding process has lower computational complexity.

[Description of the Drawings]

1 is a schematic diagram showing the computational complexity of the unit coding of the array code C(p-4, 4, p), the cyclic Cauchy code, and the CRS code of the present invention at r=4;

2 is a schematic diagram showing the computational complexity of unit decoding of the array code C(p-4, 4, p) and the Rabin-Like code, the cyclic Cauchy code, and the CRS code of the present invention at r=4;

FIG. 3 is a schematic diagram showing the computational complexity of unit decoding of the array code C(p-5, 5, p), the cyclic Cauchy code, and the CRS code of the present invention at r=5.

【detailed description】

The invention will now be further described with reference to the drawings and specific embodiments.

Abbreviations and definitions of key terms

MDS Maximum Distance Separable Maximum Distance Separable

RDP Row-Diagonal Parity line diagonal check

A mds array code code for repairing multi-node failures, the component of which is a C(k, r, p) code, which is constructed by constructing a matrix of (p-1)×(k+r) to store original information data blocks and redundancy. a remainder block, where p is a prime number, and p is greater than k and r, k and r are less than any integer less than p greater than 0; k columns are referred to as information columns, which correspond to k data blocks; r columns are redundant columns, It corresponds to r redundant blocks, and the addition and subtraction operations in the C(k, r, p) code are all exclusive OR operations.

The C(k, r, p) code is constructed by: k (p-1) information bits are given, k check bits are calculated and added, k data polynomials are obtained, and the polynomial is calculated by calculating the coding polynomial. The number of coefficients is a coefficient of a polynomial from 0 to p-2.

The p-1 information bits stored in the jth information column and the first data polynomial represented by the check bits thereof are s _j (x)=s _0,j +s _1,j x+...+s _p-2,j x ^P-2 +s _p-1,j x ^p-1 , where s _i,j represents the i-th bit of the j-th information column of the array code C(k,r,p); the third coding polynomial is

The division in the C(k, r, p) code is performed as follows: a division operation is calculated for each data polynomial, and then the results of the corresponding data polynomial division operations are added.

The C(k,r,p) code is a binary Cauchy array code whose generating matrix is G=[I C], where matrix I represents a unit matrix of k×k, and matrix C is a k×r Cauchy matrix.

The invention also provides a decoding method for repairing multi-node failure MDS array code coding, comprising the following steps: when ρ≤r columns are lost in the array code C(k,r,p), γ information columns are set a ₁ , a ₂ ,..., a _γ and δ redundant columns b ₁ , b ₂ , . . . , b _{δ are} lost, where 1≤a ₁ <a ₂ <...<a _γ ≤k, ₁ ≤ b ₁ <b ₂ <...<b _γ ≤ r, k ≥ γ ≥ 0, r ≥ δ ≥ 0 and γ + δ = ρ ≤ r, and the column of information that is not lost is represented as M = { 1,2,...,k}\{a ₁ ,a ₂ ,...,a _γ }, denote redundant columns without loss as P={1,2,...,r}\{ b ₁ , b ₂ ,...,b _δ }, first download the surviving k-γ information columns i ₁ , i ₂ , . . . , i _k-γ ∈M, and γ redundant columns l ₁ , l ₂ ,...,l _γ ∈P, recovers the lost γ information columns, and then multiplies the coded vector of the missing redundant column by k data polynomials to calculate the missing redundant polynomial.

Add a check bit to the information column i _τ and form a data polynomial

Solving the missing γ data polynomials by solving linear equations, where the linear equations are

Then γ missing columns of information are obtained.

The lost δ redundant columns are obtained by multiplying the corresponding coding vectors by k information polynomials.

In the present invention, the new MDS array code based on the binary polynomial ring and the Cauchy matrix structure generally solves the problem that the existing MDS code accommodates a small number of storage node failures and high computational and decoding computational complexity, and proposes a repair. The new Cauchy array code with any nk nodes failing and the codec calculation complexity is low, which improves the fault tolerance of the system. The codec of the new Cauchy array code is realized by binary exclusive OR operation, and the computational complexity of the codec process is lower than that of the CRS code.

Referring to this new Cauchy array code is a C (k, r, p) code, all the addition and subtraction operations herein refer to a binary exclusive OR operation, where r = n - k. C(k,r,p) code by constructing a A matrix of (p-1) x (k + r) stores original information data blocks and redundant blocks, where p is a prime number, and p is greater than k and r, and k and r are smaller than any integer less than p greater than zero. The first k columns of the C(k, r, p) code are called information columns, corresponding to k data blocks, and the latter r columns are redundant columns, corresponding to r redundant blocks.

In an embodiment, the C(k, r, p) code is constructed as follows: let i=0,1,...,p-2,j=1,2,...,k, with s _{i , j} denotes the i-th bit of the j-th information column of the array code C(k, r, p). Let i=0,1,...,p-2,l=1,2,...,r, denote the first redundant column of the array code C(k,r,p) by c _i,j The ith bit. For the p-1 information bits s _0,j , s _1,j ,...,s _p-2,j stored in the jth information column, the check bits s _p-1,j are defined as

s _p-1,j =s _0,j +s _1,j +...+s _p-2,j .

Let j=1, 2,...,k, represent p-1 information bits stored in the jth information column and its check bits as data polynomial

s _j (x)=s _0,j +s _1,j x+...+s _p-2,j x ^p-2 +s _p-1,j x ^p-1 . (1)

Similarly, let l=1, 2,...,r, define the parity bit c _p-1,l of the redundant column _l as

c _p-1,l =c _0,l +c _1,l +...+c _p-2,l ,

And p-1 redundant bits stored in the 1st redundant column and their check bits are expressed as coding polynomials

c _l (x)=c _0,l +c _1,l x+...+c _p-2,l x ^p-2 +c _p-1,l x ^p-1 . (2)

Therefore, there are now k data polynomials and r coding polynomials in the polynomial ring F ₂ [x]/(1+x ^p ). And r coding polynomials are calculated by the following equation

Note that the above calculations are all operations in the polynomial loop F ₂ [x]/(1+x ^p ). Therefore, the generator matrix of the array code C(k, r, p) is

G=[I C],

Where matrix I represents the unit matrix of k × k, matrix C is the k × r Cauchy matrix

The encoding process of the binary Cauchy array code C(k, r, p) can be described as follows. Given k(p-1) information bits, k check bits are calculated and added, and k data polynomials as shown in equation (1) are obtained. Pass The coding polynomial shown in equation (3) is calculated, the term of the number of times p-1 among these polynomials is ignored, and then the number of times the polynomial is stored is a coefficient of a polynomial from 0 to p-2.

In the encoding process of the array code C(k, r, p), it is necessary to calculate many shapes in the polynomial ring F ₂ [x] / (1 + x ^p )

The division operation, in which the polynomial s(x) has an even number of terms, and b is a positive integer smaller than the prime number p. How to effectively calculate this division operation is given in the following lemma.

Lemma 1. In the polynomial ring F ₂ [x]/(1+x ^p ), the polynomial s(x)=s ₀ +s ₁ x+...+s _p-1 x ^p-1 has an even number of terms, given Positive integer b and equation

Where b is a positive integer less than the prime number p. Then, the coefficient of the polynomial c(x)=c ₀ +c ₁ x+...+c _p-1 x ^p-1 can be calculated by the following equation.

c _p-1 =0, c _pb-1 = s _p-1 , c _p-2b-1 = s _p-2b-1 + c _pb-1 , c _p-3b-1 = s _p-3b-1 + c _p-2b-1 ,...,

.

c _p-(p-3)b-1 =s _p-(p-3)b-1 +c _p-(p-2)b-1 ,c _p-(p-2)b-1 =s _{p -(p-2)b-1} +c _p-(p-1)b-1 ,c _b-1 =s _b-1

Proof: by expanding the equation s(x)=c(x)(1+x ^b ),

s ₀ =c ₀ +c _pb

s ₁ =c ₁ +c _p-b+1

s ₂ =c ₂ +c _p-b+2

.

·

s _p-2 =c _p-2 +c _pb-2

s _p-1 =c _p-1 +c _pb-1

A coefficient of the polynomial c(x) is selected to be 0, and then other coefficients of the polynomial c(x) can be calculated by the above equation iteration. Specifically, let the coefficient c _{p-1 be} equal to 0, and then calculate

c _pb-1 =s _p-1

with

c _b-1 = s _b-1 .

The other coefficients of the polynomial c(x) can be calculated by an XOR operation iteration. The certificate is completed.

It is known from the proof process of Lemma 1 that p-3 exclusive ORs are required in the process of calculating the coefficients of the polynomial c(x).

Taking the parameters k=2, r=2, p=5 as an example, the encoding process of Cauchy array code C(2, 2, 5) is introduced.

In this example, there are two data polynomials, as shown in equations (5) and (6).

s ₁ (x)=s _0,1 +s _1,1 x+s _2,1 x ² +s _3,1 x ³ +(s _0,1 +s _1,1 +s _2,1 +s _{3, 1} )x ⁴ (5)

s ₂ (x)=s _0,2 +s _1,2 x+s _2,2 x ² +s _3,2 x ³ +(s _0,2 +s _1,2 +s _2,2 +s _{3, 2} )x ⁴ (6)

The two coding polynomials c ₁ (x) and c ₂ (x) are calculated by the equation shown by the following formula (7).

According to Lemma 1, in the above calculation of the coding polynomial, each division operation involves 2 exclusive OR operations. Table 1 lists the information column bits and redundant column bits for this example. In calculating the coding polynomial of a binary Cauchy array code, it is first necessary to calculate a division operation for each data polynomial, and then add the results of the corresponding data polynomial division operations.

Table 1 Information column and redundancy column of array code C (2, 2, 5)

信息列1 Information column 1	信息列2Information column 2	冗余列1 Redundant column 1	冗余列2Redundant column 2
s_0,1 s _0,1	s_0,2 s _0,2	(s_0,1+s_1,1+s_3,1)+(s_0,2+s_2,2)(s _0,1 +s _1,1 +s _3,1 )+(s _0,2 +s _2,2 )	(s_0,1+s_1,1+s_3,1)+(s_0,2+s_3,2)(s _0,1 +s _1,1 +s _3,1 )+(s _0,2 +s _3,2 )
s_1,1 s _1,1	s_1,2 s _1,2	s_1,1+(s_0,2+s_1,2+s_2,2+s_3,2)s _1,1 +(s _0,2 +s _1,2 +s _2,2 +s _3,2 )	s_1,1+s_2,2 s _1,1 +s _2,2
s_2,1 s _2,1	s_2,2 s _2,2	(s_0,1+s_1,1+s_2,1+s_3,1)+s_2,2 (s _0,1 +s _1,1 +s _2,1 +s _3,1 )+s _2,2	(s_0,1+s_1,1+s_2,1+s_3,1)+s_0,2 (s _0,1 +s _1,1 +s _2,1 +s _3,1 )+s _0,2
s₃₁ s ₃₁	s₃₂ s ₃₂	(s₁₁+s₃₁)+(s₀₂+s₂₂+s₃₂)(s ₁₁ +s ₃₁ )+(s ₀₂ +s ₂₂ +s ₃₂ )	(s₁₁+s₃₁)+(s₀₂+s₁₂+s₃₂)(s ₁₁ +s ₃₁ )+(s ₀₂ +s ₁₂ +s ₃₂ )

In yet another embodiment, the decoding process of the array code C(k, r, p) is as follows:

When some of the data columns of the binary Cauchy array code are lost, it is desirable to download data from other surviving data columns and decode the missing data. This process is called the decoding process. This section gives a fast decoding method for the array code C(k, r, p), which is based on the LU decomposition of the Cauchy matrix. This decoding method is suitable for the loss of any data column. First, let's introduce the LU decomposition of the Cauchy matrix.

Given 2k mutually unequal variables x ₁ , x ₂ , . . . , x _k , y ₁ , y ₂ , . . . , y _k , the Western kernel is defined as shown in equation (8).

[T.Boros, T.Kailath and V.Olshevsky.A fast parallel bjorck-pereyra-type algorithm for solving cauchy linear equations. Elsevier Linear Algebra and Its Applications, vol. 1999, 302(1): 265-293.] The inverse matrix of the Western matrix is given and the LU decomposition of the inverse matrix of Cauchy matrix is proposed. The main conclusions are as follows.

Theorem 2. The inverse matrix of the Western kernel C (x _{1: k} , y _{1: k} ) can be decomposed into the equation (9).

Wherein formula (10) and formula (11) are satisfied.

Where i = 1, 2, ..., k-1, and (12) are satisfied.

D _k =diag{(x ₁ -y ₁ ) (x ₂ -y ₂ ) ... (x _k -y _k )} (12)

For example, when k=2, the Cauchy inverse matrix C(x _1:2 , y _1:2 ) ^-1 can be decomposed into the formula (13).

Based on the conclusion of Theorem 2, a method for quickly solving linear equations in the form of Cauchy matrix is proposed. Given a linear equation system C(x _1:k , y _1:k )z=b in the form of a k × k Cauchy matrix, where z = (z ₁ , z ₂ ,..., z _k ) ^t and b =(b ₁ , b ₂ ,...,b _k ) ^t are all column vectors of length k. When the values of the vector b = (b ₁ , b ₂ , ..., b _k ) ^t and the matrix C (x _{1: k} , y _{1: k} ) are given, the vector z = (z ₁ , z can be solved _{The value of 2} ,...,z _k ) ^t is as shown in equation (14).

See pseudo-code for the algorithm.

In Algorithm 1, three operations need to be processed: (i) the product of the polynomial b _l and x _i , (ii) the division operation b _l /(x _j -y _ji ), (iii) the addition of the polynomial b _i +b _j . It is possible to statistically calculate a total of 5k(k-1) times of the first type of multiplication in algorithm 1, k(k-1) times of the second type of division operation, and k+3k(k-1) times of the third type of addition. In the decoding process corresponding to the array code C(k, r, p), both x _i and y _i in Algorithm 1 are replaced by the power of x, and the variables b _l and then are polynomial rings F ₂ [x]/( A polynomial in 1+x ^p ) is substituted. All type 2 division operations in Algorithm 1 can be calculated by the method given in Lemma 1, and a division operation involves a p-3 XOR operation. For the first type of multiplication, in the polynomial ring F ₂ [x] / (1 + x ^p ), multiplying a polynomial by x ⁱ is the cyclically shifted i-bit of this polynomial, so no XOR operation is involved. In the polynomial ring F ₂ [x]/(1+x ^p ), the addition of two polynomials uses a p-exclusive OR operation.

In Algorithm 1, steps 5 through 7 are computation matrices

The product of the right matrix and the column vector b has a total of 3 (ki) polynomial addition operations, and the computational complexity is at most 3p (ki) XOR. Steps 8 through 10 are calculation matrices

The product of the left matrix and the column vector b has a computational complexity of at most 3k(k-1)(p-3)/2 times XOR. Therefore, steps 5 to 10, calculate the matrix

The computational complexity of the product of the column vector b is at most 3pk(k-1)/2+3k(k-1)(p-3)/2 times XOR. Steps 12 through 14 are the product of calculating the diagonal matrix D _k and the polynomial vector obtained in the above step, and the computational complexity is at most p + (k - 1) (p - 2). Steps 16 through 18 are calculation matrices

The product of the right matrix and the polynomial vector obtained in the above step has a computational complexity of at most (k-3)k(k-1)/2 times XOR. Steps 19 through 26 are calculation matrices

The product of the left matrix and the polynomial vector obtained in the above step has a computational complexity of at most k(k-1)(3p-4)/2 times XOR. Therefore, in the polynomial ring F ₂ [x]/(1+x ^p ), the computational complexity of Algorithm 1 is at most as shown in Equation (15).

Considering the example given above, the two coding polynomials calculate the equation shown by equation (16) by the following equation.

Assume that the two data polynomials are s ₁ (x) = 1 + x and s ₂ (x) = x + x ^{3 , respectively} . Then, it can be concluded that its two coding polynomials are c ₁ (x)=x and c ₂ (x)=x+x ² +x ^{3 , respectively} .

According to Theorem 1, this 2 × 2 Cauchy matrix can be decomposed into the equation (17).

It can be verified that two data polynomials can be calculated by the following equation from two coding polynomials (18).

In the process of solving two coding polynomials, there are 32 XOR operations.

When ρ ≤ r columns are lost in the array code C(k, r, p), the decoding process of the array code will be described next. Suppose that γ information columns a ₁ , a ₂ , . . . , a _γ and δ redundant columns b ₁ , b ₂ , . . . , b _{δ are} lost, where 1≤a ₁ <a ₂ <...< a _γ ≤ k, ₁ ≤ b ₁ < b ₂ <... < b _γ ≤ r, k ≥ γ ≥ 0, r ≥ δ ≥ 0, and γ + δ = ρ ≤ r. The list of information that has not been lost is represented by equation (19).

M = {1, 2, ..., k}\{a ₁ , a ₂ , ..., a _γ } Equation (19) represents a redundant column that is not lost as Equation (20).

P={1,2,...,r}\{b ₁ ,b ₂ ,...,b _δ } (20)

First download the surviving k-γ information columns i ₁ , i ₂ , . . . , i _k-γ ∈M, and γ redundant columns l ₁ , l ₂ , . . . , l _γ ∈P The lost γ information columns are recovered, and then the coded vector of the missing redundant column is multiplied by k data polynomials to calculate the missing redundant polynomial. The decoding process is described in detail below.

For τ = 1, 2, ..., k- γ, a parity bit is first added to the information column i _τ and a data polynomial is formed as in equation (21).

For h=1, 2,..., γ, add bit zero to the redundancy column l _h and form a coding polynomial such as equation (22)

Order (23)

For the polynomial obtained by subtracting k-γ data polynomials for γ coding polynomials, respectively, there is equation (24).

Where h = 1, 2, ..., γ. The missing γ data polynomials can then be solved by solving the following system of linear equations. As shown in equation (25).

By calling Algorithm 1, γ missing columns of information can be obtained. Then, the missing δ redundant columns can be obtained by multiplying the corresponding coding vectors by k information polynomials.

Let's analyze the complexity of codec.

The decoding complexity of the binary Cauchy array code is counted below. Adding a check bit to the k-γ information columns and forming a data polynomial uses a (k-γ) (p-2) sub-OR operation. The γ polynomial represented by the equation (22) requires γ((k-γ)(p-3)+(k-γ)(p-1))=γ(k-γ)(2p-4) times. XOR operation. The 4γ ² p-3γp-5γ ² +3γ+2 XOR operation was used in solving the linear equations of the γ×γ Cauchy form. A δ(k(p-3)+(k-1)(p-1)) XOR operation is required to recover δ redundant columns. Therefore, the decoding computational complexity D _{γ, δ} of the γ information columns and the δ redundant columns is decoded as Equation (26).

Sub-OR operation. When there is no redundant column loss, only the information column is lost, and the decoding complexity is the sub-OR operation shown in equation (27).

D _γ = (k - γ) (p - 2) + γ (k - γ) (2p - 4) + 4 γ ² p - 3 γp - 5 γ ² + 3 γ + 2 Formula (27).

For convenience of comparison, the unit coding complexity is defined as the number of XOR operations involved in the encoding process divided by the number of information bits, and the unit decoding complexity is the number of XOR operations involved in the decoding process divided by the information. The number of bits.

Compared with the previous coding scheme, such as the EVENODD code, the codec complexity of the array code C(k, r, p) is almost unchanged, but it can greatly improve the fault tolerance of the system, and can repair any r node failures at most; Compared with the CRS code, the array code C(k, r, p) can recover multiple nodes at the same time, and the codec complexity is very low. Secondly, the C(k, r, p) code does not fix the number of original information data blocks, and the value is more flexible. Moreover, the construction process and reconstruction process of C(k,r,p) code only involve XOR operation, so the computational complexity is low and the computational overhead is small, which greatly reduces the system calculation delay, saves time and Resources, can reduce the cost of consumption, suitable for the actual storage system; C (k, r, p) code can meet the MDS attributes, while saving storage space, the system can accommodate multiple node failures, increasing the data Fault tolerance and stability.

Encoding calculation complexity:

In the array code C(k, r, p), there are k information columns and r redundant columns. First, a parity bit is formed for each information column to form a data polynomial, which requires a k(p-2) XOR operation. Then, r coding polynomials are calculated according to the coding calculation formula. In the process of calculating each coding polynomial, k division operations need to be calculated, and according to Lemma 8, a k(p-3) exclusive OR operation is required. Further, according to the conclusion of Lemma 1, the coefficient of the polynomial obtained by the division operation is p-1 and the coefficient is 0. Therefore, in the k-1 addition in the process of calculating each coding polynomial, a (k-1) (p-1) XOR operation is required. Therefore, the number of exclusive ORs E ₁ required to calculate a coding polynomial is as shown in equation (28).

E ₁ =k(p-3)+(k-1)(p-1) (28)

The calculation amount E _r required to calculate r redundant columns is as shown in equation (29).

E _r =k(p-2)+r(2kp-4k-p+1) (29)

The unit coding computational complexity NE _{C(k, r, p) of the} binary Cauchy array code C(k, r, p) is expressed by equation (30).

Circulant Cauchy array code [C. Schindelhauer and C. Ortolf. Maximum distance separable codes based on circulant cauchy matrices. Structural Information and Communication Complexity. Springer, 2013, when given the number r of redundant columns. The unit coding computational complexity of 3(1): 334-345.] is as shown in equation (31).

In order to fairly compare the coding complexity of the Cauchy array code, the parameter k is equal to p-r, so the unit coding computational complexity of the array code C(k, r, p) is as shown in equation (32).

When r=4, for array code C(p-4, 4, p), let k = p-4. When the value of p ranges from 11 to 47, the unit coding computational complexity of the array code C(p-4, 4, p), the cyclic Cauchy array code, and the CRS code is as shown in FIG. For the CRS code, let k = p-4.

In Figure 1, the values of the CRS codes are the average of 1000 running data. The results of Fig. 1 show that the unit coding computational complexity of the array code C(p-4, 4, p) is the lowest of the three array codes, and the computational complexity of the CRS code is the largest of the three array codes.

Decoding computational complexity:

Similarly, let k = pr, the unit of the array code C (pr, r, p) decodes the computational complexity ND _{C (pr, r, p)} as shown in equation (33).

When r=4, Rabin-Like code [GLFeng, RHDeng, F.Bao, et al.New efficient MDS array codes for RAID.Part II.Rabin-like codes for tolerating multiple (≥4) disk failures.IEEE Transactions on The decoding computational complexity of Computers, 2006, 54(12): 1473-1483] is p(9k+95) XOR operation, and when r is greater than 4, the decoding computational complexity is large and the author does not give Exactly the number of XOR. When r information columns are lost in the cyclic Cauchy array code, the unit decoding computation complexity ND _CC is as shown in equation (34).

When r=4, and when the parameter p takes values from 11 to 37, the array code C(p-4, 4, p), The unit decoding computational complexity of the Rabin-Like code, the Cauchy cyclic code, and the CRS code is shown in Figure 2. The results in Fig. 2 show that the cyclic Cauchy code, Rabin-Like code and array code C (p-4, 4, p) decrease with the increase of the parameter p, while the CRS code increases with the parameter p. And increase. This is because the decoding optimization algorithm of the CRS code performs better for the parameter value hourly, and when the parameter value is large, the optimization effect is not good. Therefore, when the parameter is large, the decoding computation complexity of the CRS code is the largest, and when the parameter is small, the decoding computation complexity of the CRS code is relatively low. In the array code C (p-4, 4, p), Rabin-Like code and Cauchy cyclic code, the decoding complexity of the array code C (p-4, 4, p) is the lowest, whether it is a parameter The value of p is small or large.

When r=5, and when the parameter p takes a value ranging from 11 to 59, the unit decoding computational complexity of the array code C(p-5, 5, p), the Cauchy cyclic code and the CRS code is as shown in FIG. Similar to the case where r is equal to 4, the decoding computation complexity of the CRS code becomes larger as the parameter p increases, and the computational complexity of the array code C(p-5, 5, p) and the Cauchy cyclic code follows The parameter p increases and becomes smaller. The decoding computational complexity of the array code C(p-5, 5, p) is the lowest at all tested data points. When the parameter p is greater than 47, the decoding computation complexity of the CRS code is greater than the decoding computation complexity of the cyclic Cauchy code. Conversely, when the parameter p is less than 47, the decoding computational complexity of the CRS code is less than the decoding computational complexity of the cyclic Cauchy code.

Compared with other MDS codes, such as CRS codes, the array code C(k, r, p) has the greatest advantage that its coding computation complexity and decoding computation complexity are much lower, and the number of original information data blocks is not fixed. Any integer from 2 to p can be taken. Compared with the EVENODD code that can recover two nodes, the array code C(k, r, p) improves the fault tolerance of the system when the codec complexity is almost unchanged, and can repair any r nodes at most. .

The array code C(k, r, p) has better codec complexity, and greatly improves the fault tolerance of the system, and the number of original information data blocks is not fixed. It can take any integer from 2 to p, which is more flexible. , to achieve the optimal compromise between storage overhead and system reliability.

The above is a further detailed description of the present invention in connection with the specific preferred embodiments, and the specific embodiments of the present invention are not limited to the description. It will be apparent to those skilled in the art that the present invention may be made without departing from the spirit and scope of the invention.

Claims

A MDS array code coding for repairing multi-node failures, characterized in that its component is a C(k, r, p) code, and a primitive (p-1)*(k+r) matrix is constructed to store original information. Data blocks and redundant blocks, where p is a prime number and p is greater than k and r, k and r are smaller than any integer less than p greater than 0; k columns are called information columns, which correspond to k data blocks; r columns are Redundant columns, which correspond to r redundant blocks, and the addition and subtraction operations in the C(k, r, p) code are XOR operations.
The MDS array code encoding for repairing multi-node failure according to claim 1, wherein the C (k, r, p) code is constructed by: adding k (p-1) information bits, calculating and adding k check bits are obtained, and k data polynomials are obtained. By calculating the coding polynomial, the number of times the polynomial is stored is a coefficient of a polynomial from 0 to p-2.
The MDS array code encoding for repairing multi-node failure according to claim 2, wherein the p-1 information bits stored in the jth information column and the first data polynomial represented by the check bits thereof are s j (x )=s 0,j +s 1,j x+...+s p-2,j x p-2 +s p-1,j x p-1 , where s i,j denotes array code C(k,r , p) the ith bit of the jth information column; the third coding polynomial is
The MDS array code encoding for repairing multi-node failure according to claim 1, wherein the division in the C(k, r, p) code is performed in the following manner: calculating a division operation for each data polynomial, and then The results of the corresponding data polynomial division operations are added.
The MDS array code encoding for repairing multi-node failure according to claim 1, wherein the C(k, r, p) code is a binary Cauchy array code, and the generator matrix is G=[I C], wherein the matrix I represents the unit matrix of k × k, and the matrix C is the k × r Cauchy matrix.
A decoding method for repairing multi-node failure MDS array code coding, comprising: the following steps: when ρ≤r columns are lost in the array code C(k, r, p), set γ information columns a 1, a 2, ..., a γ redundant columns and [delta] b 1, b 2, ..., b δ loss, wherein 1≤a 1 <a 2 <... < a γ ≤k, 1≤b 1 <b 2 <...<b γ ≤r,k≥γ≥0, r≥δ≥0 and γ+δ=ρ≤r, and the column of information that has not survived is represented as Μ={1,2,...,k}\ {a 1 , a 2 ,...,a γ }, denote redundant columns without loss as Ρ={1,2,...,r}\{b 1 ,b 2 ,...,b δ }, first download and survive There are no missing k-γ information columns i 1 , i 2 , ..., i k-γ ∈Μ, and γ redundant columns l 1 , l 2 , ..., l γ ∈Ρ, recovering the lost γ The information column then multiplies the coded vector of the missing redundant column by k data polynomials to calculate the missing redundant polynomial.
The method for decoding a multi-node failure MDS array code code according to claim 6, wherein: adding a parity bit to the information column i τ and forming a data polynomial
Where τ=1,2,...,k-γ; adding a bit zero to the redundant column l h and forming a coding polynomial
Where h = 1, 2, ..., γ; the polynomial obtained by subtracting k-γ data polynomials from γ coding polynomials is

Solving the missing γ data polynomials by solving linear equations, where the linear equations are

Then γ missing columns of information are obtained.
The method for decoding a multi-node failure MDS array code code according to claim 6, wherein the lost δ redundant columns are obtained by multiplying the corresponding coding vectors by k information polynomials.