CN107395207B - The MDS array code of more fault-tolerances encodes and restorative procedure - Google Patents

Abstract

The present invention relates to data processing fields, and it discloses the MDS array code of more fault-tolerances coding, component part C(k, 3, p) data block is expressed as k(p-1 by code) τ information bit and encode generation 3 (p-1) τ redundant digit, (p-1) τ is positive integer, τ=2^k‑2, p is prime number, k >=3；Each (p-1) τ information bit adds τ extra bits and forms message vector.The beneficial effects of the present invention are: improving the fault-tolerance of system;The computation complexity of encoding-decoding process is lower, greatly reduces and repairs broadband.

Description

The MDS array code of more fault-tolerances encodes and restorative procedure

[technical field]

The present invention relates to a kind of MDS array code of data processing field more particularly to more fault-tolerances coding and reparation sides Method.

[background technique]

Modern distributive storage system maintains availability of data using correcting and eleting codes, to avoid the failure of memory node.Two System maximum distance can divide (MDS) array code to be a kind of special correcting and eleting codes, multiple by minimum memory redundancy and lower calculating Miscellaneous degree realizes fault-tolerance.Specifically, binary array code is made of the array that k+r is arranged, and each column has L, for the k + r column, k column information column storage information bit therein, and r column parity column stores redundant digit.The position L in each column is stored in phase In same memory node.If any k in k+r column is enough to rebuild all k column information column, this code can be referred to as For MDS (i.e. it may be allowed any r column failure).The example of binary system MDS array code includes: X code [1] and RDP code [2], the two It is double volume shift (i.e. r=2)；There are also STAR codes [3], the RDP code [4] and TIP code that generalize, and three is three error-tolerance types (i.e. r=3).

When a node failure in distributed memory system, it should download bit by never failure node d to repair The column to fail again, wherein k≤d≤k+r-1.It is the downloading digit repaired in operation that definitions of bandwidth, which will be repaired,.It reduces and repairs as far as possible Multiple bandwidth is most important for the window for accelerating to repair operation and minimum fragility, especially usually becomes point of bottleneck in network transmission It is even more so in cloth storage system.Reparation problem is based on information flow chart by Dimakis et al. [5] and illustrates and study for the first time.Such as [5] it is illustrated in, minimum repairs bandwidth and is limited by minimum memory redundancy, is also referred to as minimum memory regeneration (MSR) point, meter It is as follows to calculate formula:

D=k+1 at this time.Although minimum bandwidth of repairing is achievable [5], [6] in sufficiently large finite field how The problem of binary system MDS array code that constitution realization minimum repairs bandwidth is still one and is rich in challenge.

Many researchs existing at present are conceived to the reparation bandwidth for reducing the single failure column in binary system MDS array code.Have A little methods minimize the reading of the disk of RDP code [7] and X code [8] as far as possible, but it can only be suboptimum that it, which repairs bandwidth, according to It is so big by 50% than the minimum value in (1).MDR code [9], [10] and ButterFly code [11] are binary system MDS array code, Optimal reparation is realized, but they only provide double fault-tolerances (i.e. r=2).How to construct and has both optimal reparation and more high fault tolerance The binary system MDS array code of (i.e. r > 2) is still a problem to be resolved.Such construct will be conducive to maintain in reality Availability of data in the distributed memory system of Frequent Troubles.

[summary of the invention]

In order to solve the problems in the prior art, the present invention provides a kind of MDS array code of more fault-tolerances coding and Restorative procedure solves the problems, such as that optimal reparation and more high fault tolerance can not be had both in the prior art.

The present invention is achieved by the following technical solutions: the MDS array code for designing, having manufactured a kind of more fault-tolerances is compiled Code, component part are C (k, 3, p) code, and data block is divided into k (p-1) τ information bit and encodes generation 3 (p-1) τ redundant digit, (p-1) τ is positive integer, τ=2^k-2, p is prime number, k >=3, and information bit is expressed asRedundancy Position is expressed asWherein j is k+1, k+2 and k+3, i=1,2 ..., k；Each (p-1) τ Information bit adds τ extra bits and forms message vector.

As a further improvement of the present invention: information being arranged and is indicated by multinomial, corresponding each information is classified as Data polynomial, corresponding three parity columns form encoded multinomial, data polynomial and encoded multinomial shape At column vector [s₁(x),s₂(x),…,s_k+3(x)]。

As a further improvement of the present invention: the column vector passes through R_pτ:=F₂[x]/(1+x^pτ) embodied in algorithm Take [s₁(x),s₂(x),…,s_k+3(x)]=[s₁(x),s₂(x),…,s_k(x)] product of .G is calculated, and wherein G is by k x K x (k+3) generator matrix composed by a k unit matrix I and 3 encoder matrix P of k x.

As a further improvement of the present invention: C (k, 3, p) code is in R_pτInside penetrate a systematization liner code.

Invention also provides a kind of restorative procedure of the MDS array code of more fault-tolerances, include the following steps: to obtain Information column through failing；If no longer valid information arrangesLetter is then repaired by the first even-odd check Cease position s_l,f, wherein l mod 2^f∈{0,1,2,...2^f-1- 1 }, otherwise by second even-odd check come restoration information position s_{L, f}, Wherein l mod 2^f∈{2^f-1, 2^f-1+ 1,2^f-1+ 2 ..., 2^f-1}；If no longer valid information arrangesBy first even-odd check come restoration information position s_{L, f}, wherein l mod 2^f∈ 0,1, 2 ... 2^f-1- 1 }, otherwise, by third even-odd check come restoration information position s_{L, f}, wherein l mod 2^f∈{2^f-1, 2^f-1+ 1, 2^f-1+ 2 ..., 2^f-1}。

As a further improvement of the present invention: no longer valid information columnReparation broadband be (p-1) ((k+2)2^k-3-2^k-f-2)。

As a further improvement of the present invention: the even-odd check collection of the second parity column and third parity column is not Corresponding to information column those of linear in array, and correspond in those of broken line information column；The line number of array is by 2^k-2It is whole It removes.

The beneficial effects of the present invention are: improving the fault-tolerance of system；The computation complexity of encoding-decoding process is lower, greatly Reduce repair broadband.

[Detailed description of the invention]

Fig. 1 is the embodiment schematic diagram of three parity columns of the invention storage code used.

[specific embodiment]

The present invention is further described for explanation and specific embodiment with reference to the accompanying drawing.

Abbreviation and Key Term definition

MDS Maximum Distance Separable maximum distance separable

The verification of RDP Row-Diagonal Parity row diagonal line

A kind of MDS array code coding of more fault-tolerances, component part is C (k, 3, p) code, and data block is divided into k (p-1) τ information bit simultaneously encodes generation 3 (p-1) τ redundant digit, and (p-1) τ is positive integer, τ=2^k-2, p is prime number, k >=3, information bit table It is shown asRedundant digit is expressed asWherein j is k+ 1, k+2 and k+3, i=1,2 ..., k；Each (p-1) τ information bit adds τ extra bits and forms message vector.

Information is arranged and is indicated by multinomial, corresponding each information is classified as data polynomial, and corresponding three Parity column forms encoded multinomial, and data polynomial and encoded multinomial form column vector [s₁(x), s₂(x) ..., s_k+3(x)]。

The column vector passes through R_pτ:=F₂[x]/(1+x^pτ) embodied in algorithm take [s₁(x), s₂(x) ..., s_k+s (x)]=[s₁(x), s₂(x) ..., s_k(x)] product of .G is calculated, and wherein G is by k x k unit matrix I and a k x K x (k+3) generator matrix composed by 3 encoder matrix P.

C (k, 3, p) code is in R_pτInside penetrate a systematization liner code.

Invention also provides a kind of restorative procedure of the MDS array code of more fault-tolerances, include the following steps: to obtain Information column through failing；If no longer valid information arrangesLetter is then repaired by the first even-odd check Cease position s_{L, f}, wherein l mod 2^f∈ 0,1,2 ... 2^f-1- 1 }, otherwise by second even-odd check come restoration information position s_{L, f}, Wherein l mod 2^f∈{2^f-1, 2^f-1+ 1,2^f-1+ 2 ..., 2^f-1}；If no longer valid information arrangesBy first even-odd check come restoration information position s_{L, f}, wherein l mod 2^f∈ 0,1, 2 ...^2f-1- 1 }, otherwise, by third even-odd check come restoration information position s_{L, f}, wherein l mod 2^f∈{2^f-1, 2^f-1+ 1, 2^f-1+ 2 ..., 2^f-1}。

No longer valid information columnReparation broadband be (p-1) ((k+2) 2^k-3-2^k-f-2)。

The even-odd check collection of second parity column and third parity column does not correspond to that in array linearly A little information column, and correspond in those of broken line information column；The line number of array is by 2^k-2Divide exactly.

In one embodiment, the construction of the MDS array code of more fault-tolerances is as follows:

It enables k >=3 and L=(p-1) τ is positive integer, wherein τ=2^k-2, and p is prime number and 2 is Z_pPrimitive in domain.Consider One document size is k (p-1) τ bit, by information bitIt indicates (i=1,2 ..., k), It can be used for generating 3 (p-1) τ redundant digits(j=k+1, k+2, k+3).

For 1=1,2 ..., k+3 and μ=0,1 ... τ -1 is defined as follows shorthand notation:

ClaimFor additional bit, withIt is related.For example, work as p=3, k= When 4 and τ=4,Extra bits be

For l=1,2 ..., k+3, pass through ring F₂Multinomial s on [x]₁(x) bit in l will be arrangedWith τ additional bitCo-expressing is multinomial s_l(x):

Multinomial s corresponding to the i-th information column (i=1,2 ..., k)_i(x) it is referred to as data polynomial, it is odd corresponds to j-k The multinomial of even parity check column (j=k+1 .k+2 .k+3) is referred to as coding polynomial.

K number is written as column vector according to multinomial and 3 coding polynomials

[s₁(x), s₂(x) ..., s_k+3(x)], (3)

It can pass through utilizationEmbodied in algorithm take [s₁(x), s₂(x) ..., s_k+3(x)]= [s₁(x), s₂(x) ..., s_k(x)] product of G is calculated.K x (k+3) generator matrix G is by k x k unit matrix I and one A 3 encoder matrix P of k x is formed,

In ring R_pτIn, variable x represents the ring shift right operator on multinomial.This is for reducing an information column failure It is most important for reparation bandwidth.The code proposed is expressed as C (k, 3, p).Magnetic will not be stored in for additional bit by please noting that On disk；They are only used for the convenience indicated.Consider k=4 and such a example of p=3,32 information bits are by s_{0, i}, s_{1, i}..., s_{7, i}It indicates, wherein i=1,2,3,4.The exemplary encoder matrix is

The example is illustrated in Fig. 1, and wherein the bit-cell of font-weight is extra bits.

Cataloged procedure can be described with following polynomial form.Given k (p-1) τ information bit, for each (p-1) τ letter It ceases bit and adds τ extra bits, and form message vector [s₁(x), s₂(x) ..., s_k(x)]。

After the vector in acquisition (3), storage number is 0 term coefficient arrived in (p-1) τ -1 multinomial.The battle array proposed The system linear code that column code is seen as the operation in Rp τ.

The asymptotic optimization reparation of primary information failure, in one embodiment,

It will show how to repair by accessing the bit from k-1 other information column and 2 parity columns The bit s being stored in information column f_{0, f}, s_{1, f}..., s_{(p-1) τ -1, f}And there is asymptotic optimization to repair bandwidth, wherein 1≤f≤k. It please remember additional bit can be calculated by (2).For the convenience of expression, the bit-cell for arranging i is expressed as τ s of p_{0, i}, s_{1, i}..., s_{P τ -1, i}.Before providing reparation algorithm, formally even-odd check collection is defined as follows.

1. are defined for τ -1 0≤1≤p, first row, secondary series and tertial 1st even-odd check collection are defined respectively For

With

Please note that all index modulus defined in 1 and full text are p τ.By defining 1, even-odd check collectionIncluding multiple letters Bit is ceased, can be used for generating redundant digitWhen saying that an information bit is repaired by a parity column, this meaning Other than having wiped bit, have accessed all information ratios in the redundant digit and the parity column of parity column It is special.Consider the example provided in Fig. 1.It is assumed that first row is wiped free of.The accessible position bit s of people_0,2, s_0,3, s_0,4And redundancy Bit s_0,1+s_0,2+s_0,3+s_0,4To pass through s_0,2+s_0,3+s_0,4+(s_0,1+s_0,2+s_0,3+s_0,4) Lai Chongjian s_0,1。

Another embodiment below.

Algorithm is repaired to state in algorithm 1.Consider example given by Fig. 1 again, was repaired with elaborating Journey.In this example, k=5, d=5 and τ=4.Assuming that first information column (i.e. node 1) failure, that is to say, that f=1.According to calculation In method 1 the 2nd, 3 steps, bit can be repaired by first parity columnWhereinAnd 0≤1≤ 7.More specifically, bit s_0,1, s_2,1, s_4,1, s_6,1ByIt rebuilds.

Due to f=1 ∈ { 1,2 }, other information bitBy the second parity column reparation, whereinMod 2 and 0 ≤1≤7。

Therefore, bit s_1,1, s_3,1, s_5,1, s_7,1By

s_1,1=s_0,2+s_10,3+s_2,4+(s_1,1+s_0,2+s_10,3+s_2,4)

s_3,1=s_2,2+s_0,3+s_4,4+(s_3,1+s_2,2+s_0,3+s_4,4)

s_5,1=s_4,2+s_2,3+s_6,4+(s_5,1+s_10,2+s_8,3+s_0,4)

s_7,1=s_6,2+s_4,3+s_8,4+(s_11,1+s_10,2+s_8,3+s_0,4)+(s_3,1+s_2,2+s_0,3+s_4,4)

It rebuilds.

Because s can be passed through_6,3+s_2,3Calculate s_10,3And pass through s_4,4+s_0,4Calculate s_8,4, so without downloading bit s_10,3 And s_8,4.Accordingly, it is believed to be desirable to download four bits from each column of three information column and two parity columns.It repairs The bit of first information column is needed in total from five column 20 bits of downloading.That is, the bit of the data column for reparation is only There is half to receive access.In Fig. 1, the bit in solid box can be downloaded, with restoration information bit s_0,1, s_2,1, s_4,1, s_6,1, and the bit in dotted line frame can be used to restoration information bit s_1,1, s_3,1, s_5,1, s_7,1。

Assuming that the second information column (i.e. node 2) failure, that is to say, that f=2.According to the 2nd, 3 in algorithm 1

Step, can pass through

Repair bit s_0,2, s_1,2, s_4,2, s_5,2。

Similarly, can pass through

s_2,2=s_3,1+s_0,3+s_4,4+(s_3,1+s_2,2+s_0,3+s_4,4)

s_3,2=s_4,1+s_1,3+s_5,4+(s_4,1+s_3,2+s_1,3+s_5,4)

s_6,2=s_7,1+s_4,3+s_0,4+s_4,4+

(s_11,1+s_10,2+s_{S, 3}+s_0,4)+(s_3,1+s_2,2+s_0,3+s_4,4)

s_7,2=s_0,1+s_4,1+s_5,3+s_{Isosorbide-5-Nitrae}+s_5,4+

(s_0,1+s_11,2+s_9,3+s_{Isosorbide-5-Nitrae})+(s_4,1+s_3,2+s_1,3+s_5,4).

Repair bit s_2,2, s_3,2, s_6,2, s_7,2。

Therefore, the 8 bits being stored in the second information column can by download six bits arrange from the first information with Four bits of each column in third information column, the 4th information column, the first parity column and the second parity column Restored.22 bits have been downloaded in repair process in total.For the code gone out given in Fig. 1, third information column and last Information column can be rebuild by accessing 22 bits arranged from 5 and 20 bits respectively.

Theorem 3: whenWhen, the information column f obtained by algorithm 1 repairs bandwidth and is

(p-1)((k+2)2^k-3-2^k-f-2)。

It proves: according to algorithm 1, bitPass through the even-odd check collection of the first parity columnIt is repaired, wherein 1mod2f ∈ { 0,1,2 ..., 2^f-1- 1 } andTherefore, it is necessary to each column arranged from remaining k-1 information to access (p-1) τ/2 bitWherein i ∈ { 1,2 ..., f-1, f+1 ..., k } and 1mod 2^f∈ 0,1,2 ... 2^f-1- 1 }, And τ/2 (p-1) redundant bit is downloaded from the first parity columnWherein 2 1modⁱ∈ { 0,1,2 .., 2^i-1- 1}.It follows that there is τ/2 k (p-1) bit to need to download.

For 1mod 2^f∈{2^f-1, 2^f-1+ 1,2^f-1+ 2 ..., 2^f- 1 }, bitPass throughIt is repaired.It please remember FirmlyTherefore it needs to access τ/2 (p-1) redundant digit

For the column i of i ∈ { 1,2..., f-1 }, τ/2 (p-1) are neededWherein whole values of 1mod 2f are located at collection { 0,1 ..., 2^f-1-2^i-1- 1,2^f-2^i-1, 2^f-2^i-1+ 1 ..., 2^f- 1 } among.And for the column of i ∈ { f+1, f+2 ..., k } I needs τ/2 (p-1)Wherein 2 1mod^f∈ 0,1,2 ... and, 2^f-1-1}。

It note that in repair process, bit(wherein 1mod2^f∈ 0,1,2 ... 2^f-1- 1 } and The downloading of the first parity column is passed through.Thus, it is only necessary to τ/2 (p-1) redundant digit is downloaded from the second parity column, with And (p-1) 2 is downloaded from column i^k+i-f-3A bit, wherein i=1,2 ..., f-1.

Restoration information column f can be calculated and need to arrange the bit sum downloaded from k+2 and be

WhenWhen, according to algorithm 1, the reparation bandwidth of information column k+1-f is identical as the column reparation bandwidth of f.

Therefore, only considerSituation.According to theorem 3, when f increases, repairs bandwidth and increase therewith Greatly.As f=1, repairing bandwidth isThus optimal value is obtained in (1).Even forWorst-case, reparation bandwidth be (p-1) ((k+2) 2^k-3-2^k-[k/2]-2) < (p-1) (k+2) 2^k-3, this is stringent Lower than (1) intermediate valueTimes.Therefore, when k is sufficiently large, the reparation bandwidth of any one information failure can progressively exist (1) optimal reparation is obtained in.

It should be pointed out that in proposed code the in even-odd check collection and the RDP and EVENODD of the first parity column The even-odd check collection of one parity column is identical.The key area between code and existing binary system MDS array code proposed It is not the construction of second and third parity column.Firstly, second and third parity column in the code proposed Even-odd check collection does not correspond to those of linear bit in array, and corresponds in those of broken line bit.Secondly, institute It is proposed that the line number of array in code can be by 2^k-2Divide exactly.The two properties are most important for reducing reparation bandwidth.

Bibliography

[1] L.Xu and J.Bruck, " X code: the MDS array code with optimum code " IEEE information theory can be reported, and 1999 The 1st phase page 272-276 of volume 45.

[2] P.Corbett, B.English, A.Goel, T.Grcanac, S.Kleiman, J.Leong and S.Sankar, " row-Diagonal Parity Check for the correction of double disk failures ", the 3rd USENIX meeting document and memory technology collection of thesis, 2004 page 1-14.

[3] C.Huang and L.Xu, " STAR: a kind of for correcting the efficient coding scheme of triple memory node failures ", IEEE computer can be reported, the 7th phase page 889-901 of volume 57 in 2008.

[4] M.Blaum, " a series of MDS array codes with minimum code operand ", the discussion of the IEEE information theory world Meeting, 2006 page 2784-2788.

[5] A.Dimakis, P.Godfrey, Y.Wu, M.Wainwright and K.Ramchandran " are deposited for distribution The network code of storage system ", IEEE information theory can be reported, in September, 2010 the 9th phase page 4539-4551 of volume 56.

[6] I.Tamo, Z.Wang and J.Bruck, " tortuous code: the MDS array code with optimal reconstruction " IEEE information reason By can report, the 3rd phase page 1597-1616 of volume 59 in 2013.

[7] L.Xiang, Y.Xu, J.Lui and Q.Chang, " single the optimal of disk failure is repaired in RDP code storage system It is multiple " ACM SIGMETRICS Performance Evaluation version, the 1st phase page 119-130 of volume 38 of ACM in 2010.

[8] S.Xu, R.Li, P.P.Lee, Y.Zhu, L.Xiang, Y.Xu and J.Lui " are used for X code base stored parallel system Single disk failure restore " IEEE computer can report, the 4th phase page 995-1007 of volume 6 in 2014.

[9] Y.Wang, X.Yin and X.Wang, a kind of " MDR coding: RAID-6 with optimal reconstruction and coding mode Coding sorts " IEEE communicates selection area periodical, the 5th phase page 1008-1018 of volume 32 in 2013.

[10] ---, " two kinds of double even-odd check MDS array code new categories with optimal reparation ", IEEE communicates flash report, The 7th phase page 1293-1296 of volume 20 in 2016.

[11] L.Pamies-Juarez, F.Blagojevic, R.Mateescu, C.Gyuot, E.E.Gad and Z.Bandic, " breaking the bond: the true repairing performance of MSR coding " the 14th USENIX document and memory technology meeting (FAST 16), page 81-94 in 2016.

[12] K.W.Shum, H.Hou, M.Chen, H.Xu and H.Li, " code is regenerated in cyclic binary code ", IEEE information theory conference Papers collection, in July, 2014 Honolulu, page 1046-1050.

[13] H.Hou, K.W.Shum, M.Chen and H.Li, " the novel MDS array code of correction multiple disks failure ", entirely Ball communicate meeting, 2015 page 2369-2374.

[14] ---, " BASIC code: the low complex degree for distributed memory system regenerates code " IEEE information theory It can report, the 6th phase page 3053-3069 of volume 62 in 2016.

The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be said that Specific implementation of the invention is only limited to these instructions.For those of ordinary skill in the art to which the present invention belongs, In Under the premise of not departing from present inventive concept, a number of simple deductions or replacements can also be made, all shall be regarded as belonging to of the invention Protection scope.

Claims (7)

1. a kind of MDS array code encoding method of more fault-tolerances is applied to distributed memory system, it is characterised in that: how fault-tolerant Property MDS array code coding component part be C (k, 3, p) code, data block is expressed as k (p-1) τ information bit and encodes generation 3 (p-1) τ redundant digit, (p-1) τ are positive integer, τ=2^k-2, p is prime number, k >=3, and information bit is expressed asRedundant digit is expressed asWherein j is k+1, k+ 2 and k+3, i=1,2 ..., k；τ extra bits are added for each (p-1) τ information bit and form message vector.

2. the MDS array code encoding method of more fault-tolerances according to claim 1, it is characterised in that: pass through information column Multinomial is indicated, and corresponding each information is classified as data polynomial, and corresponding three parity columns form encoded Multinomial, data polynomial and encoded multinomial form column vector [s₁(x),s₂(x),…,s_k+3(x)]。

3. the MDS array code encoding method of more fault-tolerances according to claim 1, it is characterised in that: column vector passes through R_pτ: =F₂[x]/(1+x^pτ) embodied in algorithm take [s₁(x),s₂(x),…,s_k+3(x)]=[s₁(x),s₂(x),…,s_k(x)].G Product calculated, wherein G is as composed by k x k unit matrix I and a 3 encoder matrix P of k x k x (k+3) raw At matrix.

4. the MDS array code encoding method of more fault-tolerances according to claim 1, it is characterised in that: C (k, 3, p) code is In R_pτInside penetrate a systematization liner code.

5. a kind of restorative procedure of the MDS array code of more fault-tolerances as described in claim 1, it is characterised in that: including as follows Step: no longer valid information column are obtained；It is arranged if no longer valid informationIt is then odd by first Even parity check carrys out restoration information position s_l,f, wherein l mod 2^f∈{0,1,2,...2^f-1- 1 }, otherwise by second even-odd check come Restoration information position s_l,f, wherein l mod 2^f∈{2^f-1,2^f-1+1,2^f-1+2,...,2^f-1}；It is arranged if no longer valid informationBy first even-odd check come restoration information position s_l,f, wherein l mod 2^f∈ 0,1, 2 ..., 2^f-1- 1 }, otherwise, by third even-odd check come restoration information position s_l,f, wherein l mod 2^f∈{2^f-1,2^f-1+1, 2^f-1+2,...,2^f-1}。

6. restorative procedure according to claim 5, it is characterised in that: no longer valid information column's Reparation broadband is (p-1) ((k+2) 2^k-3-2^k-f-2)。

7. restorative procedure according to claim 5, it is characterised in that: the second parity column and third parity column Even-odd check collection does not correspond to those of linear information column in array, and corresponds in those of broken line information column；Battle array The line number of column is by 2^k-2Divide exactly.