CN104407807B - A kind of storage expansion method for RS code storage clusters - Google Patents

A kind of storage expansion method for RS code storage clusters Download PDF

Info

Publication number
CN104407807B
CN104407807B CN201410579759.XA CN201410579759A CN104407807B CN 104407807 B CN104407807 B CN 104407807B CN 201410579759 A CN201410579759 A CN 201410579759A CN 104407807 B CN104407807 B CN 104407807B
Authority
CN
China
Prior art keywords
piecemeal
row
data
node
deblocking
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201410579759.XA
Other languages
Chinese (zh)
Other versions
CN104407807A (en
Inventor
黄建忠
曹强
谢长生
梁先海
黄文胜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN201410579759.XA priority Critical patent/CN104407807B/en
Publication of CN104407807A publication Critical patent/CN104407807A/en
Application granted granted Critical
Publication of CN104407807B publication Critical patent/CN104407807B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0647—Migration mechanisms

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a kind of for the RS yards of efficient storage expansion method of storage cluster, it is related to data to redistribute and two processes of checksum update.Specifically include：Increase Δ k according to node, and legacy data is redistributed according to transposition formula data layout, wherein initial data piecemeal is only migrated between legacy data node and new data node, and new data piecemeal vacates region after being directly filled into migration；Complete when data are redistributed, it is necessary to accordingly be verified the renewal operation of piecemeal.Above-mentioned data redistribution supports multiple dilatation and keeps the uniformity of data distribution.Because data redistribution uses transposition formula data layout, when verification piecemeal updates, legacy data node can directly calculate verification difference piecemeal Δ P, and check-node completes the renewal of verification piecemeal using this verification difference piecemeal Δ P, so as to reduce checksum update expense.Expansion method is stored in the present invention can not only improve storage efficiency, and can complete efficient dilatation.

Description

A kind of storage expansion method for RS code storage clusters
Technical field
The invention belongs to Computer Storage field, more particularly, to a kind of for the RS yards of storage dilatation of storage cluster Method.
Background technology
Correcting and eleting codes are widely used in distributed storage, and cluster-based storage enters multiple memory nodes according to erasure code mode Row connection, constitutes the storage system with certain fault-tolerant ability.Erasure code mainly includes that Reed-Solomon encodes (Reed Solomon Codes, RS Codes) and parity matrix coding.Under identical capacity conditions, RS codings have than odd even school Test matrix coder fault-tolerant ability higher.
With storage cluster operation and use, in cluster memory capacity will gradually decrease, it is necessary to pass through increase storage section The mode of point completes the purpose of dilatation.Here, cluster dilatation is the addition for being primarily referred to as carrying out RS storage clusters back end.
Existing disk array expansion scheme is mainly for RAID-0, RAID-4, RAID-5 and RAID-6, and they are all uncomfortable Together in cluster-based storage dilatation.Data are not verified by the RAID-0 migration of processing data, and Information Security is not high.RAID- 4 require there is uniform data distribution on all data disks, and this can cause substantial amounts of Data Migration.Although having a kind of optional RAID-4 expansion schemes, it is being filled by reducing the migration of data on movable part deblocking to new disk During new data, the cost that this scheme updates verification piecemeal is very high.RAID-5 is applied to RS using the data distribution based on poll During storage cluster, due to data movement and checksum update, cluster will face the too high problem of I/O expenses.The extension of RAID-6 is only For specific RAID-6 code Designs, correcting and eleting codes storage cluster is not suitable for.
The content of the invention
For the disadvantages described above or Improvement requirement of prior art, deposited for Reed-Solomon code the invention provides one kind The storage expansion method of accumulation, can not only improve the memory capacity of storage cluster, and energy minimization Data Migration amount and Checksum update expense.
In order to realize foregoing invention purpose, the invention provides a kind of storage dilatation side for RS code storage clusters Method, comprises the following steps：
(1) process that the dilatation in cluster-based storage space is redistributed with data, including following sub-step：
(1.1) increase memory node, storage cluster is physically increased to the back end for adding new in storage cluster Capacity；
(1.2) redistribution of legacy data piecemeal, including following sub-step：
(1.2.1) reads the logical block number (LBN) x of legacy data piecemeal, then presses row, column number and calculates deblocking coordinate (row0, col0), wherein row0=x/k represents business of the x divided by k, col0=x%k represents the remainder for seeking x divided by k, and k is the number of storage cluster According to node number；
(1.2.2) is according to the deblocking coordinate (row for above calculating0,col0) judge whether deblocking needs to move Move；Make p=row0%k', the standard of judgement is：If 0≤p ＜ k, deblocking need not be migrated, then the address of the deblocking It is still (row0,col0)；Otherwise, the deblocking needs to be migrated；
(1.2.3) calculates school after the deblocking to be migrated in legacy data node completes migration by current data node Test difference piecemeal Δ P and be sent to check-node；By storage deblocking DCiNode N (DCi) it is row row deblocking meters Calculate r verification difference piecemeal { Δ Prow,1,ΔProw,2,...,ΔProw,rAnd it is separately sent to r check-node { N (PC1),N (PC2) .., N (PCr)}；Whereinαj,iIt is the i-th row, the element of jth row in redundant matrices, Di,rowIt is the deblocking of (i, row) to represent address, and r represents the number of check-node；
(1.2.4) repeats step (1.2.1), (1.2.2), (1.2.3), until whole legacy data piecemeals have been processed Into, all deblockings is evenly distributed in storage cluster, complete data redistribution process；
(1.3) filling of new data piecemeal；Including following sub-step：
(1.3.1) calculates its filling destination address (row, col) according to the logical block number (LBN) x of new data piecemeal；Make y=x-k × C, whereinCol=y%k', C represent the deblocking number that memory node can be accommodated Amount；
New data piecemeal is filled into corresponding filling destination address by (1.3.2)；
(1.3.3) is filled by new data piecemeal if data line piecemeal and completed, then using RS coding generation verifications point Block is simultaneously sent to check-node；
(1.3.4) repeats (1.3.1), (1.3.2), (1.3.3) until new data piecemeal is stuffed entirely with completing；
(2) process that corresponding verification piecemeal updates after data redistribution, including following sub-step：
(2.1) step (1.2) can produce the verification difference piecemeal Δ P, step (1.3) to generate brand-new verification piecemeal Pnew, Former verification piecemeal, when check-node receives the verification difference piecemeal Δ P produced by Data Migration, check-node are represented with P Former verification piecemeal is updated with P+ Δs P；When check-node receives the P produced by new data fillingnewWhen, check-node PnewReplace Change corresponding former verification piecemeal；
(2.2) (2.1) are repeated, until all verification piecemeals all complete to update.
Further, the new of deblocking after dilatation is calculated in the step (1.2.2) according to transposition formula data layout Address (row, col), wherein row=(row0/k')*k'+col0, col=row%k', k'=k+ Δ k, Δ k are storage clusters Newly-increased back end number.In general, by the contemplated above technical method of the present invention compared with prior art, Neng Gouqu Obtain following beneficial effect:
1st, the method for the present invention is redistributed according to transposition formula data layout to data, realizes minimum Data Migration With being uniformly distributed for data.
2nd, the method for the present invention uses read-modify-write locally formula update mode in the renewal to verifying piecemeal, by legacy data Node directly generates verification difference piecemeal, it is to avoid the transmission of legacy data piecemeal during checksum update, so as to minimize verification piecemeal Update expense.
3rd, the method for the present invention performs Data Migration and checksum update with asynchronous system, accelerates storage cluster dilation process Carrying out.
Brief description of the drawings
Fig. 1 is (k+r, k) RS yards of storage cluster data layout schematic diagram；
(a) is data layout change schematic diagram before and after cluster dilatation in Fig. 2, and (b) is transposition formula data layout schematic diagram；
Fig. 3 is Data Migration schematic diagram in the embodiment of the present invention；
Fig. 4 is the inventive method overview flow chart；
Fig. 5 is legacy data redistribution flow chart in the present invention；
Fig. 6 is new data filling flow chart in the present invention；
Fig. 7 is checksum update flow chart in the present invention.
Specific embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, it is right below in conjunction with drawings and Examples The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and It is not used in the restriction present invention.As long as additionally, technical characteristic involved in invention described below each implementation method Not constituting conflict each other can just be mutually combined.
Hereinafter technical term of the invention is explained and illustrated first：
The RS yards of data layout of storage cluster：As shown in figure 1, (k+r, k) RS yards of storage cluster include k according to node { N (DC1),N(DC2) ..., N (DCk) and r check-node { N (PC1), N (PC2) ..., N (PCr), each node is wrapped Containing many deblockings, a row can be referred to as, for example, N (DC1) include deblocking { D1,1, D2,1... } and represent deblocking First row, N (PC1) include deblocking { P1,1, P2,1... } and the first row for verifying piecemeal is represented, wherein { D1,1, D1,2..., D1,kConstitute the first row deblocking, { P1,1, P1,2..., P1,rConstitute the first row verification piecemeal.Cluster expands The individual data node table for holding newly-increased Δ k is shown as { N (DCk+1), N (DCk+2) ..., N (DCk+Δk)}。
Transposition formula data layout：As shown in Fig. 2 the k that preceding k rows k is arranged in Fig. 2 (a)2Individual deblocking is nonmigratory number According to, and the row of kth+1 needs to be migrated according to the matrix transposed way shown in Fig. 2 (b) to the Δ k × k of k+ Δ k rows according to piecemeal, For example, the k of the row of kth+1 according to piecemeal move to kth+1 arrange in, the k of the row of kth+2 moves to kth+2 and arranges according to piecemeal In ..., the k of kth+Δ k rows moved to during kth+Δ k arranges according to piecemeal.Δ k × the k vacated after migration is used according to piecemeal In storage new data piecemeal.
For simplicity of exposition, below to some variables such as k, Δ k, k' in involved formula in the explanation of embodiment etc. Unified explanation will be carried out herein, without being laid down a definition again in the specific steps of embodiment.Here is that the implication of each variable is said It is bright：X represents the logical block number (LBN) of deblocking, and k represents storage cluster original back end number, and Δ k represents newly-increased back end Number, k'=k+ Δs k is represented increases the quantity of total data node that cluster after back end is included；αj,iRepresent in redundant matrices The element of the i-th row, jth row；C represents the deblocking quantity that memory node can be accommodated, and k' is equal in embodiment.In addition, The logical block number (LBN) of deblocking includes { 0,1 ..., 35 } in embodiment, and wherein logical block 0-23 belongs to legacy data piecemeal, 24-35 Belong to new data piecemeal.
In embodiments of the present invention, it is the whole flow process such as figure of (8,6) RS storage clusters from the dilatation of (6,4) RS storage clusters Shown in 4, mainly include the following steps that：
(1) process that the dilatation in cluster-based storage space is redistributed with data.Including following sub-step：
(1.1) memory node is increased.Storage cluster is physically increased to the back end for adding new in storage cluster Capacity.
In the embodiment of the present invention, as shown in figure 3, (6,4) RS storage clusters originally have 4 back end { N (DC1),N (DC2), N (DC3), N (DC4), wherein back end N (DCi) include 6 deblocking { D1, i, D2, i..., D6, i}；And 2 Individual check-node { N (PC1), N (PC2), wherein N (PC1) comprising 6 verification piecemeal { P1, i, P2, i..., P6, i, now increase by two Individual back end { N (DC5), N (DC6) expand the capacity of storage cluster.
(1.2) redistribution of legacy data piecemeal.The flow of this step is as shown in figure 5, comprise the following steps：
(1.2.1) reads the logical block number (LBN) x of legacy data piecemeal, then presses row, column number and calculates deblocking coordinate (row0, col0), wherein row0=x/k represents business of the x divided by k, col0=x%k represents the remainder for seeking x divided by k, and k is the number of storage cluster According to node number.
In the embodiment of the present invention, it then follows convention of the matrix since the 1st row, the 1st row, when representing deblocking in figure 3 Row and column coordinate in deblocking address all adds 1, for example, the logical block number (LBN) of preceding 16 deblockings is 0 to 15, calculates The address of logical block 0 is (0,0), and row and column coordinate is expressed as D in figure 3 after all adding 11,1, logical block { 1,2 ..., 15 } is respectively It is expressed as { D1,2..., D4,4}。
(1.2.2) is according to the deblocking coordinate (row for above calculating0,col0) judge whether deblocking needs to move Move.Making the standard of p=row0%k', judgement is：If 0≤p ＜ k, deblocking need not be migrated, then the address of the deblocking It is still (row0,col0)；Otherwise, the deblocking needs to be migrated.Number after dilatation is calculated according to transposition formula data layout According to the new address (row, col) of piecemeal, wherein row=(row0/k')*k'+col0, col=row%k', k'=k+ Δ k, Δ k It is the newly-increased back end number of storage cluster.
In embodiments of the present invention, k=4, Δ k=2, k'=6, the actual row coordinate of the deblocking of preceding 4 row is respectively 0th, 1,2,3, they are both less than 4 divided by 6 remainder；The actual row coordinate of the deblocking of 2 rows is 4,5 afterwards, and they are divided by more than 6 Number is both greater than or equal to 4.Drawn a conclusion according to criterion：Deblocking { the D of preceding 4 row1,1..., D4,4Need not migrate, Deblocking { the D of 2 rows afterwards5,1..., D6,4Need migration.For deblocking { D5,1..., D6,4, by formula row= (row0/k')*k'+col0, the move target address that col=row%k' is calculated be respectively (1,5), (2,5) ..., (4, 6)}.As shown in figure 3, deblocking { D5,1, D5,2..., D6,4Move to { D1,5, D2,5..., D4,6}。
(1.2.3) calculates school after the deblocking to be migrated in legacy data node completes migration by current data node Test difference piecemeal Δ P and be sent to check-node.By storage deblocking DCiNode N (DCi) it is row row deblocking meters Calculate r verification difference piecemeal { Δ Prow,1,ΔProw,2,...,ΔProw,rAnd it is separately sent to r check-node { N (PC1),N (PC2) .., N (PCr)}.Whereinαj,iIt is the element of i-th, jth row in redundant matrices, Di,row It is the deblocking of (i, row) to represent address, and r represents the number of check-node.
In embodiments of the present invention, as shown in figure 3, working as deblocking { D5, i, D6, iComplete migration after, by N (DCi) calculate ΔPi,1With Δ Pi,2And they are respectively transmitted to N (PC1), N (PC2), wherein Δ Pi,1=α1,5DC5,i+α1,6DC6,i, Δ Pi,2= α2,5DC5,i+α2,6DC6,i, i ∈ { 1,2,3,4 }.
(1.2.4) repeats step (1.2.1), (1.2.2), (1.2.3), until whole legacy data piecemeals have been processed Into, all deblockings is evenly distributed in storage cluster, complete legacy data redistribution process.
In embodiments of the present invention, N (DC have been processed as shown in figure 3, working as4) when, the redistribution of legacy data is completed.
(1.3) filling of new data piecemeal.The flow of this step is as shown in fig. 6, comprise the following steps：
(1.3.1) calculates its filling destination address (row, col) according to the logical block number (LBN) x of new data piecemeal.Make y=x-k × C, whereinCol=y%k', C=k'.
In embodiments of the present invention, as shown in figure 3, calculating the deblocking that logical block number (LBN) is { 24,25 ..., 29 } Filling destination address be respectively { D5,1, D5,2..., D6,4}。
New data piecemeal is filled into corresponding filling destination address by (1.3.2).
In embodiments of the present invention, as shown in figure 3, logical block { 24,25 ..., 29 } is sequentially filled into { D5,1, D5,2..., D6,4}。
(1.3.3) is filled by new data piecemeal if data line piecemeal and completed, then using RS coding generation verifications point Block is simultaneously sent to check-node.
In embodiments of the present invention, as shown in figure 3, when 6 deblockings of the 5th row are by logical block { 24,25 ..., 29 } When filling is completed, using RS encryption algorithms generation verification piecemeal P5,1、P5,2And it is respectively transmitted to check-node { N (PC1), N (PC2), when the 6th row is filled by deblocking { 30,31 ..., 35 } to be completed, using RS encryption algorithms generation verification piecemeal P6,1、P6,2And it is respectively transmitted to check-node { N (PC1), N (PC2)}。
(1.3.4) repeats step (1.3.1), (1.3.2), (1.3.3) until new data piecemeal is stuffed entirely with completing.
In embodiments of the present invention, in figure 3, when the deblocking of the 6th row is filled by logical block { 30,31 ..., 35 }, And verify piecemeal P6,1、P6,2And be respectively transmitted and give check-node { N (PC1), N (PC2) when, new data filling is fully completed.
(2) process that corresponding verification piecemeal updates after data redistribution.The flow of this step as shown in fig. 7, comprises Following steps：
(2.1) when check-node have received the difference verification piecemeal Δ P produced by Data Migration, the former verification piecemeal of order is P, check-node P+ Δs P complete the renewal to verifying piecemeal.
Step (1.2) can produce the verification difference piecemeal Δ P, step (1.3) to generate brand-new verification piecemeal Pnew, use P tables Show former verification piecemeal, when check-node receives the verification difference piecemeal Δ P produced by Data Migration, check-node is with P+ Δs P updates former verification piecemeal；When check-node receives the P produced by new data fillingnewWhen, check-node PnewReplace correspondence Former verification piecemeal.
In embodiments of the present invention, as shown in figure 3, working as check-node N (PC1) receive Δ Pi,1Or N (PC2) receive ΔPi,2When, check-node N (PC1) update PI, 1It is former PI, 1With Δ Pi,1Sum, check-node N (PC2) update PI, 2It is former PI, 2With ΔPi,2Sum, wherein i ∈ { 1,2,3,4 }.When check-node receives the P produced by new data fillingnewWhen, check-node is used PnewReplace original verification piecemeal.In the embodiment shown in fig. 3, when check-node receives the { P of step (1.3.3) generation5,1, P5,2, P6,1, P6,2When, check-node N (PC1)、N(PC1) the 5th, 6 rows verification piecemeal is updated to { P respectively5,1, P6,1And {P5,2, P6,2}。
(2.2) (2.1) are repeated, until all verification piecemeals all complete to update.
In embodiments of the present invention, as shown in figure 3, working as complete verification piecemeal { P1,1, P1,2..., P6,1, P6,2All update During completion, the checksum update process of RS yards of storage cluster just calculates completion.
, with step (2) in the absence of mutually restriction, they carry out that whole dilatation can be accelerated parallel for above-mentioned steps (1.2), (1.3) Process, and when working as step (2) completion, completion is just calculated in the dilatation to RS storage clusters.
As it will be easily appreciated by one skilled in the art that the foregoing is only presently preferred embodiments of the present invention, it is not used to The limitation present invention, all any modification, equivalent and improvement made within the spirit and principles in the present invention etc., all should include Within protection scope of the present invention.

Claims (2)

1. a kind of storage expansion method for RS code storage clusters, it is characterised in that the method is comprised the following steps：
(1) process that the dilatation in cluster-based storage space is redistributed with data, including following sub-step：
(1.1) increase memory node, the capacity of storage cluster is physically increased to new back end is added in storage cluster；
(1.2) redistribution of legacy data piecemeal, including following sub-step：
(1.2.1) reads the logical block number (LBN) x of legacy data piecemeal, then presses row, column number and calculates deblocking coordinate (row0,col0), Wherein row0=x/k represents business of the x divided by k, col0=x%k represents the remainder for seeking x divided by k, and k is the back end of storage cluster Number；
(1.2.2) is according to the deblocking coordinate (row for above calculating0,col0) come judge deblocking whether need migration； Make p=row0%k', k' are represented increases the quantity of total data node that cluster after back end is included, and the standard of judgement is： If 0≤p ＜ k, deblocking need not be migrated, then the address of the deblocking is still (row0,col0)；Otherwise, data point Block needs to be migrated；
It is poor that (1.2.3) calculates verification after the deblocking to be migrated in legacy data node completes migration, by current data node Different piecemeal Δ P is simultaneously sent to check-node；By storage deblocking DCiBack end N (DCi) it is row row deblocking meters Calculate r verification difference piecemeal { Δ Prow,1,ΔProw,2,...,ΔProw,rAnd it is separately sent to r check-node { N (PC1),N (PC2) .., N (PCr)}；Whereinαj,iIt is the i-th row, the element of jth row in redundant matrices, Di,rowIt is the deblocking of (i, row) to represent address, and r represents the number of check-node；
(1.2.4) repeats step (1.2.1), (1.2.2), (1.2.3), until the treatment of whole legacy data piecemeals is completed, makes All deblockings are evenly distributed in storage cluster, complete data redistribution process；
(1.3) filling of new data piecemeal；Including following sub-step：
(1.3.1) calculates its filling destination address (row, col) according to the logical block number (LBN) x of new data piecemeal；Y=x-k × C is made, WhereinCol=y%k', C represent the deblocking quantity that memory node can be accommodated；
New data piecemeal is filled into corresponding filling destination address by (1.3.2)；
(1.3.3) is filled by new data piecemeal if data line piecemeal and completed, then using RS coding generation a line verifications point Block is simultaneously sent to check-node；
(1.3.4) repeats (1.3.1), (1.3.2), (1.3.3) until new data piecemeal is stuffed entirely with completing；
(2) process that corresponding verification piecemeal updates after data redistribution, including following sub-step：
(2.1) that the step (1.3.3) in verification difference piecemeal Δ P, step (1.3) can be produced to generate is brand-new for step (1.2) Verification piecemeal Pnew, former verification piecemeal is represented with P, when check-node receives the verification difference piecemeal Δ P produced by Data Migration When, check-node updates former verification piecemeal with P+ Δs P；When check-node receives the P produced by new data fillingnewWhen, verification Node PnewReplace corresponding former verification piecemeal；
(2.2) (2.1) are repeated, until all verification piecemeals all complete to update.
2. expansion method according to claim 1, it is characterised in that according to transposition formula data in the step (1.2.2) It is laid out to calculate the new address (row, col) of deblocking after dilatation, wherein row=(row0/k')*k'+col0, col= Row%k', k'=k+ Δ k, Δ k are the newly-increased back end numbers of storage cluster, if the transposition formula data layout is by kth+1 Row enters row matrix transposed way and migrates to the Δ k × k of k+ Δ k rows according to piecemeal, then the k of the row of kth+1 is migrated according to piecemeal In being arranged to kth+1, the k of the row of kth+2 according to piecemeal move to kth+2 arrange in ..., the k of kth+Δ k rows moves according to piecemeal In moving on to kth+Δ k row.
CN201410579759.XA 2014-10-24 2014-10-24 A kind of storage expansion method for RS code storage clusters Active CN104407807B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410579759.XA CN104407807B (en) 2014-10-24 2014-10-24 A kind of storage expansion method for RS code storage clusters

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410579759.XA CN104407807B (en) 2014-10-24 2014-10-24 A kind of storage expansion method for RS code storage clusters

Publications (2)

Publication Number Publication Date
CN104407807A CN104407807A (en) 2015-03-11
CN104407807B true CN104407807B (en) 2017-06-27

Family

ID=52645440

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410579759.XA Active CN104407807B (en) 2014-10-24 2014-10-24 A kind of storage expansion method for RS code storage clusters

Country Status (1)

Country Link
CN (1) CN104407807B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104881372A (en) * 2015-05-31 2015-09-02 上海交通大学 Data migration method capable of improving RAID-6 (redundant array of independent disks-6) expandability
CN105260376B (en) * 2015-08-17 2018-08-14 北京京东尚科信息技术有限公司 Method, apparatus and system for clustered node reducing and expansion
CN107547606B (en) * 2016-06-29 2021-01-26 中兴通讯股份有限公司 Data processing method, cluster manager, resource manager and data processing system
WO2018120939A1 (en) * 2016-12-29 2018-07-05 华为技术有限公司 Storage system expanded data migration method and storage system
CN108255413A (en) 2016-12-29 2018-07-06 华为技术有限公司 Method, the storage system of Data Migration after a kind of storage system dilatation
CN106951340B (en) * 2017-03-14 2019-07-09 华中科技大学 A kind of RS correcting and eleting codes data layout method and system preferential based on locality
CN108536396B (en) * 2018-04-08 2020-05-19 华中科技大学 Storage expansion method based on network coding
CN112162702B (en) * 2020-09-25 2024-05-31 科大讯飞股份有限公司 Distributed storage system, virtual machine creation and data volume expansion method
CN112463043B (en) * 2020-11-20 2023-01-10 苏州浪潮智能科技有限公司 Storage cluster capacity expansion method, system and related device
CN114048222A (en) * 2021-11-25 2022-02-15 成都信息工程大学 Dynamic expansion storage cluster method, device, terminal and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103336785A (en) * 2013-06-04 2013-10-02 华中科技大学 Distributed storage method and distributed storage device based on network coding
CN103561057A (en) * 2013-10-15 2014-02-05 深圳清华大学研究院 Data storage method based on distributed hash table and erasure codes

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9183076B2 (en) * 2013-04-19 2015-11-10 Futurewei Technologies, Inc. Using carry-less multiplication (CLMUL) to implement erasure code

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103336785A (en) * 2013-06-04 2013-10-02 华中科技大学 Distributed storage method and distributed storage device based on network coding
CN103561057A (en) * 2013-10-15 2014-02-05 深圳清华大学研究院 Data storage method based on distributed hash table and erasure codes

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
赵浩天.基于网络编码的分布式存储容错及扩容问题研究.《中国博士学位论文全文数据库（电子期刊）信息科技辑》.2013,I137-17. *

Also Published As

Publication number Publication date
CN104407807A (en) 2015-03-11

Similar Documents

Publication Publication Date Title
CN104407807B (en) A kind of storage expansion method for RS code storage clusters
US9026729B1 (en) Data recovery after triple disk failure
CN103336785B (en) A kind of distributed storage method based on network code and device thereof
CN103746774B (en) The fault-tolerant coding method that a kind of efficient data is read
Wang et al. MDR codes: A new class of RAID-6 codes with optimal rebuilding and encoding
CN109684127A (en) Locality node restorative procedure based on complete graph minimum bandwidth regeneration code
CN106484559A (en) A kind of building method of check matrix and the building method of horizontal array correcting and eleting codes
CN103729151B (en) A kind of fail data restorative procedure based on modified model correcting and eleting codes
CN107203442B (en) Method and equipment for evaluating reconstruction performance of redundant array of independent disks
CN112799604A (en) N-Code-based RAID6 disk array capacity expansion method and data filling method
WO2022174537A1 (en) Stripe data storage structure and construction, repair, and updating methods therefor
CN103106124B (en) Intersection reconstruction method based on erasure code cluster memory system
CN107395207A (en) The MDS array codes coding and restorative procedure of more fault-tolerances
CN106788891A (en) A kind of optimal partial suitable for distributed storage repairs code constructing method
CN105356892B (en) The method and system of network code
US10922201B2 (en) Method and device of data rebuilding in storage system
CN110058789B (en) Method for managing storage system, storage system and computer program product
WO2023151290A1 (en) Data encoding method and apparatus, device, and medium
CN106951340A (en) A kind of RS correcting and eleting codes data layout method and system preferential based on locality
CN108536396B (en) Storage expansion method based on network coding
CN108762978B (en) Grouping construction method of local part repeated cyclic code
CN104516679B (en) A kind of RAID data processing method and processing device
CN107153661A (en) A kind of storage, read method and its device of the data based on HDFS systems
CN110058788A (en) Distribute method, electronic equipment, storage system and the computer program product of storage
CN104881365A (en) RAID-6 extensible method based on erasure code similarity

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant