CN104407807A - Storage and expansion method aiming at RS coding storage cluster - Google Patents
Storage and expansion method aiming at RS coding storage cluster Download PDFInfo
- Publication number
- CN104407807A CN104407807A CN201410579759.XA CN201410579759A CN104407807A CN 104407807 A CN104407807 A CN 104407807A CN 201410579759 A CN201410579759 A CN 201410579759A CN 104407807 A CN104407807 A CN 104407807A
- Authority
- CN
- China
- Prior art keywords
- data
- piecemeal
- row
- check
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a high-efficient storage and expansion method aiming at an RS coding storage cluster. The method relates to two processes of data redistribution and check update, and concretely comprises the following steps of increasing delta k data nodes, and redistributing stale data according to a transposition type data layout, wherein original data segmentations only transfer between stale data nodes and new data nodes, and new data segmentations directly fill an empty area after transferring; when data redistribution is completed, carrying out update operation on corresponding check segmentations. The data redistribution supports multiple expansion and keeps the uniformity of data distribution. Since the data redistribution adopts the transposition type data layout, when the check segmentations are updated, the stale data nodes can directly calculate check difference segmentations delta P, and the check nodes utilize the check difference segmentations delta P to complete the update of the check segmentations, so that the check update expense is reduced. According to the storage and expansion method provided by the invention, the storage efficiency can be improved, and the high-efficient expansion can be completed.
Description
Technical field
The invention belongs to Computer Storage field, more specifically, relate to a kind of storage expansion method for RS code storage cluster.
Background technology
Correcting and eleting codes is widely used in distributed storage, and multiple memory node connects according to erasure code mode by cluster-based storage, forms the storage system with certain fault-tolerant ability.Erasure code mainly comprises Reed-Solomon coding (Reed Solomon Codes, RS Codes) and parity matrix coding.Under identical capacity conditions, RS coding has the fault-tolerant ability higher than parity matrix coding.
Along with operation and the use of storage cluster, in cluster, memory capacity will reduce gradually, need the mode by increasing memory node to complete the object of dilatation.Here, cluster dilatation mainly refers to interpolation RS storage cluster being carried out to back end.
Existing disk array expansion scheme is mainly for RAID-0, RAID-4, RAID-5 and RAID-6, and they are all not suitable for cluster-based storage dilatation.RAID-0 only processes the migration of data, does not verify data, and data security is not high.RAID-4 requires there is uniform Data distribution8 on all data disks, and this can cause a large amount of Data Migrations.Although there is a kind of optional RAID-4 expansion scheme, it is by movable part deblocking to new disk reducing the migration of data, and when filling new data, the cost that this scheme upgrades verification piecemeal is very high.RAID-5 adopts the Data distribution8 based on poll, and when being applied to RS storage cluster, due to data mobile and checksum update, cluster will face the too high problem of I/O expense.The expansion of RAID-6, only for specific RAID-6 code Design, is not suitable for correcting and eleting codes storage cluster.
Summary of the invention
For above defect or the Improvement requirement of prior art, the invention provides a kind of storage expansion method for Reed-Solomon code storage cluster, not only can improve the memory capacity of storage cluster, and energy minimization Data Migration amount and checksum update expense.
In order to realize foregoing invention object, the invention provides a kind of storage expansion method for RS code storage cluster, comprising the steps:
(1) process that redistributes of the dilatation in cluster-based storage space and data, comprises following sub-step:
(1.1) increase memory node, in storage cluster, add the capacity that new back end increases storage cluster physically;
(1.2) redistribution of legacy data piecemeal, comprises following sub-step:
(1.2.1) read the logical block number (LBN) x of legacy data piecemeal, then press row, column number and calculate deblocking coordinate (row
0, col
0), wherein row
0=x/k represents the business of x divided by k, col
0=x%k represents the remainder asking x divided by k, and k is the back end number of storage cluster;
(1.2.2) according to the deblocking coordinate (row that previous calculations goes out
0, col
0) judge that deblocking is the need of migration; Make p=row
0%k', the standard of judgement is: if 0≤p < is k, and deblocking does not need migration, then the address of this deblocking is still (row
0, col
0); Otherwise this deblocking needs to move;
(1.2.3) after the deblocking to be migrated in legacy data node completes migration, verify difference piecemeal Δ P by current data node calculate and be sent to check-node; By store data piecemeal DC
inode N (DC
i) be row capable data section technique r verification difference piecemeal { Δ P
row, 1, Δ P
row, 2..., Δ P
row, rand be sent to r check-node { N (PC respectively
1), N (PC
2) .., N (PC
r); Wherein
α
j,ithe element of the i-th row in redundant matrices, jth row, D
i, rowrepresent the deblocking that address is (i, row), r represents the number of check-node;
(1.2.4) repeated execution of steps (1.2.1), (1.2.2), (1.2.3), until all the process of legacy data piecemeal completes, make all deblockings be evenly distributed in storage cluster, complete data redistribution process;
(1.3) filling of new data piecemeal; Comprise following sub-step:
(1.3.1) calculate it according to the logical block number (LBN) x of new data piecemeal and fill destination address (row, col); Make y=x-k × C, wherein
col=y%k', C represent the deblocking quantity that memory node can hold;
(1.3.2) new data piecemeal is filled into corresponding filling destination address;
If (1.3.3) there is data line piecemeal to be filled by new data piecemeal, then utilizes RS to encode and generate verification piecemeal and be sent to check-node;
(1.3.4) (1.3.1), (1.3.2), (1.3.3) is repeated until new data piecemeal has all been filled;
(2) process that after data redistribution, corresponding verification piecemeal upgrades, comprises following sub-step:
(2.1) step (1.2) can produce verification difference piecemeal Δ P, step (1.3) can generate brand-new verification piecemeal P
new, represent former verification piecemeal with P, when check-node receives the verification difference piecemeal Δ P produced by Data Migration, check-node upgrades former verification piecemeal with P+ Δ P; Fill by new data the P produced when check-node receives
newtime, check-node P
newreplace corresponding former verification piecemeal;
(2.2) repeat (2.1), until all verification piecemeals all complete renewal.
Further, the new address (row, col) of deblocking after dilatation is calculated in described step (1.2.2) according to transposition formula data layout, wherein row=(row
0/ k') * k'+col
0, col=row%k', k'=k+ Δ k, Δ k are that storage cluster increases back end number newly.In general, the above technical method conceived by the present invention compared with prior art, can obtain following beneficial effect:
1, method of the present invention redistributes data according to transposition formula data layout, realizes being uniformly distributed of minimum Data Migration and data.
2, method of the present invention the renewal of verification piecemeal is adopted read-revise-write local formula update mode, directly generate verification difference piecemeal by legacy data node, the transmission of legacy data piecemeal when avoiding checksum update, thus minimize verification piecemeal and upgrade expense.
3, method asynchronous system of the present invention performs Data Migration and checksum update, accelerates the carrying out of storage cluster dilation process.
Accompanying drawing explanation
Fig. 1 is (k+r, k) RS code storage cluster data layout schematic diagram;
In Fig. 2, (a) is data layout's change schematic diagram before and after cluster dilatation, and (b) is transposition formula data layout schematic diagram;
Fig. 3 is Data Migration schematic diagram in the embodiment of the present invention;
Fig. 4 is the inventive method overview flow chart;
Fig. 5 is legacy data redistribution process flow diagram in the present invention;
Fig. 6 is that in the present invention, new data fills process flow diagram;
Fig. 7 is checksum update process flow diagram in the present invention.
Embodiment
In order to make object of the present invention, technical scheme and advantage clearly understand, below in conjunction with drawings and Examples, the present invention is further elaborated.Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.In addition, if below in described each embodiment of the present invention involved technical characteristic do not form conflict each other and just can mutually combine.
Below first just technical term of the present invention is explained and illustrated:
The data layout of RS code storage cluster: as shown in Figure 1, (k+r, k) RS code storage cluster comprises k according to node { N (DC
1), N (DC
2) ..., N (DC
k) and r check-node { N (PC
1), N (PC
2) ..., N (PC
r), each node comprises many deblockings, can be referred to as row, such as, and N (DC
1) comprise deblocking { D
1,1, D
2,1... } and first row, the N (PC of representative data piecemeal
1) comprise deblocking { P
1,1, P
2,1... } and represent the first row verifying piecemeal, wherein { D
1,1, D
1,2..., D
1, kconstitute the first row deblocking, { P
1,1, P
1,2..., P
1, rconstitute the first row verification piecemeal.The individual data node table that cluster dilatation increases Δ k newly shows for { N (DC
k+1), N (DC
k+2) ..., N (DC
k+ Δ k).
Transposition formula data layout: as shown in Figure 2, the k of the capable k row of k before in Fig. 2 (a)
2individual deblocking is nonmigratory data, and kth+1 row needs to move according to the matrix transpose mode shown in Fig. 2 (b) to Δ k × k that k+ Δ k is capable according to piecemeal, such as, kth+1 row k according to piecemeal move to kth+1 arrange in, the k of kth+2 row according to piecemeal move to kth+2 arrange in ..., the capable k of kth+Δ k moves to during kth+Δ k arranges according to piecemeal.Δ k × the k vacated after migration according to piecemeal for storing new data piecemeal.
In order to simplicity of exposition, below unified explanation will be carried out herein to some variablees such as k, Δ k, k' etc. in formula involved in the explanation of embodiment, and do not lay down a definition again in the concrete steps of embodiment.Here is that the implication of each variable illustrates: x represents the logical block number (LBN) of deblocking, and k represents the former back end number of storage cluster, and Δ k represents newly-increased back end number, the quantity of the total data node that cluster comprises after k'=k+ Δ k represents increase back end; α
j,irepresent the element of the i-th row in redundant matrices, jth row; C represents and the deblocking quantity that memory node can hold is equal to k' in an embodiment.In addition, in embodiment the logical block number (LBN) of deblocking comprise 0,1 ..., 35}, wherein logical block 0-23 belongs to legacy data piecemeal, and 24-35 belongs to new data piecemeal.
In embodiments of the present invention, from (6,4) RS storage cluster dilatation be the whole flow process of (8,6) RS storage cluster as shown in Figure 4, mainly comprise the following steps:
(1) process that redistributes of the dilatation in cluster-based storage space and data.Comprise following sub-step:
(1.1) memory node is increased.The capacity that new back end adds storage cluster is physically added in storage cluster.
In the embodiment of the present invention, as shown in Figure 3, (6,4) RS storage cluster originally had 4 back end { N (DC
1), N (DC
2), N (DC
3), N (DC
4), wherein back end N (DC
i) comprise 6 deblocking { D
1, i, D
2, i..., D
6, i; And 2 check-node { N (PC
1), N (PC
2), wherein N (PC
1) comprise 6 verification piecemeal { P
1, i, P
2, i..., P
6, i, now increase by two back end { N (DC
5), N (DC
6) expand the capacity of storage cluster.
(1.2) redistribution of legacy data piecemeal.The flow process of this step as shown in Figure 5, comprises the steps:
(1.2.1) read the logical block number (LBN) x of legacy data piecemeal, then press row, column number and calculate deblocking coordinate (row
0, col
0), wherein row
0=x/k represents the business of x divided by k, col
0=x%k represents the remainder asking x divided by k, and k is the back end number of storage cluster.
In the embodiment of the present invention, follow the convention of matrix from the 1st row, the 1st row, all 1 is added at the row and column coordinate of deblocking address when representing deblocking in figure 3, such as, the logical block number (LBN) of front 16 deblockings is 0 to 15, calculate the address of logical block 0 for (0,0), after row and column coordinate all adds 1, be expressed as D in figure 3
1,1, logical block 1,2 ..., 15} is expressed as { D
1,2..., D
4,4.
(1.2.2) according to the deblocking coordinate (row that previous calculations goes out
0, col
0) judge that deblocking is the need of migration.Make p=row0%k', the standard of judgement is: if 0≤p < is k, and deblocking does not need migration, then the address of this deblocking is still (row
0, col
0); Otherwise this deblocking needs to move.The new address (row, col) of deblocking after dilatation is calculated, wherein row=(row according to transposition formula data layout
0/ k') * k'+col
0, col=row%k', k'=k+ Δ k, Δ k are that storage cluster increases back end number newly.
In embodiments of the present invention, k=4, Δ k=2, k'=6, the actual row-coordinate of the deblocking of front 4 row is 0,1,2,3 respectively, and they are all less than 4 divided by the remainder of 6; The actual row-coordinate of the deblocking of rear 2 row is 4,5, and they are all more than or equal to 4 divided by the remainder of 6.Reach a conclusion according to criterion: the deblocking { D of front 4 row
1,1..., D
4,4do not need migration, the deblocking { D of rear 2 row
5,1..., D
6,4need migration.For deblocking { D
5,1..., D
6,4, by formula row=(row
0/ k') * k'+col
0, the move target address that col=row%k' calculates be respectively (1,5), (2,5) ..., (4,6) }.As shown in Figure 3, deblocking { D
5,1, D
5,2..., D
6,4move to { D
1,5, D
2,5..., D
4,6.
(1.2.3) after the deblocking to be migrated in legacy data node completes migration, verify difference piecemeal Δ P by current data node calculate and be sent to check-node.By store data piecemeal DC
inode N (DC
i) be row capable data section technique r verification difference piecemeal { Δ P
row, 1, Δ P
row, 2..., Δ P
row, rand be sent to r check-node { N (PC respectively
1), N (PC
2) .., N (PC
r).Wherein
α
j,ibe in redundant matrices i-th, the element of jth row, D
i, rowrepresent the deblocking that address is (i, row), r represents the number of check-node.
In embodiments of the present invention, as shown in Figure 3, as deblocking { D
5, i, D
6, icomplete migration after, by N (DC
i) calculate Δ P
i, 1with Δ P
i, 2and they are sent to N (PC respectively
1), N (PC
2), wherein Δ P
i, 1=α
1,5dC
5, i+ α
1,6dC
6, i, Δ P
i, 2=α
2,5dC
5, i+ α
2,6dC
6, i, i ∈ { 1,2,3,4}.
(1.2.4) repeated execution of steps (1.2.1), (1.2.2), (1.2.3), until all the process of legacy data piecemeal completes, make all deblockings be evenly distributed in storage cluster, complete legacy data redistribution process.
In embodiments of the present invention, as shown in Figure 3, when processing N (DC
4) time, the redistribution of legacy data completes.
(1.3) filling of new data piecemeal.The flow process of this step as shown in Figure 6, comprises the steps:
(1.3.1) calculate it according to the logical block number (LBN) x of new data piecemeal and fill destination address (row, col).Make y=x-k × C, wherein
col=y%k', C=k'.
In embodiments of the present invention, as shown in Figure 3, calculate logical block number (LBN) for 24,25 ..., the filling destination address of the deblocking of 29} is { D respectively
5,1, D
5,2..., D
6,4.
(1.3.2) new data piecemeal is filled into corresponding filling destination address.
In embodiments of the present invention, as shown in Figure 3, by logical block 24,25 ..., 29} is filled into successively { D
5,1, D
5,2..., D
6,4.
If (1.3.3) there is data line piecemeal to be filled by new data piecemeal, then utilizes RS to encode and generate verification piecemeal and be sent to check-node.
In embodiments of the present invention, as shown in Figure 3, when the 5th row 6 deblockings by logical block 24,25 ..., when 29} has filled, utilize RS encryption algorithm to generate verification piecemeal P
5,1, P
5, 2and be sent to check-node { N (PC respectively
1), N (PC
2), when the 6th row by deblocking 30,31 ..., when 35} has filled, utilize RS encryption algorithm to generate verification piecemeal P
6,1, P
6,2and be sent to check-node { N (PC respectively
1), N (PC
2).
(1.3.4) repeated execution of steps (1.3.1), (1.3.2), (1.3.3) are until new data piecemeal has all been filled.
In embodiments of the present invention, in figure 3, when the 6th row deblocking by logical block 30,31 ..., 35} fills, and verifies piecemeal P
6,1, P
6,2and send check-node { N (PC respectively to
1), N (PC
2) time, new data is filled and is all completed.
(2) process that after data redistribution, corresponding verification piecemeal upgrades.The flow process of this step as shown in Figure 7, comprises the steps:
(2.1) when check-node have received the difference verification piecemeal Δ P produced by Data Migration, make former verification piecemeal be P, check-node P+ Δ P completes the renewal to verification piecemeal.
Step (1.2) can produce verification difference piecemeal Δ P, step (1.3) can generate brand-new verification piecemeal P
new, represent former verification piecemeal with P, when check-node receives the verification difference piecemeal Δ P produced by Data Migration, check-node upgrades former verification piecemeal with P+ Δ P; Fill by new data the P produced when check-node receives
newtime, check-node P
newreplace corresponding former verification piecemeal.
In embodiments of the present invention, as shown in Figure 3, as check-node N (PC
1) receive Δ P
i, 1or N (PC
2) receive Δ P
i, 2time, check-node N (PC
1) upgrade P
i, 1for former P
i, 1with Δ P
i, 1sum, check-node N (PC
2) upgrade P
i, 2for former P
i, 2with Δ P
i, 2sum, wherein i ∈ { 1,2,3,4}.Fill by new data the P produced when check-node receives
newtime, check-node P
newreplace original verification piecemeal.In the embodiment shown in fig. 3, when check-node receives the { P that step (1.3.3) generates
5, 1, P
5,2, P
6,1, P
6,2time, check-node N (PC
1), N (PC
1) respectively the 5th, 6 row verification piecemeals are updated to { P
5,1, P
6,1and { P
5,2, P
6,2.
(2.2) repeat (2.1), until all verification piecemeals all complete renewal.
In embodiments of the present invention, as shown in Figure 3, as complete verification piecemeal { P
1,1, P
1,2..., P
6,1, P
6,2when all having upgraded, the checksum update process of RS code storage cluster just completes.
There is not mutual restriction with step (2) in above-mentioned steps (1.2), (1.3), they walk abreast and carry out accelerating whole dilation process, and when step (2) completes, just complete the dilatation of RS storage cluster.
Those skilled in the art will readily understand; the foregoing is only preferred embodiment of the present invention; not in order to limit the present invention, all any amendments done within the spirit and principles in the present invention, equivalent replacement and improvement etc., all should be included within protection scope of the present invention.
Claims (2)
1., for a storage expansion method for RS code storage cluster, it is characterized in that, the method comprises the following steps:
(1) process that redistributes of the dilatation in cluster-based storage space and data, comprises following sub-step:
(1.1) increase memory node, in storage cluster, add the capacity that new back end increases storage cluster physically;
(1.2) redistribution of legacy data piecemeal, comprises following sub-step:
(1.2.1) read the logical block number (LBN) x of legacy data piecemeal, then press row, column number and calculate deblocking coordinate (row
0, col
0), wherein row
0=x/k represents the business of x divided by k, col
0=x%k represents the remainder asking x divided by k, and k is the back end number of storage cluster;
(1.2.2) according to the deblocking coordinate (row that previous calculations goes out
0, col
0) judge that deblocking is the need of migration; Make p=row0%k', the standard of judgement is: if 0≤p < is k, and deblocking does not need migration, then the address of this deblocking is still (row
0, col
0); Otherwise this deblocking needs to move;
(1.2.3) after the deblocking to be migrated in legacy data node completes migration, verify difference piecemeal Δ P by current data node calculate and be sent to check-node; By store data piecemeal DC
inode N (DC
i) be row capable data section technique r verification difference piecemeal { Δ P
row, 1, Δ P
row, 2..., Δ P
row, rand be sent to r check-node { N (PC respectively
1), N (PC
2) .., N (PC
r); Wherein
α
j,ithe element of the i-th row in redundant matrices, jth row, D
i, rowrepresent the deblocking that address is (i, row), r represents the number of check-node;
(1.2.4) repeated execution of steps (1.2.1), (1.2.2), (1.2.3), until all the process of legacy data piecemeal completes, make all deblockings be evenly distributed in storage cluster, complete data redistribution process;
(1.3) filling of new data piecemeal; Comprise following sub-step:
(1.3.1) calculate it according to the logical block number (LBN) x of new data piecemeal and fill destination address (row, col); Make y=x-k × C, wherein
col=y%k', C represent the deblocking quantity that memory node can hold;
(1.3.2) new data piecemeal is filled into corresponding filling destination address;
If (1.3.3) there is data line piecemeal to be filled by new data piecemeal, then utilizes RS to encode and generate verification piecemeal and be sent to check-node;
(1.3.4) (1.3.1), (1.3.2), (1.3.3) is repeated until new data piecemeal has all been filled;
(2) process that after data redistribution, corresponding verification piecemeal upgrades, comprises following sub-step:
(2.1) step (1.2) can produce verification difference piecemeal Δ P, step (1.3) can generate brand-new verification piecemeal P
new, represent former verification piecemeal with P, when check-node receives the verification difference piecemeal Δ P produced by Data Migration, check-node upgrades former verification piecemeal with P+ Δ P; Fill by new data the P produced when check-node receives
newtime, check-node P
newreplace corresponding former verification piecemeal;
(2.2) repeat (2.1), until all verification piecemeals all complete renewal.
2. expansion method according to claim 1, it is characterized in that, the new address (row, col) of deblocking after dilatation is calculated according to transposition formula data layout, wherein row=(row0/k') * k'+col in described step (1.2.2)
0, col=row%k', k'=k+ Δ k, Δ k are that storage cluster increases back end number newly.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410579759.XA CN104407807B (en) | 2014-10-24 | 2014-10-24 | A kind of storage expansion method for RS code storage clusters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410579759.XA CN104407807B (en) | 2014-10-24 | 2014-10-24 | A kind of storage expansion method for RS code storage clusters |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104407807A true CN104407807A (en) | 2015-03-11 |
| CN104407807B CN104407807B (en) | 2017-06-27 |
Family
ID=52645440
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410579759.XA Active CN104407807B (en) | 2014-10-24 | 2014-10-24 | A kind of storage expansion method for RS code storage clusters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104407807B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104881372A (en) * | 2015-05-31 | 2015-09-02 | 上海交通大学 | Data migration method capable of improving RAID-6 (redundant array of independent disks-6) expandability |
| CN105260376A (en) * | 2015-08-17 | 2016-01-20 | 北京京东尚科信息技术有限公司 | Method, equipment and system used for cluster node contraction and expansion |
| CN106951340A (en) * | 2017-03-14 | 2017-07-14 | 华中科技大学 | A kind of RS correcting and eleting codes data layout method and system preferential based on locality |
| WO2018001200A1 (en) * | 2016-06-29 | 2018-01-04 | 中兴通讯股份有限公司 | Data processing method, cluster manager, resource manager and data processing system |
| WO2018120939A1 (en) * | 2016-12-29 | 2018-07-05 | 华为技术有限公司 | Storage system expanded data migration method and storage system |
| CN108536396A (en) * | 2018-04-08 | 2018-09-14 | 华中科技大学 | A kind of storage extended method based on network code |
| CN112162702A (en) * | 2020-09-25 | 2021-01-01 | 科大讯飞股份有限公司 | Distributed storage system, virtual machine creation method and data volume expansion method |
| US10908833B2 (en) | 2016-12-29 | 2021-02-02 | Huawei Technologies Co., Ltd. | Data migration method for a storage system after expansion and storage system |
| CN112463043A (en) * | 2020-11-20 | 2021-03-09 | 苏州浪潮智能科技有限公司 | Storage cluster capacity expansion method, system and related device |
| CN114048222A (en) * | 2021-11-25 | 2022-02-15 | 成都信息工程大学 | Dynamic expansion storage cluster method, device, terminal and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103336785A (en) * | 2013-06-04 | 2013-10-02 | 华中科技大学 | Distributed storage method and distributed storage device based on network coding |
| CN103561057A (en) * | 2013-10-15 | 2014-02-05 | 深圳清华大学研究院 | Data storage method based on distributed hash table and erasure codes |
| US20140317162A1 (en) * | 2013-04-19 | 2014-10-23 | Futurewei Techologies, Inc. | Using Carry-less Multiplication (CLMUL) to Implement Erasure Code |
-
2014
- 2014-10-24 CN CN201410579759.XA patent/CN104407807B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140317162A1 (en) * | 2013-04-19 | 2014-10-23 | Futurewei Techologies, Inc. | Using Carry-less Multiplication (CLMUL) to Implement Erasure Code |
| CN103336785A (en) * | 2013-06-04 | 2013-10-02 | 华中科技大学 | Distributed storage method and distributed storage device based on network coding |
| CN103561057A (en) * | 2013-10-15 | 2014-02-05 | 深圳清华大学研究院 | Data storage method based on distributed hash table and erasure codes |
Non-Patent Citations (1)
| Title |
|---|
| 赵浩天: "基于网络编码的分布式存储容错及扩容问题研究", 《中国博士学位论文全文数据库（电子期刊）信息科技辑》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104881372A (en) * | 2015-05-31 | 2015-09-02 | 上海交通大学 | Data migration method capable of improving RAID-6 (redundant array of independent disks-6) expandability |
| CN105260376A (en) * | 2015-08-17 | 2016-01-20 | 北京京东尚科信息技术有限公司 | Method, equipment and system used for cluster node contraction and expansion |
| CN105260376B (en) * | 2015-08-17 | 2018-08-14 | 北京京东尚科信息技术有限公司 | Method, apparatus and system for clustered node reducing and expansion |
| WO2018001200A1 (en) * | 2016-06-29 | 2018-01-04 | 中兴通讯股份有限公司 | Data processing method, cluster manager, resource manager and data processing system |
| US10908833B2 (en) | 2016-12-29 | 2021-02-02 | Huawei Technologies Co., Ltd. | Data migration method for a storage system after expansion and storage system |
| WO2018120939A1 (en) * | 2016-12-29 | 2018-07-05 | 华为技术有限公司 | Storage system expanded data migration method and storage system |
| CN106951340A (en) * | 2017-03-14 | 2017-07-14 | 华中科技大学 | A kind of RS correcting and eleting codes data layout method and system preferential based on locality |
| CN106951340B (en) * | 2017-03-14 | 2019-07-09 | 华中科技大学 | A kind of RS correcting and eleting codes data layout method and system preferential based on locality |
| CN108536396A (en) * | 2018-04-08 | 2018-09-14 | 华中科技大学 | A kind of storage extended method based on network code |
| CN108536396B (en) * | 2018-04-08 | 2020-05-19 | 华中科技大学 | Storage expansion method based on network coding |
| CN112162702A (en) * | 2020-09-25 | 2021-01-01 | 科大讯飞股份有限公司 | Distributed storage system, virtual machine creation method and data volume expansion method |
| CN112162702B (en) * | 2020-09-25 | 2024-05-31 | 科大讯飞股份有限公司 | Distributed storage system, virtual machine creation and data volume expansion method |
| CN112463043A (en) * | 2020-11-20 | 2021-03-09 | 苏州浪潮智能科技有限公司 | Storage cluster capacity expansion method, system and related device |
| CN112463043B (en) * | 2020-11-20 | 2023-01-10 | 苏州浪潮智能科技有限公司 | Storage cluster capacity expansion method, system and related device |
| CN114048222A (en) * | 2021-11-25 | 2022-02-15 | 成都信息工程大学 | Dynamic expansion storage cluster method, device, terminal and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104407807B (en) | 2017-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104407807A (en) | Storage and expansion method aiming at RS coding storage cluster | |
| US9509342B2 (en) | Error correcting code decoder | |
| US9026729B1 (en) | Data recovery after triple disk failure | |
| EP3399419B1 (en) | Method and system for multi-dimensional raid | |
| WO2020005336A1 (en) | Multi-device storage system with distributed read/write processing | |
| CN105630423A (en) | Erasure code cluster storage expansion method based on data caching | |
| US20120030425A1 (en) | Parity declustered storage device array with partition groups | |
| CN112799604B (en) | N-Code-based RAID6 disk array capacity expansion method and data filling method | |
| US20150160999A1 (en) | Solid state drive controller, solid state drive, data processing method of solid state drive, multi-channel solid state drive, raid controller and computer-readable recording medium having recorded therein computer program for providing sequence information to solid state drive | |
| CN105468534A (en) | Method and system for improving flash storage utilization using read-threshold tables | |
| CN107203442B (en) | Method and equipment for evaluating reconstruction performance of redundant array of independent disks | |
| CN102122235A (en) | RAID4 (redundant array of independent disks) system and data reading and writing method thereof | |
| CN106484559A (en) | A kind of building method of check matrix and the building method of horizontal array correcting and eleting codes | |
| US10922201B2 (en) | Method and device of data rebuilding in storage system | |
| CN110058789B (en) | Method for managing storage system, storage system and computer program product | |
| CN103229243A (en) | Memory controller and system for storing blocks of data in non-olatile memory devices in redundant manner | |
| CN108536396B (en) | Storage expansion method based on network coding | |
| CN109684127A (en) | Locality node restorative procedure based on complete graph minimum bandwidth regeneration code | |
| CN114816837A (en) | Erasure code fusion method and system, electronic device and storage medium | |
| CN105811996A (en) | Data processing method and system based on quasi-cyclic LDCP | |
| CN104881365A (en) | RAID-6 extensible method based on erasure code similarity | |
| US9524213B2 (en) | Storage control apparatus, recording medium having stored therein storage control program and storage control method | |
| CN108268339A (en) | The method and system of estimation is counted for online programming/erasing | |
| CN110032338B (en) | Erasure code oriented data copy placement method and system | |
| WO2016039686A1 (en) | Method of encoding data and data storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |