CN106293526A - The expandable method of a kind of three dish fault-tolerant arrays and system - Google Patents

The expandable method of a kind of three dish fault-tolerant arrays and system Download PDF

Info

Publication number
CN106293526A
CN106293526A CN201610637554.1A CN201610637554A CN106293526A CN 106293526 A CN106293526 A CN 106293526A CN 201610637554 A CN201610637554 A CN 201610637554A CN 106293526 A CN106293526 A CN 106293526A
Authority
CN
China
Prior art keywords
band
block
verification
check block
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201610637554.1A
Other languages
Chinese (zh)
Other versions
CN106293526B (en
Inventor
吴晨涛
过敏意
蒋妍冰
黄洵松
李璐雨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University
Original Assignee
Shanghai Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University filed Critical Shanghai Jiaotong University
Priority to CN201610637554.1A priority Critical patent/CN106293526B/en
Publication of CN106293526A publication Critical patent/CN106293526A/en
Application granted granted Critical
Publication of CN106293526B publication Critical patent/CN106293526B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0689—Disk arrays, e.g. RAID, JBOD

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Detection And Correction Of Errors (AREA)
  • Techniques For Improving Reliability Of Storages (AREA)

Abstract

The invention discloses expandable method and the system of a kind of three dish fault-tolerant arrays, the method comprises the steps: the preparatory stage, collects relevant parameter from storage system, calculates necessary parameter for ensuing two stages；Migration phase, selects a part of band as band collection, and according to the parameter calculated in the preparatory stage, selects some data row in each band, is used in the disk migrating to newly add；Merging phase, by empty band polymerization, by the invention it is possible to data distribution is equilibrium after guaranteeing extension, the I/O expense in expansion process is less.

Description

The expandable method of a kind of three dish fault-tolerant arrays and system
Technical field
The present invention relates to big data and cloud storage field, particularly relate to a kind of three dish fault-tolerant arrays based on xor operation Expandable method and system.
Background technology
In large-scale data center, mass data storage is in bulk redundancy disk, and user can be with concurrent access number therein According to.These redundancy magnetic disks are referred to as RAID (RAID).Recently, three dish fault-tolerant arrays (3DFTs) are at large-scale number According to the most popular, because they can provide high reliability and low cost expense in the heart.
In past 20 years, scientist proposes many correcting and eleting codes and reaches the effect that three dishes are fault-tolerant, including ultimate range Separable code (MDS) and non-MDS coding.MDS coding is under given a number of redundancy, it is provided that maximum data are protected Protect.Some coding, as STAR encode, Triple-Star encode, TIP-Code coding and EH-Code coding coded system by Three kinds of different verification (row verification, tiltedly verification, the most tiltedly verification) compositions, thus high reliability is provided.Non-MDS coding due to Sacrifice part storage efficiency, higher performance or reliability can be gone all out with.
Recently, along with the development of cloud computing, the extensibility of disk array is by extensive discussions.This is because:
1, in the three dish fault-tolerant arrays encoded based on MDS, increase extra disk and can improve storage efficiency, reduce into This cost.
2, the disk array newly adding disk is had can to provide higher I/O handling capacity and bigger capacity.More number According to accessing concurrently, decrease I/O expense, improve the performance of whole system.
3, three dish fault-tolerant arrays are widely used in large-scale data center, are because it and have high reliability.Reliability exists Large-scale data center plays important role.
Unfortunately, existing extended method is poorly suited for use in three dish fault-tolerant arrays based on xor operation.One (such as Round-Robin (RR) and Semi-RR) being proved to, in expansion process, they are in Data Migration and school for a little methods Running repair creates substantial amounts of I/O expense on changing.Semi-RR also suffers from unbalanced problem of data distribution after extension simultaneously.Its His method in order under the layout of this complexity of correcting and eleting codes equilibrium criterion distribution take substantial amounts of I/O, such as SDM and RS6.SDM Sacrificing a part of band so that concentrate at a band, data distribution is equilibrium.It can locate two kinds of different check of reason The coding (but having three kinds of different verifications in three dish fault-tolerant encodings based on xor operation) of coding, this can cause verification Amendment and I/O expense are the highest.RS6 is a kind of RAID-6 extended method for RDP coding, and it can cause extra Data Migration Expense.Under some encode, after RS6 is to coding extension so that data skewness weighs, as STAR coding and TIP-Code compile Code.Some other method, such as FastScale, GSR and CRAID, they are for RAID-0 or RAID-5 array, it is impossible to fortune Use three dish fault-tolerant arrays.
Visible, existing extended method is all not suitable for the extension of three dish fault-tolerant arrays based on xor operation, therefore, has in fact A kind of technological means of necessary proposition, to solve the problems referred to above.
Summary of the invention
For the deficiency overcoming above-mentioned prior art to exist, what the purpose of the present invention was to provide a kind of three dish fault-tolerant arrays can Extended method and system, it can apply on three dish fault-tolerant arrays based on xor operation so that the data distribution after extension It it is equilibrium.
For reaching above and other purpose, the present invention proposes the expandable method of a kind of three dish fault-tolerant arrays, including walking as follows Rapid:
Preparatory stage, from storage system, collect relevant parameter, calculate necessary parameter for ensuing two stages；
Migration phase, select a part of band as band collection, and according to the parameter calculated in the preparatory stage, often Individual band is selected some data row, is used in the disk migrating to newly add；
Merging phase, by empty band polymerization.
Further, in the preparatory stage, the layout encoded according to MDS in three dish fault-tolerant arrays and its configuration parameter, meter Calculate ns, neAnd L, it is provided that to two later stages, wherein, nsRepresent the band number that a band is concentrated, neRepresent at this Band concentrates the total quantity being arranged to null, and L represents in a band, the minimum number of null.
Further, migration phase comprises the steps:
Band collection is read in caching；
Minimal modifications amount according to check block, migrates suitable data block；
Update and recalculate check block.
Further, the data block of whole band collection and check block are disposably read in the caching X being positioned at internal memory, band collection In selected null be placed into caching Y in.
Further, the described minimal modifications amount according to check block, the step migrating suitable data block is to row After verification, tiltedly verification, the relation of the most tiltedly verification are analyzed, change the position of some data blocks so that these blocks still exist In the verification chain that they are original.
Further, described renewal and recalculate in check block step, by calculating raw line check block and Xin Jia The XOR of data block and Lai Geng newline check block.
Further, described renewal and recalculate in check block step, updating tiltedly verification needs to calculate original oblique verification Block, data block in relevant null and the XOR of row check block updated accordingly and.
Further, described renewal and recalculate in check block step, recalculate according to the coding formula of correcting and eleting codes The most tiltedly check block.
Further, described merging phase comprises the steps:
Determine which band needs merged or is split so that overall computing cost is minimum；
Concentrate at band or between band collection, merge band.
For reaching above-mentioned purpose, the present invention also provides for the expansible system of a kind of three dish fault-tolerant arrays, including:
Preparatory unit, collects relevant parameter from storage system, provides necessity for migration units and combining unit Parameter；
Migration units, select a part of band as band collection, and according to the parameter calculated in the stage one, each Selecting some data row in band, be used in the disk migrating to newly add, in transition process, the movement of data block is by difference Association between verification is determined；
Combining unit, for by empty band polymerization.
Compared with prior art, the expandable method of a kind of three dish fault-tolerant arrays of the present invention and system are according to different check Relatedness, selects suitable data block to migrate, thus the Data Migration minimized and the quantity of verification amendment, meanwhile, The band with sky data block is merged by the present invention so that the I/O cost reduction of verification amendment, therefore the present invention can be true After protecting extension, data distribution is equilibrium, and the I/O expense in expansion process is less.
Accompanying drawing explanation
Fig. 1 is the flow chart of steps of the expandable method of a kind of three dish fault-tolerant arrays of the present invention；
Fig. 2 is the overall construction drawing of the method；
The example of Fig. 3 the method, represents and extends the band 0 that front/rear band is concentrated；
The example of Fig. 4 the method, represents and extends the band 1 that front/rear band is concentrated；
The example of Fig. 5 the method, represents and extends the band 2 that front/rear band is concentrated；
The example of Fig. 6 the method, represents the merging process of the band 0 and 1 of band concentration；
Fig. 7 is the system architecture diagram of the expansible system of a kind of three dish fault-tolerant arrays of the present invention.
Detailed description of the invention
Below by way of specific instantiation accompanying drawings embodiments of the present invention, those skilled in the art can Further advantage and effect of the present invention is understood easily by content disclosed in the present specification.The present invention also can be different by other Instantiation implemented or applied, the every details in this specification also can based on different viewpoints and application, without departing substantially from Various modification and change is carried out under the spirit of the present invention.
Fig. 1 is the flow chart of steps of the expandable method of a kind of three dish fault-tolerant arrays of the present invention.As it is shown in figure 1, the present invention The expandable method of a kind of three dish fault-tolerant arrays, comprises the steps:
Step 101, stage one, i.e. preparatory stage: collect relevant parameter from storage system, for ensuing two rank Section calculates necessary parameter.
Step 102, stage two, i.e. migration phase: select a part of band as band collection, and fall into a trap according to the stage one The parameter calculated, selects some data row in each band, is used in the disk migrating to newly add.In transition process, number According to the movement of block be by different check between association determined, this can ensure that equilibrium data distribution.
Step 103, stage three, i.e. merging phase: be aggregating by empty band, reduces quantity and the I/O of verification amendment Expense.
Fig. 2 is the overall construction drawing of the preferred embodiment of the expandable method of a kind of three dish fault-tolerant arrays of the present invention, above Left, center, right part in three stages corresponding diagram 2 respectively, Fig. 3 is the example of the method, represents and extends the band that front/rear band is concentrated 0；Fig. 4 is the example of the method, represents and extends the band 1 that front/rear band is concentrated；Fig. 5 is the example of the method, represents extension The band 2 that front/rear band is concentrated；Fig. 6 is the example of the method, represents the merging process of the band 0 and 1 of band concentration.Below The present invention is further illustrated by concrete example by coordinating Fig. 2-Fig. 6:
One, the preparatory stage
In this stage, the parameter of necessity is calculated for ensuing two steps.
The layout encoded according to MDS in three dish fault-tolerant arrays and its configuration parameter, can calculate n by below equations, neAnd L, wherein, nsRepresent the band number that a band is concentrated, neRepresent the sum being arranged to null in this band concentration Amount, L represents in a band, the minimum number of null.According to Fig. 2, a band collection contains many bands.Each In band, partial row is set as " null ", and in order to reach data balancing, they can be used to move to other dishes.
n s = l c m { m × n r , n d + m } m × n r
n e = l c m { m × n r , n d + m } n d + m
L = 1 m × n r ≤ n d + m f l o o r ( m × n r n d + m ) m × n r > n d + m
By the three dish fault-tolerant arrays that encode based on Triple-Star as a example by 7 pieces of disks extend to 9 pieces of disks.nd=4, nr =4 and m=2, wherein, ndRepresent the quantity of data disks, n in extension front disk arrayrRepresent the line number extended in previous band Amount, m represents the quantity of extension disk.According to above-mentioned formula, n can be calculateds=3, ne=4, L=1.This represents a band Collection comprises three bands, and one has 4 row data is set as null, and for equilibrium criterion load, (data block in null is moved Move to other disks).Each band includes at least a null.Therefore, after expansion, band concentrates the sky that three bands are corresponding The quantity of row is to be not 1,1,2 (1+1+2=4=ne)。
Two, migration phase
After obtaining relevant parameter on last stage, band is concentrated suitable data block to migrate by this stage.Can divide Operate for following three steps:
(1) band collection is read in caching.
In order to reduce the quantity reading I/O operation in expansion process, the present invention is by the data block of whole band collection and check block Disposable reading is positioned at the caching X of internal memory.Band concentrates selected null to be placed in caching Y.Concentrate at a band, read Data on one piece of disk are that once order is read, and correspondence once reads I/O.
(2) data block is migrated
Suitable data block, according to the minimal modifications amount of check block, is migrated by the present invention.In the school different to three kinds Testing after the relation of block (row verification, tiltedly verification, the most tiltedly verification) is analyzed, the present invention changes the position of some data blocks, Make these blocks still in the verification chain that they are original.This way may insure that the check block that amendment is the fewest.This Bright STAR coding and the Triple-Star coding of giving is about row verification and the migration algorithm of tiltedly verification, and this algorithm can be by number According to the quantity migrated and the computational minimization of verification amendment.TheRow is seen as basis row, this Row need not move any data block in the horizontal direction.It follows that change the data block in same disk overhead and be positioned at The mapping relations between data block in null, this will not cause Data Migration expense.The remaining data block being positioned at null with Polling sequence is written in the data block of sky.
Wherein, STAR coding and Triple-Star encode about row verification and the migration algorithm tiltedly verified as follows:
Obtain and calculate the value of following parameter, n, m, nr ndAnd L, newly add the label of disk fromArrive
If
IfAndAnd i is not The ID of null, i, j represent data block line number in band and disk number respectively,
Then
If i is ＜ rb
Then i'=i, j'=j-m
I ＞ r else ifb
Then i'=i, j'=j+m
Such as, as in Figure 3-5,This represents in each band, 1st capable does not move any data block.The disk newly added is numberedArrive After being newly added disk, block 2 (C0,2) use C0,4Represent.Block 2 (C0,4, i=0, j=4) and meet the condition of above-mentioned algorithm, therefore the party Method is migrated to C0,2(i'=0, j=4-2=2).Be similar to, the method by the data block of circular labelling (such as block 9,18 and 25 Deng) migrate to different disks, but they still verify chain at original row and tiltedly verify chain.
It follows that the present invention changes reflecting between data block and the data block in null in same disk overhead Penetrate relation.From caching Y, select the data block (such as block 14,13 and 30 etc.) of diamond indicia, write the empty data block of same disk In.Therefore, this can be by Data Migration minimizing overhead.Finally caching Y in remaining data block (such as block 12,15,18 and 31 Deng) it is written into empty data block with polling sequence.In rare cases, the data block in caching Y is not enough to fill up same magnetic The empty data block of dish, this can increase a part of Data Migration quantity.
(3) update and recalculate check block
Three kinds of check blocks need to be updated or recalculate.Row check block can be and another by one-time write check disk Outer two kinds of check blocks need to be temporarily stored in caching X.
More newline check block
Here by the XOR of data block and the Lai Geng newline check block that calculate raw line check block and Xin Jia.It follows that I By its disposable writing line check disk because row verification will not be changed at merging phase.Such as, V12And V14Represent the value of data block 12 and 14.
Update oblique check block
Update tiltedly verification to need to calculate original oblique check block, the data block in relevant null and the row verification updated accordingly The XOR of block and.The tiltedly verification that oblique part newly increases can use the data in caching X to calculate.Such as,It follows that oblique check block to be put into caching X.
Recalculate the most tiltedly check block
Coding formula according to correcting and eleting codes, recalculates the most tiltedly check block used here as the data block in caching X, and will It is put in caching X.
Up to the present, the expansion process concentrated at a band is complete.If some data is in expansion process Lost, they can be resumed out by caching X and Y.
Three, merging phase
The present invention needs merging phase to be because it can reduce the I/O quantity of verification amendment.Because it is (or empty to have null Data block) band need nonetheless remain for check block and carry out data protection, the school needed in the quantity of check block and a complete band It is the same for testing number of blocks.Therefore we are by bar tape merge so that all check blocks are fully utilized, and reduce verification and repair Change the expense with I/O.
Merging phase needs following steps:
1) determine which band needs merged or is split so that overall computing cost is minimum.
After previous stage, band concentrates the line number of each band to know.The present invention selects some lucky The band joined merges, and forms a complete band.Without suitable band, then band is split.Due to I/O The reduction of expense, splitting the computing cost brought can undertake, and expansion process will not be produced impact.
By the three dish fault-tolerant arrays that encode based on Triple-Star as a example by 7 pieces of disks extend to 9 pieces of disks.Such as Fig. 3-4 Shown in, it is assumed that having three bands collection I, II and III, each band collection comprises band 0,1 and 2.Band 0 and 1 in band collection I can To be merged into a complete band, as shown in Figure 6.Band 2 in band collection I can be with the band 2 in band collection II and II Merge.Therefore, after merging completes, extend front 3 × 3=9 the band with 7 pieces of disks and become 1 × 3+1=4 There is the band of 9 pieces of disks.
2) concentrate at band or between band collection, merge band
This step can be divided into two parts:
Merging data block and row check block；
Merge tiltedly verification and the most tiltedly verification, and by its one-time write check disk.
In the first portion, the operation of merging data block and row check block only changes data block and the rower of row check block Number, extra I/O expense will not be produced.Block Ci,j(band a) is mapped to block Ci+x,j(band b), x are the numbers of the row in band b Amount (not comprising null).As in Figure 3-5, block C1,2(band 1) is mapped to C4,2The position of (band 0), x=3.
In the second portion, merge tiltedly verification and the most tiltedly verification, and by its one-time write check disk.We can be simple Ground calculates above two by XOR and verifies.Row check block Ci,p+1(band a) and(the value phase of band b) Add and obtain a new Ci,p+1, x is the quantity (not comprising null) of the row in band a.I.e. As in Figure 3-5,R'0Can be with C in band 00,8Represent.In band 1,R'10Can be with C in band 04,8Represent.Data and row check block in band 1 are mapped to bar After band 0,Therefore the C in band 0 is updated4,8It is original in calculating band 0 C0,8With the C in band 14,8XOR and (X=3, p=7).The most tiltedly calculating of check block and tiltedly verification The calculating of block is similar to.It follows that they are write check disk and discharge spatial cache.Aforesaid operations can be carried out parallel, accelerates Whole extension process.
Fig. 7 is the system architecture diagram of the expansible system of a kind of three dish fault-tolerant arrays of the present invention.As it is shown in fig. 7, the present invention The expansible system of a kind of three dish fault-tolerant arrays, including preparatory unit 701, migration units 702 and combining unit 703.
Wherein, preparatory unit 701 collects relevant parameter from storage system, for migration units 702 and combining unit 703 provide necessary parameter；Migration units 702 selects a part of band as band collection, and calculates according in the stage one Parameter, each band is selected some data row, is used for migrating in the disk that newly adds, in transition process, data block Movement be by different check between association determined, this can ensure that equilibrium data distribution；Combining unit 703, uses In by empty band polymerization, to reduce quantity and the I/O expense of verification amendment.
Preparatory unit 701, according to the layout of MDS coding in three dish fault-tolerant arrays and its configuration parameter, can use following public affairs Formula calculates ns, neAnd L.According to Fig. 2, a band collection contains many bands.In each band, partial row is set as " null ", in order to reach data balancing, they can be used to move to other dishes.
n s = l c m { m × n r , n d + m } m × n r
n e = l c m { m × n r , n d + m } n d + m
L = 1 m × n r ≤ n d + m f l o o r ( m × n r n d + m ) m × n r > n d + m
Band is concentrated suitable data block to migrate by migration units 702, and following three steps can be divided into operate:
(1) band collection is read in caching.
In order to reduce the quantity reading I/O operation in expansion process, migration units 702 is by the data block of whole band collection and school Test block and disposably read in the caching X being positioned at internal memory.Band concentrates selected null to be placed in caching Y.At a band collection In, reading the data on one piece of disk is that once order is read, and correspondence once reads I/O.
(2) data block is migrated
Suitable data block, according to the minimal modifications amount of check block, is migrated by migration units 702.To three kinds of differences The relation of check block (row verification, tiltedly verification, the most tiltedly verification) be analyzed after, change the position of some data blocks, make Obtain these blocks still in the verification chain that they are original.This way may insure that the check block that amendment is the fewest.Migrate single Unit 702 gives STAR coding and Triple-Star coding about row verification and the migration algorithm of tiltedly verification, and this algorithm can be by The quantity of Data Migration and the computational minimization of verification amendment.TheRow is seen as basis row, this A line need not move any data block in the horizontal direction.It follows that change data block and the position in same disk overhead The mapping relations between data block in null, this will not cause Data Migration expense.The remaining data block being positioned at null It is written in the data block of sky with polling sequence.
Wherein, STAR coding and Triple-Star encode about row verification and the migration algorithm tiltedly verified as follows:
Obtain and calculate the value of following parameter, n, m, nr ndAnd L, newly add the label of disk fromArrive
If
IfAndAnd i is not The ID of null
Then
If i is ＜ rb
Then i'=i, j'=j-m
I ＞ r else ifb
Then i'=i, j'=j+m
Such as, as in Figure 3-5,This represents in each band, 1st capable does not move any data block.The disk newly added is numberedArrive After being newly added disk, block 2 (C0,2) use C0,4Represent.Block 2 (C0,4, i=0, j=4) and meet the condition of above-mentioned algorithm, therefore migrate Unit 702 is migrated to C0,2(i'=0, j=4-2=2).Be similar to, migration units 702 by the data block of circular labelling (as Block 9,18 and 25 etc.) migrate to different disks, but they still verify chain at original row and tiltedly verify chain.
It follows that migration units 702 data block that changes same disk overhead and the data block that is positioned in null it Between mapping relations.From caching Y, select the data block (such as block 14,13 and 30 etc.) of diamond indicia, write the sky of same disk In data block.Therefore, this can be by Data Migration minimizing overhead.Finally caching Y in remaining data block (such as block 12,15,18 With 31 etc.) data block of sky it is written into polling sequence.In rare cases, the data block in caching Y is not enough to fill up together The empty data block of one disk, this can increase a part of Data Migration quantity.
(3) update and recalculate check block
Three kinds of check blocks need to be updated or recalculate.Row check block can be and another by one-time write check disk Outer two kinds of check blocks need to be temporarily stored in caching X.
More newline check block
Here by the XOR of data block and the Lai Geng newline check block that calculate raw line check block and Xin Jia.It follows that I By its disposable writing line check disk because row verification will not be changed at merging phase.Such as, V12And V14Represent the value of data block 12 and 14.
Update oblique check block
Update tiltedly verification to need to calculate original oblique check block, the data block in relevant null and the row verification updated accordingly The XOR of block and.The tiltedly verification that oblique part newly increases can use the data in caching X to calculate.Such as,It follows that oblique check block to be put into caching X.
Recalculate the most tiltedly check block
Coding formula according to correcting and eleting codes, recalculates the most tiltedly check block used here as the data block in caching X, and will It is put in caching X.
Up to the present, the expansion process concentrated at a band is complete.If some data is in expansion process Lost, they can be resumed out by caching X and Y.
Combining unit 703 is by bar tape merge so that all check blocks are fully utilized, and reduces verification amendment and I/ The expense of O.
Combining unit needs following steps:
1) determine which band needs merged or is split so that overall computing cost is minimum.
After previous stage, band concentrates the line number of each band to know.The present invention selects some lucky The band joined merges, and forms a complete band.Without suitable band, then band is split.Due to I/O The reduction of expense, splitting the computing cost brought can undertake, and expansion process will not be produced impact.
By the three dish fault-tolerant arrays that encode based on Triple-Star as a example by 7 pieces of disks extend to 9 pieces of disks.Such as Fig. 3-4 Shown in, it is assumed that having three bands collection I, II and III, each band collection comprises band 0,1 and 2.Band 0 and 1 in band collection I can To be merged into a complete band, as shown in Figure 6.Band 2 in band collection I can be with the band 2 in band collection II and II Merge.Therefore, after merging completes, extend front 3 × 3=9 the band with 7 pieces of disks and become 1 × 3+1=4 There is the band of 9 pieces of disks.
2) concentrate at band or between band collection, merge band
This step can be divided into two parts:
Merging data block and row check block；
Merge tiltedly verification and the most tiltedly verification, and by its one-time write check disk.
In the first portion, the operation of merging data block and row check block only changes data block and the rower of row check block Number, extra I/O expense will not be produced.Block Ci,j(band a) is mapped to block Ci+x,j(band b), x are the numbers of the row in band b Amount (not comprising null).As in Figure 3-5, block C1,2(band 1) is mapped to C4,2The position of (band 0), x=3.
In the second portion, merge tiltedly verification and the most tiltedly verification, and by its one-time write check disk.We can be simple Ground calculates above two by XOR and verifies.Row check block Ci,p+1(band a) and(the value phase of band b) Add and obtain a new Ci,p+1, x is the quantity (not comprising null) of the row in band a.I.e. As in Figure 3-5,R'0Can be with C in band 00,8Represent.In band 1,R'10Can be with C in band 04,8Represent.Data and row check block in band 1 are mapped to bar After band 0,Therefore the C in band 0 is updated4,8It is original in calculating band 0 C0,8With the C in band 14,8XOR and (X=3, p=7).The most tiltedly calculating of check block and tiltedly verification The calculating of block is similar to.It follows that they are write check disk and discharge spatial cache.Aforesaid operations can be carried out parallel, accelerates Whole extension process.
In sum, the expandable method of a kind of three dish fault-tolerant arrays of the present invention and system are according to the association of different check Property, select suitable data block to migrate, thus the Data Migration minimized and the quantity of verification amendment, meanwhile, this Bright the band with sky data block is merged so that the I/O cost reduction of verification amendment, therefore present invention can assure that expansion After exhibition, data distribution is equilibrium, and the I/O expense in expansion process is less.
The principle of above-described embodiment only illustrative present invention and effect thereof, not for limiting the present invention.Any Above-described embodiment all can be modified under the spirit and the scope of the present invention and change by skilled person.Therefore, The scope of the present invention, should be as listed by claims.

Claims (10)

1. an expandable method for three dish fault-tolerant arrays, comprises the steps:
Preparatory stage, from storage system, collect relevant parameter, calculate necessary parameter for ensuing two stages；
Migration phase, select a part of band as band collection, and according to the parameter calculated in the preparatory stage, at each Band is selected some data row, is used in the disk migrating to newly add；
Merging phase, by empty band polymerization.
A kind of expandable method of three dish fault-tolerant arrays, it is characterised in that: in the preparatory stage, root The layout encoded according to MDS in three dish fault-tolerant arrays and its configuration parameter, calculate ns, neAnd L, it is provided that to two later rank Section, wherein, nsRepresent the band number that a band is concentrated, neRepresent the total quantity being arranged to null in this band concentration, L Represent in a band, the minimum number of null.
A kind of expandable method of three dish fault-tolerant arrays, it is characterised in that migration phase include as Lower step:
Band collection is read in caching；
Minimal modifications amount according to check block, migrates suitable data block；
Update and recalculate check block.
A kind of expandable method of three dish fault-tolerant arrays, it is characterised in that: by whole band collection Data block and check block disposably read in the caching X being positioned at internal memory, and band concentrates selected null to be placed in caching Y.
A kind of expandable method of three dish fault-tolerant arrays, it is characterised in that: described according to check block Minimal modifications amount, row verification, tiltedly verification, the relation of the most tiltedly verification are carried out by the step migrating suitable data block After analysis, change the position of some data blocks so that these blocks are still in the verification chain that they are original.
A kind of expandable method of three dish fault-tolerant arrays, it is characterised in that: described renewal and again Calculate in check block step, by calculating XOR and the Lai Geng newline check block of the data block of raw line check block and Xin Jia.
A kind of expandable method of three dish fault-tolerant arrays, it is characterised in that: described renewal and again Calculating in check block step, updating tiltedly verification needs to calculate original oblique check block, data block in relevant null and corresponding the most more The XOR of new row check block and.
A kind of expandable method of three dish fault-tolerant arrays, it is characterised in that: described renewal and again Calculate in check block step, recalculate the most tiltedly check block according to the coding formula of correcting and eleting codes.
The expandable method of a kind of three dish fault-tolerant arrays, it is characterised in that described merging phase bag Include following steps:
Determine which band needs merged or is split so that overall computing cost is minimum；
Concentrate at band or between band collection, merge band.
10. an expansible system for three dish fault-tolerant arrays, including:
Preparatory unit, collects relevant parameter from storage system, provides necessary parameter for migration units and combining unit；
Migration units, select a part of band as band collection, and according to the parameter calculated in the stage one, at each band In select some data row, be used for migrating in the disk that newly adds, in transition process, the movement of data block is by different check Between association determined；
Combining unit, for by empty band polymerization.
CN201610637554.1A 2016-08-05 2016-08-05 A kind of expandable method and system of three disks fault-tolerant array Active CN106293526B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610637554.1A CN106293526B (en) 2016-08-05 2016-08-05 A kind of expandable method and system of three disks fault-tolerant array

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610637554.1A CN106293526B (en) 2016-08-05 2016-08-05 A kind of expandable method and system of three disks fault-tolerant array

Publications (2)

Publication Number Publication Date
CN106293526A true CN106293526A (en) 2017-01-04
CN106293526B CN106293526B (en) 2019-04-12

Family

ID=57665961

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610637554.1A Active CN106293526B (en) 2016-08-05 2016-08-05 A kind of expandable method and system of three disks fault-tolerant array

Country Status (1)

Country Link
CN (1) CN106293526B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107094186A (en) * 2017-06-15 2017-08-25 深圳市云舒网络技术有限公司 A kind of correcting and eleting codes optimization method of distributed memory system
CN111880963A (en) * 2020-07-29 2020-11-03 北京浪潮数据技术有限公司 Data reconstruction method, device, equipment and storage medium
CN112835533A (en) * 2021-02-25 2021-05-25 上海交通大学 Cloud storage array expansion method and device based on rack level
CN114816278A (en) * 2022-06-30 2022-07-29 苏州浪潮智能科技有限公司 Data migration method, system, equipment and storage medium of storage server
CN117194095A (en) * 2022-07-26 2023-12-08 慧与发展有限责任合伙企业 Changing error correction configuration

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040210731A1 (en) * 2003-04-16 2004-10-21 Paresh Chatterjee Systems and methods for striped storage migration
US20080195808A1 (en) * 2007-02-14 2008-08-14 Via Technologies, Inc. Data migration systems and methods for independent storage device expansion and adaptation
CN102200892A (en) * 2011-04-29 2011-09-28 华中科技大学 Capacity expansion method based on dynamic redundant array of independent disks (RAID) system
CN104881365A (en) * 2015-05-31 2015-09-02 上海交通大学 RAID-6 extensible method based on erasure code similarity
CN104881372A (en) * 2015-05-31 2015-09-02 上海交通大学 Data migration method capable of improving RAID-6 (redundant array of independent disks-6) expandability
CN105630423A (en) * 2015-12-25 2016-06-01 华中科技大学 Erasure code cluster storage expansion method based on data caching

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040210731A1 (en) * 2003-04-16 2004-10-21 Paresh Chatterjee Systems and methods for striped storage migration
US20080195808A1 (en) * 2007-02-14 2008-08-14 Via Technologies, Inc. Data migration systems and methods for independent storage device expansion and adaptation
CN102200892A (en) * 2011-04-29 2011-09-28 华中科技大学 Capacity expansion method based on dynamic redundant array of independent disks (RAID) system
CN104881365A (en) * 2015-05-31 2015-09-02 上海交通大学 RAID-6 extensible method based on erasure code similarity
CN104881372A (en) * 2015-05-31 2015-09-02 上海交通大学 Data migration method capable of improving RAID-6 (redundant array of independent disks-6) expandability
CN105630423A (en) * 2015-12-25 2016-06-01 华中科技大学 Erasure code cluster storage expansion method based on data caching

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107094186A (en) * 2017-06-15 2017-08-25 深圳市云舒网络技术有限公司 A kind of correcting and eleting codes optimization method of distributed memory system
CN107094186B (en) * 2017-06-15 2019-10-01 深圳市云舒网络技术有限公司 A kind of correcting and eleting codes optimization method of distributed memory system
CN111880963A (en) * 2020-07-29 2020-11-03 北京浪潮数据技术有限公司 Data reconstruction method, device, equipment and storage medium
CN111880963B (en) * 2020-07-29 2022-06-10 北京浪潮数据技术有限公司 Data reconstruction method, device, equipment and storage medium
CN112835533A (en) * 2021-02-25 2021-05-25 上海交通大学 Cloud storage array expansion method and device based on rack level
CN112835533B (en) * 2021-02-25 2023-02-17 上海交通大学 Cloud storage array expansion method and device based on rack level
CN114816278A (en) * 2022-06-30 2022-07-29 苏州浪潮智能科技有限公司 Data migration method, system, equipment and storage medium of storage server
CN114816278B (en) * 2022-06-30 2022-11-11 苏州浪潮智能科技有限公司 Data migration method, system, equipment and storage medium of storage server
WO2024001112A1 (en) * 2022-06-30 2024-01-04 苏州元脑智能科技有限公司 Data migration method, system and device for storage server, and nonvolatile readable storage medium
CN117194095A (en) * 2022-07-26 2023-12-08 慧与发展有限责任合伙企业 Changing error correction configuration

Also Published As

Publication number Publication date
CN106293526B (en) 2019-04-12

Similar Documents

Publication Publication Date Title
CN106293526A (en) The expandable method of a kind of three dish fault-tolerant arrays and system
JP6771018B2 (en) Improved performance of 2D array processor
CN104871138B (en) For the method and system that multidimensional RAID reconstruction and defect are avoided
CN111062472B (en) Sparse neural network accelerator based on structured pruning and acceleration method thereof
US8296515B1 (en) RAID-6 computation system and method
US20140188748A1 (en) Methods and systems for determining estimated package delivery/pick-up times
US7797612B2 (en) Storage accelerator
US11494265B2 (en) Securing against errors in an error correcting code (ECC) implemented in an automotive system
CN107038087A (en) Method and system for multidimensional RAID
CN104850468A (en) Check matrix based erasure code decoding method
JPS60237578A (en) 2-d image generation for 3-d object
CN101587425A (en) A kind of method and device that increases magnetic disc redundant array redundancy
US20050114727A1 (en) Uniform and symmetric double failure correcting technique for protecting against two disk failures in a disk array
CN109792256A (en) For the device and correlation technique coded and decoded to data to erasure codes
CN104932835A (en) Erasure code based distributed storage system capacity expansion and reduction method
CN103605582B (en) Erasure code storage and reconfiguration optimization method based on redirect-on-write
CN112799604A (en) N-Code-based RAID6 disk array capacity expansion method and data filling method
CN103809919B (en) Efficiently hold many wrong fast quick-recovery coded methods and validation matrix generates method
CN107240155A (en) Method, server and system that a kind of model object is built
US7971092B2 (en) Methods and devices for high performance consistency check
CN100470489C (en) A method for processing data multiplication in a disk array and its controller
CN107193685A (en) Entangling based on flash memory device deletes method and device
CN105824578A (en) Equation selection method used for reducing data volume in RAID-6 decoding I/O
CN105933386A (en) Construction method and device of storage system
CN108270449A (en) Support the encoder of a variety of encoding rates and code length

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant