CN103699494A - Data storage method, data storage equipment and distributed storage system - Google Patents
Data storage method, data storage equipment and distributed storage system Download PDFInfo
- Publication number
- CN103699494A CN103699494A CN201310657140.1A CN201310657140A CN103699494A CN 103699494 A CN103699494 A CN 103699494A CN 201310657140 A CN201310657140 A CN 201310657140A CN 103699494 A CN103699494 A CN 103699494A
- Authority
- CN
- China
- Prior art keywords
- data
- verification
- memory system
- distributed memory
- data fragmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a data storage method, data storage equipment and a distributed storage system. The method, the equipment and the system belong to the technical field of computers. The data storage method comprises the steps that access nodes of the distributed storage system cut a data block which is written in by a user into a plurality of data fragments with preset sizes; a plurality of check fragments corresponding to the data fragments which are obtained by cutting are calculated through a redundancy check algorithm; the data fragments and the check fragments are stored into the data nodes of the distributed storage system, and each data fragment and each check fragment only store one copy. According to the data storage method, the data storage equipment and the distributed storage system, the proportion of redundant data can be reduced on the premise of ensuring data reliability, so as to save the storage space.
Description
Technical field
The present invention relates to field of computer technology, be specifically related to a kind of date storage method, data storage device and distributed memory system.
Background technology
Cassandra is the distributed memory system that relies on a kind of typical loop configuration without Centroid of distributed hashtable (Distributed Hash Table, DHT) technology realization.DHT is a kind of distributed storage technology, and in the situation that not needing Centroid, each memory node is responsible for a route among a small circle, and is responsible for storage fraction data, thereby realizes addressing and the storage of whole DHT distributed type assemblies.
The data space of Cassandra can abstractly be a loop configuration, and data are dispersed on this annular storage space by hash (Hash) exactly.Each node of distributed memory system is in charge of a certain continuous scope (being also Range) on this annular storage space, and the data that drop on this Range space are just stored on this node.As shown in Figure 1, in having the cluster of tetra-node scales of A/B/C/D, storage space has been divided into tetra-Range of R0/R1/R2/R3, and the responsible Range of each node distributes as shown in the table:
| Range | Node |
| R3 | A |
| R0 | B |
| R1 | C |
| R2 | D |
For example, when the hash of data drops on R0, these data are just stored in Node B.
The role of Cassandra node is divided into two kinds: access node and back end.Access node is responsible for the distribution of computational data and copy, and back end is responsible for the storage of data trnascription.In Cassandra system, each node, both as access node, exists as back end again.
In order to guarantee the reliability of data, Cassandra generally will write multiple copies by data, and is dispersed on different back end.When certain copy is because disk failure or mechanical disorder etc. is lost extremely, other copies also can normally provide access, and can under some condition after fault recovery, complete the reparation of losing copy.
Under the production environment of Cassandra, generally can adopt 3 replication policies: for every data recording (or being called data block) except back end storage raw data, also other 2 copies of data to be stored in other two back end.As shown in Figure 2, the hash of data block drops in R0 spatial dimension, and raw data is stored in Node B, and two other copy is stored in respectively in node C and node D.
From the realization of prior art, data are recorded in Cassandra and store, and in fact need to store three data trnascriptions, have taken a large amount of storage spaces, and redundant copy expense is excessive, cause space waste, stores service high cost.And if reduce the quantity of copy, as change two copies storages into, can greatly reduce again the reliability of data.
Summary of the invention
In view of the above problems, the present invention has been proposed to a kind of overcome the problems referred to above or the date storage method addressing the above problem at least in part, data storage device and distributed memory system are provided.
According to an aspect of the present invention, provide a kind of date storage method, having comprised:
The data block that the access node of distributed memory system writes user is cut into the data fragmentation of some pre-sizings;
Adopt redundancy check algorithm to calculate some verification bursts corresponding to some data fragmentations that cutting obtains;
Described data fragmentation and described verification burst are stored in the back end of described distributed memory system, and each data fragmentation is only stored a copy with each verification burst.
Alternatively, described described data fragmentation and described verification burst are stored in the back end of described distributed memory system, comprising:
Calculate respectively the digital signature of each data fragmentation and each verification burst;
According to digital signature separately, each data fragmentation and each verification burst are stored into respectively in a back end of described distributed memory system.
Alternatively, described date storage method also comprises:
All digital signature are encoded and formed metadata;
By described metadata store in a plurality of back end of described distributed memory system.
Alternatively, described employing redundancy check algorithm calculates some verification bursts corresponding to some data fragmentations that cutting obtains, and comprising:
If the individual data fragmentation that cutting is obtained is divided into some groups of data fragmentations, wherein, every group of data fragmentation comprises the first predetermined number data fragmentation;
Adopt redundancy check algorithm to calculate respectively and respectively organize the second predetermined number verification burst that data fragmentation is corresponding.
Alternatively, described date storage method also comprises:
Using every group of data fragmentation and corresponding verification burst as a stripe cell, all digital signature corresponding to described stripe cell are encoded and formed metadata;
By described metadata store in a plurality of back end of described distributed memory system.
Alternatively, described in, be encoded to JSON coding.
Alternatively, described distributed memory system is Cassandra system.
According to another aspect of the present invention, provide a kind of data storage method, apparatus, be arranged in the access node of distributed memory system, described data storage device comprises:
Data cutting unit, the data block that is suitable for user to write is cut into the data fragmentation of some pre-sizings;
Verification computing unit, is suitable for adopting redundancy check algorithm to calculate some verification bursts corresponding to some data fragmentations that cutting obtains;
Unit of memory allocation, is suitable for described data fragmentation and described verification burst to store in the back end of described distributed memory system, and each data fragmentation is only stored a copy with each verification burst.
Alternatively, described unit of memory allocation is further adapted for:
Calculate respectively the digital signature of each data fragmentation and each verification burst;
According to digital signature separately, each data fragmentation and each verification burst are stored into respectively in a back end of described distributed memory system.
Alternatively, described data storage device also comprises:
Coding unit, is suitable for all digital signature to encode and form metadata;
Metadata storage unit, is suitable for described metadata store in a plurality of back end of described distributed memory system.
Alternatively, described verification computing unit is further adapted for:
If the individual data fragmentation that cutting is obtained is divided into some groups of data fragmentations, wherein, every group of data fragmentation comprises the first predetermined number data fragmentation;
Adopt redundancy check algorithm to calculate respectively and respectively organize the second predetermined number verification burst that data fragmentation is corresponding.
Alternatively, described data storage device also comprises:
Coding unit, is suitable for usining every group of data fragmentation and corresponding verification burst as a stripe cell, and all digital signature corresponding to described stripe cell are encoded and formed metadata;
Metadata storage unit, is suitable for described metadata store in a plurality of back end of described distributed memory system.
Alternatively, described in, be encoded to JSON coding.
Alternatively, described distributed memory system is Cassandra system.
According to a further aspect of the invention, provide a kind of distributed memory system that comprises above-mentioned data storage device.
Above-mentioned one or more technical schemes according to the present invention, in distributed memory system, pass through to use erasable code (Eraser Code, EC) verification mode is replaced existing many copies storage mode, can guarantee under the prerequisite of data reliability, reduce redundant data accounting, thereby reach the object of saving storage space.
Above-mentioned explanation is only the general introduction of technical solution of the present invention, in order to better understand technological means of the present invention, and can be implemented according to the content of instructions, and for above and other objects of the present invention, feature and advantage can be become apparent, below especially exemplified by the specific embodiment of the present invention.
Accompanying drawing explanation
By reading below detailed description of the preferred embodiment, various other advantage and benefits will become cheer and bright for those of ordinary skills.Accompanying drawing is only for the object of preferred implementation is shown, and do not think limitation of the present invention.And in whole accompanying drawing, by identical reference symbol, represent identical parts.In the accompanying drawings:
Fig. 1 shows the storage space of Cassandra system and divides schematic diagram;
Fig. 2 shows according to the date storage method schematic diagram of the Cassandra system of prior art;
Fig. 3 shows the process flow diagram of date storage method according to an embodiment of the invention;
Fig. 4 shows the corresponding relation of metadata and data block in date storage method according to an embodiment of the invention;
Fig. 5 shows the structural drawing of data storage device according to an embodiment of the invention.
Embodiment
Exemplary embodiment of the present disclosure is described below with reference to accompanying drawings in more detail.Although shown exemplary embodiment of the present disclosure in accompanying drawing, yet should be appreciated that and can realize the disclosure and the embodiment that should do not set forth limits here with various forms.On the contrary, it is in order more thoroughly to understand the disclosure that these embodiment are provided, and can by the scope of the present disclosure complete convey to those skilled in the art.
For solving prior art, in distributed memory system, adopt many copies storage mode to exist, the problem of the excessive and stores service high cost of redundant copy expense, the embodiment of the present invention provides a kind of data storage scheme, distributed memory system for example the access node of Cassandra system data block is cut into some data fragmentations, and calculate some verification bursts, data fragmentation and verification burst are disperseed to be stored on back end different in Cassandra, these bursts form a stripe cell and manage, limited burst in this stripe cell is because of after fault loses, can correct and recovery fault data burst by check information, can guarantee under the prerequisite of data reliability, reduce redundant data accounting, thereby reach the object of saving storage space.
Fig. 3 shows the process flow diagram of date storage method according to an embodiment of the invention.With reference to Fig. 3, described date storage method can comprise:
Described distributed memory system can be Cassandra system, and as previously mentioned, the role of Cassandra node is divided into two kinds: access node and back end.Access node is responsible for the distribution of computational data and copy, and back end is responsible for the storage of data trnascription.In Cassandra system, each node, both as access node, exists as back end again.
In the prior art, when access node receives the data block that user writes, first the cryptographic hash of computational data piece (or key of data block (Key)), according to the cryptographic hash calculating, data block is stored in corresponding a plurality of back end, normally store 3 back end (back end corresponding to Range at cryptographic hash place is arranged in this back end clockwise other 2 back end afterwards in annular storage space) into.
Different being from prior art, in embodiments of the present invention, when access node receives the data block that user writes, first this data block is carried out to cutting according to fixed size, the size that is the data fragmentation that obtains of cutting is fixed (being convenient to according to redundancy check algorithm calculation check information), and certainly, last burst that cutting obtains may not reach described pre-sizing, now, can be with empty data polishing.Wherein, burst size can be specified as parameter when storage creation, and is stored in the systemtable of Cassandra system.
It should be noted that, in embodiments of the present invention, described distributed memory system is not limited to Cassandra system, can be also other distributed memory system arbitrarily such as BigTable system, Dynamo system, Hadoop Hbase system.
Redundancy check algorithm is applied to erasable code (Eraser Code conventionally, EC) in technology, EC is forward error correction (Forward Error Correction, FEC) a kind of form of technology, by to polylith data calculation check, form some check informations, when partial data fault, can automatically correct and recover fault data.Like this, by EC technology, use relatively low storage overhead, just can realize good storage persistence.
Wherein, described redundancy check algorithm can adopt parity arithmetic, cyclic redundancy check (CRC) algorithm (Cyclic Redundancy Check, CRC) etc.Generally speaking, for M data fragmentation, can calculate the individual verification burst of N (N >=1) according to redundancy check algorithm, in M+N burst, if the individual burst of K (K≤N) breaks down, can be recovered by the remaining M+N-K a not breaking down burst.N can carry out value as required, and N value is larger, and the reliability of data is higher, but the storage space taking is also high, and correspondingly, N value is less, and the reliability of data is lower, but the storage space taking is also low.
Another different being from prior art, in embodiments of the present invention, in distributed memory system, only store a copy of each data fragmentation and each verification burst, be that each data fragmentation is stored in a back end corresponding with it, each verification burst is also stored in a back end corresponding with it.Like this, by adopting EC technology, after the one or more data fragmentations in data block break down, can it be recovered by data fragmentation and the verification burst not breaking down, guaranteed the reliability of data; By only storing a copy of burst, reduced redundant data accounting, thereby saved storage space.
Can adopt various ways to realize each data fragmentation and each verification burst are only stored to a copy, below provide a kind of implementation wherein.
According to a kind of implementation of the embodiment of the present invention, in step 306, described described data fragmentation and described verification burst are stored in the back end of described distributed memory system, can comprise:
Step S02, calculates respectively the digital signature of each data fragmentation and each verification burst;
The algorithm of the digital signature of computational data has multiple, and the embodiment of the present invention does not limit concrete algorithm.For example, can Secure Hash Algorithm (secure hash algorithm, SHA1) or Message Digest Algorithm 5 (Message Digest Algorithm, MD5).
Step S04, stores each data fragmentation and each verification burst respectively in a back end of described distributed memory system according to digital signature separately.
The digital signature calculating in step S02 can be used as the key (key) of burst, in step S04, can find the back end at burst place according to key, thereby burst is stored in corresponding back end.
During for convenience of read block, each data fragmentation is carried out to addressing (back end of the described data fragmentation of location storage), the method of the embodiment of the present invention can also comprise: all digital signature are encoded and are formed metadata, and by described metadata store in a plurality of back end of described distributed memory system.
For example, suppose that the data block storehouse that user writes has been cut into 10 data fragmentations, adopt redundancy check algorithm to calculate 2 verification bursts that described 10 data fragmentations are corresponding, like this, by calculate can obtain these 12 bursts respectively corresponding 12 digital signature be: K0, K1, ..., K9, Ec0, Ec1.As shown in Figure 4, these 12 digital signature sequential organizations are encoded together and formed metadata (Meta), according to described metadata, just can locate the back end that each burst is corresponding.
In the invention process, described coding can adopt JSON (JavaScript Object Notation) coding, certainly, also can adopt other encryption algorithm, and the embodiment of the present invention is not restricted this.
The implementation method that in addition, can adopt distributed memory system self by described metadata store in a plurality of back end of described distributed memory system.For example, in Cassandra system, can store 3 copies of described metadata, be about to described metadata and be stored in respectively in 3 back end.Particularly, first the cryptographic hash of Computing Meta data, in back end corresponding to the Range that stores a copy of metadata into cryptographic hash place according to the cryptographic hash calculating, other 2 copies of metadata are stored into respectively and are arranged in clockwise other 2 back end after this back end.
The size of a plurality of data blocks that write due to user is conventionally inconsistent, for realizing the standardization of data encoding, storage, in the step 304 of the embodiment of the present invention, described employing redundancy check algorithm calculates some verification bursts corresponding to some data fragmentations that cutting obtains, and can comprise:
Step S12, if the individual data fragmentation that cutting is obtained is divided into some groups of data fragmentations, wherein, every group of data fragmentation comprises the first predetermined number data fragmentation;
Described the first predetermined number can arrange as required, and for example described the first predetermined number is set to 5, and every group of data fragmentation comprises 10 data fragmentations.Certainly, if carried out after described division, the quantity of the data fragmentation that last group data fragmentation comprises may be less than described the first predetermined number, now, can use " 0 " (empty burst) polishing to described the first predetermined number.
Give an example, suppose data block to carry out obtaining 38 data fragmentations after cutting, according to above-mentioned steps, these 38 data fragmentations can be divided into 4 groups of data fragmentations, the quantity of the data fragmentation that every group of data fragmentation comprises is 10, and last group data fragmentation has comprised 2 empty bursts.
Step S14, adopts redundancy check algorithm to calculate respectively and respectively organizes the second predetermined number verification burst that data fragmentation is corresponding.
Described the second predetermined quantity can be based on to the factor that takies of data reliability factor and storage space, the two weighs consideration, for example, when described the first predetermined number value is 10, described the second predetermined number value is 2, i.e. every group of corresponding 2 verification bursts of 10 data fragmentations.
Afterwards, just can the first predetermined number data fragmentation and corresponding the second predetermined number verification burst of every group be managed as a stripe cell.Correspondingly, what aforesaid metadata was corresponding is a stripe cell, the date storage method that is the embodiment of the present invention can also comprise: using every group of data fragmentation and corresponding verification burst as a stripe cell, all digital signature corresponding to described stripe cell are encoded and are formed metadata, by described metadata store in a plurality of back end of described distributed memory system.Wherein, the coding of metadata and the specific implementation of storage can, referring to the description of abovementioned steps S02 and step S04, not repeat here.
Below provide an application example of the date storage method of the embodiment of the present invention, in this application example, described distributed memory system is Cassandra system, described in be encoded to JSON coding.
(1) user writes a data block at access node, the key of this data block (key) is ' dummy.key ', the value of this data block (value) comprises the data of 10000 bytes, this data block can be cut into 10 according to fixed size (1000 byte): 1......1000 byte, 1001......2000 byte ..., 9001......10000 byte.
(2) for example, for these 10 data fragmentations calculate key assignments (SHA1 signature), as follows respectively:
34f11e0febf1a6d43f66f5af738667fec9ac6e5c
e27efcfe1450f97d21b51c612587488dae3d356e
e4a192a7d494f3691bd1fff1a8195eecdf2ed76b
f8d9a3e143bc275e4859736f37de0ce280aae034
12116b77653d73825530056f4d9f322abc771e34
87e1e55c6684ab33eb4ea104b31bc5595340e934
61306782f48c25332729e2fca1efa29d3709067d
c04b64f9b709488fbee7282f9c7f1c6f443293ab
e401b1b341e0bfe2a1989f99c22353b8e7b68427
b29ab838829fa69443230f20e8fc5fbc1fbbfe28
(3) adopt redundancy check algorithm to calculate 2 verification bursts, for example, for these 2 verification bursts calculate key assignments (SHA1 signature), as follows respectively:
8936986814f5f38a83d2f0809f2e36f4572996c2
09e77c8ef46aaccaa76c4c71fa757fb68ccd0a7d
(4) 10 data fragmentations and 2 verification bursts are considered as to a stripe cell, the JSON of 12 key assignments that this stripe cell is corresponding is encoded to:
[″34f11e0febf1a6d43f66f5af738667fec9ac6e5c″，″e27efcfe1450f97d21b51c612587488dae3d356e″，″e4a192a7d494f3691bd1fff1a8195eecdf2ed76b″，″f8d9a3e143bc275e4859736f37de0ce280aae034″，″12116b77653d73825530056f4d9f322abc771e34″，″87e1e55c6684ab33eb4ea104b31bc5595340e934″，″61306782f48c25332729e2fcalefa29d3709067d″，″c04b64f9b709488fbee7282f9c7f1c6f443293ab″，″e401b1b341e0bfe2a1989f99c22353b8e7b68427″，″b29ab838829fa69443230f20e8fc5fbc1fbbfe28″，″8936986814f5f38a83d2f0809f2e36f4572996c2″，″09e77c8ef46aaccaa76c4c71fa757fb68ccd0a7d″]
(5) 10 data fragmentations and 2 the verification bursts that stripe cell are comprised store in the corresponding data node of Cassandra according to corresponding key assignments, particularly, burst only can be stored in the back end under the Range at key place, and not store other 2 copies; .
(6) the JSON coding of 12 key assignments is written in Cassandra cluster and is gone as metadata.Metadata still still adopts many copies redundancy scheme.
Like this, after wherein a slice data are lost due to place node disk failure, can be repaired by other 11 data, thereby guarantee the reliability of data.
In this application example, by 10 data fragmentations, calculate 2 verification bursts, be persisted to data expansion 20% after disk, actual stored is raw data 1.2 times.And according to prior art scheme, adopt in the situation of 3 copies, data expansion 200%, actual stored is raw data 3 times.
Therefore,, with respect to the technical scheme of original many copy storages, the technical program can, under the prerequisite of assurance data reliability, significantly reduce redundant data ratio, thereby significantly save storage space.
The embodiment of the present invention also provides a kind of data storage device of realizing above-mentioned date storage method.
Fig. 5 shows the structural drawing of data storage device according to an embodiment of the invention, described data storage device is arranged in the access node of distributed memory system, and described distributed memory system can be the distributed memory system of Cassandra system or other any type.With reference to Fig. 5, described data storage device can comprise data cutting unit 502, verification computing unit 504 and unit of memory allocation 506, wherein:
The data block that data cutting unit 502 is suitable for user to write is cut into the data fragmentation of some pre-sizings.If last burst that cutting obtains may not reach described pre-sizing, can be with empty data polishing.Wherein, burst size can be specified as parameter when storage creation, and is stored in the systemtable of Cassandra system.
Unit of memory allocation 506 is suitable for described data fragmentation and described verification burst to store in the back end of described distributed memory system, and each data fragmentation is only stored a copy with each verification burst.
In embodiments of the present invention, in distributed memory system, only store a copy of each data fragmentation and each verification burst, be that each data fragmentation is stored in a back end corresponding with it, each verification burst is also stored in a back end corresponding with it.Like this, after the one or more data fragmentations in data block break down, can it be recovered by data fragmentation and the verification burst not breaking down, guaranteed the reliability of data; By only storing a copy of burst, reduced redundant data accounting, thereby saved storage space.
As a kind of implementation, described unit of memory allocation 506 can be installed following mode and be selected back end to carry out data storage:
First, calculate respectively the digital signature of each data fragmentation and each verification burst;
Then, according to digital signature separately, each data fragmentation and each verification burst are stored into respectively in a back end of described distributed memory system.
During for convenience of read block, each data fragmentation is carried out to addressing (back end of the described data fragmentation of location storage), the data storage device of the embodiment of the present invention can also comprise coding unit (not shown) and metadata storage unit (not shown), by coding unit to all digital signature encode (for example adopt JSON coding) form metadata, by metadata storage unit by described metadata store in a plurality of back end of described distributed memory system.Wherein, the storage of metadata can adopt many copies scheme.
The size of a plurality of data blocks that write due to user is conventionally inconsistent, for realizing the standardization of data encoding, storage, as a kind of implementation, if the individual data fragmentation that described verification computing unit 504 also further obtains cutting is divided into some groups of data fragmentations, wherein, every group of data fragmentation comprises the first predetermined number data fragmentation, and adopts redundancy check algorithm to calculate respectively respectively to organize the second predetermined number verification burst that data fragmentation is corresponding.
Correspondingly, described coding unit is to using every group of data fragmentation and corresponding verification burst as a stripe cell, to all digital signature that described stripe cell is corresponding encode (for example adopting JSON coding) form metadata; Described metadata storage unit stores each corresponding metadata of dividing into groups respectively in a plurality of back end of described distributed memory system.Wherein, the storage of metadata can adopt many copies scheme.
In addition, the embodiment of the present invention also provides a kind of distributed memory system that comprises above-mentioned data storage device.Particularly, described distributed memory system comprises access node and back end, and described data storage device is arranged in described access node.Certainly, when described distributed memory system is Cassandra system, each node, both as access node, exists as back end again.
In sum, in distributed memory system, by using erasable code, (verification mode is replaced existing many copies storage mode to the embodiment of the present invention, can guarantee, under the prerequisite of data reliability, to reduce redundant data accounting, thereby reach the object of saving storage space.
A1, a kind of date storage method, comprising:
The data block that the access node of distributed memory system writes user is cut into the data fragmentation of some pre-sizings;
Adopt redundancy check algorithm to calculate some verification bursts corresponding to some data fragmentations that cutting obtains;
Described data fragmentation and described verification burst are stored in the back end of described distributed memory system, and each data fragmentation is only stored a copy with each verification burst.
A2, the date storage method as described in claim A1, wherein, describedly store described data fragmentation and described verification burst in the back end of described distributed memory system into, comprising:
Calculate respectively the digital signature of each data fragmentation and each verification burst;
According to digital signature separately, each data fragmentation and each verification burst are stored into respectively in a back end of described distributed memory system.
A3, the date storage method as described in claim A2, wherein, also comprise:
All digital signature are encoded and formed metadata;
By described metadata store in a plurality of back end of described distributed memory system.
A4, the date storage method as described in claim A2, wherein, described employing redundancy check algorithm calculates some verification bursts corresponding to some data fragmentations that cutting obtains, and comprising:
If the individual data fragmentation that cutting is obtained is divided into some groups of data fragmentations, wherein, every group of data fragmentation comprises the first predetermined number data fragmentation;
Adopt redundancy check algorithm to calculate respectively and respectively organize the second predetermined number verification burst that data fragmentation is corresponding.
A5, the date storage method as described in claim A4, wherein, also comprise:
Using every group of data fragmentation and corresponding verification burst as a stripe cell, all digital signature corresponding to described stripe cell are encoded and formed metadata;
By described metadata store in a plurality of back end of described distributed memory system.
A6, the date storage method as described in claim A3 or A5, wherein, described in be encoded to JSON coding.
A7, the date storage method as described in claim A2, wherein, described distributed memory system is Cassandra system.
B8, a kind of data storage device, be arranged in the access node of distributed memory system, and described data storage device comprises:
Data cutting unit, the data block that is suitable for user to write is cut into the data fragmentation of some pre-sizings;
Verification computing unit, is suitable for adopting redundancy check algorithm to calculate some verification bursts corresponding to some data fragmentations that cutting obtains;
Unit of memory allocation, is suitable for described data fragmentation and described verification burst to store in the back end of described distributed memory system, and each data fragmentation is only stored a copy with each verification burst.
B9, the data storage device as described in claim B8, wherein, described unit of memory allocation is further adapted for:
Calculate respectively the digital signature of each data fragmentation and each verification burst;
According to digital signature separately, each data fragmentation and each verification burst are stored into respectively in a back end of described distributed memory system.
B10, the data storage device as described in claim B9, wherein, also comprise:
Coding unit, is suitable for all digital signature to encode and form metadata;
Metadata storage unit, is suitable for described metadata store in a plurality of back end of described distributed memory system.
B11, the data storage device as described in claim B9, wherein, described verification computing unit is further adapted for:
If the individual data fragmentation that cutting is obtained is divided into some groups of data fragmentations, wherein, every group of data fragmentation comprises the first predetermined number data fragmentation;
Adopt redundancy check algorithm to calculate respectively and respectively organize the second predetermined number verification burst that data fragmentation is corresponding.
B12, the data storage device as described in claim B11, wherein, also comprise:
Coding unit, is suitable for usining every group of data fragmentation and corresponding verification burst as a stripe cell, and all digital signature corresponding to described stripe cell are encoded and formed metadata;
Metadata storage unit, is suitable for described metadata store in a plurality of back end of described distributed memory system.
B13, the data storage device as described in claim B10 or B12, wherein, described in be encoded to JSON coding.
B14, the date storage method as described in claim B9, wherein, described distributed memory system is Cassandra system.
B15, a kind of distributed memory system that comprises the data storage device as described in any one in claim B8～B14.
The algorithm providing at this is intrinsic not relevant to any certain computer, virtual system or miscellaneous equipment with demonstration.Various general-purpose systems also can with based on using together with this teaching.According to description above, it is apparent constructing the desired structure of this type systematic.In addition, the present invention is not also for any certain programmed language.It should be understood that and can utilize various programming languages to realize content of the present invention described here, and the description of above language-specific being done is in order to disclose preferred forms of the present invention.
In the instructions that provided herein, a large amount of details have been described.Yet, can understand, embodiments of the invention can not put into practice in the situation that there is no these details.In some instances, be not shown specifically known method, structure and technology, so that not fuzzy understanding of this description.
Similarly, be to be understood that, in order to simplify the disclosure and to help to understand one or more in each inventive aspect, in the above in the description of exemplary embodiment of the present invention, each feature of the present invention is grouped together into single embodiment, figure or sometimes in its description.Yet, the method for the disclosure should be construed to the following intention of reflection: the present invention for required protection requires than the more feature of feature of clearly recording in each claim.Or rather, as reflected in claims below, inventive aspect is to be less than all features of disclosed single embodiment above.Therefore, claims of following embodiment are incorporated to this embodiment thus clearly, and wherein each claim itself is as independent embodiment of the present invention.
Those skilled in the art are appreciated that and can the module in the equipment in embodiment are adaptively changed and they are arranged in one or more equipment different from this embodiment.Module in embodiment or unit or assembly can be combined into a module or unit or assembly, and can put them into a plurality of submodules or subelement or sub-component in addition.At least some in such feature and/or process or unit are mutually repelling, and can adopt any combination to combine all processes or the unit of disclosed all features in this instructions (comprising claim, summary and the accompanying drawing followed) and disclosed any method like this or equipment.Unless clearly statement in addition, in this instructions (comprising claim, summary and the accompanying drawing followed) disclosed each feature can be by providing identical, be equal to or the alternative features of similar object replaces.
In addition, those skilled in the art can understand, although embodiment more described herein comprise some feature rather than further feature included in other embodiment, the combination of the feature of different embodiment means within scope of the present invention and forms different embodiment.For example, in the following claims, the one of any of embodiment required for protection can be used with array mode arbitrarily.
All parts embodiment of the present invention can realize with hardware, or realizes with the software module moved on one or more processor, or realizes with their combination.It will be understood by those of skill in the art that and can use in practice microprocessor or digital signal processor (DSP) to realize the some or all functions according to the some or all parts in the data storage device of the embodiment of the present invention.The present invention for example can also be embodied as, for carrying out part or all equipment or device program (, computer program and computer program) of method as described herein.Realizing program of the present invention and can be stored on computer-readable medium like this, or can there is the form of one or more signal.Such signal can be downloaded and obtain from internet website, or provides on carrier signal, or provides with any other form.
It should be noted above-described embodiment the present invention will be described rather than limit the invention, and those skilled in the art can design alternative embodiment in the situation that do not depart from the scope of claims.In the claims, any reference symbol between bracket should be configured to limitations on claims.Word " comprises " not to be got rid of existence and is not listed as element or step in the claims.Being positioned at word " " before element or " one " does not get rid of and has a plurality of such elements.The present invention can be by means of including the hardware of some different elements and realizing by means of the computing machine of suitably programming.In having enumerated the unit claim of some devices, several in these devices can be to carry out imbody by same hardware branch.The use of word first, second and C grade does not represent any order.Can be title by these word explanations.
Claims (10)
1. a date storage method, comprising:
The data block that the access node of distributed memory system writes user is cut into the data fragmentation of some pre-sizings;
Adopt redundancy check algorithm to calculate some verification bursts corresponding to some data fragmentations that cutting obtains;
Described data fragmentation and described verification burst are stored in the back end of described distributed memory system, and each data fragmentation is only stored a copy with each verification burst.
2. date storage method as claimed in claim 1, wherein, describedly stores described data fragmentation and described verification burst in the back end of described distributed memory system into, comprising:
Calculate respectively the digital signature of each data fragmentation and each verification burst;
According to digital signature separately, each data fragmentation and each verification burst are stored into respectively in a back end of described distributed memory system.
3. date storage method as claimed in claim 2, wherein, also comprises:
All digital signature are encoded and formed metadata;
By described metadata store in a plurality of back end of described distributed memory system.
4. date storage method as claimed in claim 2, wherein, described employing redundancy check algorithm calculates some verification bursts corresponding to some data fragmentations that cutting obtains, and comprising:
If the individual data fragmentation that cutting is obtained is divided into some groups of data fragmentations, wherein, every group of data fragmentation comprises the first predetermined number data fragmentation;
Adopt redundancy check algorithm to calculate respectively and respectively organize the second predetermined number verification burst that data fragmentation is corresponding.
5. date storage method as claimed in claim 4, wherein, also comprises:
Using every group of data fragmentation and corresponding verification burst as a stripe cell, all digital signature corresponding to described stripe cell are encoded and formed metadata;
By described metadata store in a plurality of back end of described distributed memory system.
6. the date storage method as described in claim 3 or 5, wherein, described in be encoded to JSON coding.
7. date storage method as claimed in claim 2, wherein, described distributed memory system is Cassandra system.
8. a data storage device, is arranged in the access node of distributed memory system, and described data storage device comprises:
Data cutting unit, the data block that is suitable for user to write is cut into the data fragmentation of some pre-sizings;
Verification computing unit, is suitable for adopting redundancy check algorithm to calculate some verification bursts corresponding to some data fragmentations that cutting obtains;
Unit of memory allocation, is suitable for described data fragmentation and described verification burst to store in the back end of described distributed memory system, and each data fragmentation is only stored a copy with each verification burst.
9. data storage device as claimed in claim 8, wherein, described unit of memory allocation is further adapted for:
Calculate respectively the digital signature of each data fragmentation and each verification burst;
According to digital signature separately, each data fragmentation and each verification burst are stored into respectively in a back end of described distributed memory system.
10. a distributed memory system that comprises the data storage device as described in any one in claim 8～9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310657140.1A CN103699494B (en) | 2013-12-06 | 2013-12-06 | A kind of date storage method, data storage device and distributed memory system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310657140.1A CN103699494B (en) | 2013-12-06 | 2013-12-06 | A kind of date storage method, data storage device and distributed memory system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103699494A true CN103699494A (en) | 2014-04-02 |
| CN103699494B CN103699494B (en) | 2017-03-15 |
Family
ID=50361029
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310657140.1A Expired - Fee Related CN103699494B (en) | 2013-12-06 | 2013-12-06 | A kind of date storage method, data storage device and distributed memory system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103699494B (en) |
Cited By (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104932953A (en) * | 2015-06-04 | 2015-09-23 | 华为技术有限公司 | Data distribution method, data storage method, and relevant device and system |
| CN105095013A (en) * | 2015-06-04 | 2015-11-25 | 华为技术有限公司 | Data storage method, data recovery method, and relevant device and system |
| CN105159607A (en) * | 2015-08-28 | 2015-12-16 | 浪潮(北京)电子信息产业有限公司 | Discrete storage based high-speed writing method |
| CN105357294A (en) * | 2015-10-31 | 2016-02-24 | 成都华为技术有限公司 | Method for data storage and cluster management node |
| CN106469100A (en) * | 2015-08-17 | 2017-03-01 | 华为技术有限公司 | A kind of method of data recovery, the corresponding apparatus and system of method of storage |
| CN106534273A (en) * | 2016-10-31 | 2017-03-22 | 中金云金融（北京）大数据科技股份有限公司 | Block chain metadata storage system, and storage method and retrieval method thereof |
| CN106776146A (en) * | 2016-12-29 | 2017-05-31 | 华为技术有限公司 | A kind of data verification method, apparatus and system |
| CN107066503A (en) * | 2017-01-05 | 2017-08-18 | 郑州云海信息技术有限公司 | The method and device of magnanimity metadata burst distribution |
| CN107219997A (en) * | 2016-03-21 | 2017-09-29 | 阿里巴巴集团控股有限公司 | A kind of method and device for being used to verify data consistency |
| CN107436733A (en) * | 2017-06-29 | 2017-12-05 | 华为技术有限公司 | Management by district method and management by district device |
| WO2018000812A1 (en) * | 2016-06-28 | 2018-01-04 | 华为技术有限公司 | Data storage method and apparatus |
| CN107943421A (en) * | 2017-11-30 | 2018-04-20 | 成都华为技术有限公司 | A kind of subregion partitioning method and device based on distributed memory system |
| CN107948233A (en) * | 2016-10-13 | 2018-04-20 | 华为技术有限公司 | The method of processing write requests or read request, interchanger, control node |
| CN107957919A (en) * | 2016-10-14 | 2018-04-24 | 北京京东尚科信息技术有限公司 | Data disaster tolerance system, method and apparatus |
| CN108491290A (en) * | 2018-03-28 | 2018-09-04 | 新华三技术有限公司 | A kind of method for writing data and device |
| CN108491167A (en) * | 2018-03-29 | 2018-09-04 | 重庆大学 | A kind of quick random distribution storage method of industrial process floor data |
| CN108733503A (en) * | 2017-04-24 | 2018-11-02 | 慧与发展有限责任合伙企业 | Data are stored in distributed memory system |
| WO2019000949A1 (en) * | 2017-06-28 | 2019-01-03 | 华为技术有限公司 | Metadata storage method and system in distributed storage system, and storage medium |
| CN109213431A (en) * | 2017-07-04 | 2019-01-15 | 阿里巴巴集团控股有限公司 | The consistency detecting method and device and electronic equipment of more copy datas |
| CN109358809A (en) * | 2018-09-28 | 2019-02-19 | 方信息科技（上海）有限公司 | A kind of RAID data storage system and method |
| CN109413207A (en) * | 2018-12-11 | 2019-03-01 | 深圳市网心科技有限公司 | A kind of file uploading method, system, device and computer readable storage medium |
| US10303374B2 (en) | 2016-11-25 | 2019-05-28 | Huawei Technologies Co.,Ltd. | Data check method and storage system |
| CN110278222A (en) * | 2018-03-15 | 2019-09-24 | 华为技术有限公司 | The method, system and relevant device of data management in distributed file storage system |
| WO2019184012A1 (en) * | 2018-03-30 | 2019-10-03 | 华为技术有限公司 | Data writing method, client server, and system |
| CN107229535B (en) * | 2017-05-23 | 2020-01-21 | 杭州宏杉科技股份有限公司 | Multi-copy storage method, storage device and data reading method for data block |
| CN110908835A (en) * | 2019-11-11 | 2020-03-24 | 华中科技大学 | Data redundancy method and system supporting private label in distributed system |
| TWI690192B (en) * | 2019-01-15 | 2020-04-01 | 臺灣網路認證股份有限公司 | System for providing signature entities to sign electronic document in order for generating signed document and method thereof |
| CN111381767A (en) * | 2018-12-28 | 2020-07-07 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN111399766A (en) * | 2020-01-08 | 2020-07-10 | 华为技术有限公司 | Data storage method, data reading method, device and system in storage system |
| CN111831297A (en) * | 2019-04-17 | 2020-10-27 | 中兴通讯股份有限公司 | Zero-difference upgrading method and device |
| WO2021046693A1 (en) * | 2019-09-09 | 2021-03-18 | 华为技术有限公司 | Data processing method in storage system, device, and storage system |
| CN112558875A (en) * | 2020-12-14 | 2021-03-26 | 北京百度网讯科技有限公司 | Data verification method and device, electronic equipment and storage medium |
| CN112799584A (en) * | 2019-11-13 | 2021-05-14 | 杭州海康威视数字技术股份有限公司 | Data storage method and device |
| WO2022199155A1 (en) * | 2021-03-24 | 2022-09-29 | 华为技术有限公司 | Data transmission system and method, and network device |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102752402A (en) * | 2012-07-20 | 2012-10-24 | 广东威创视讯科技股份有限公司 | Cloud storage method and cloud storage system |
-
2013
- 2013-12-06 CN CN201310657140.1A patent/CN103699494B/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102752402A (en) * | 2012-07-20 | 2012-10-24 | 广东威创视讯科技股份有限公司 | Cloud storage method and cloud storage system |
Cited By (65)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9823970B2 (en) | 2015-06-04 | 2017-11-21 | Huawei Technologies Co., Ltd. | Data storage method, data recovery method, related apparatus, and system |
| CN105095013A (en) * | 2015-06-04 | 2015-11-25 | 华为技术有限公司 | Data storage method, data recovery method, and relevant device and system |
| CN107844268B (en) * | 2015-06-04 | 2021-09-14 | 华为技术有限公司 | Data distribution method, data storage method, related device and system |
| CN107844268A (en) * | 2015-06-04 | 2018-03-27 | 华为技术有限公司 | A kind of data distributing method, date storage method, relevant apparatus and system |
| JP2017004513A (en) * | 2015-06-04 | 2017-01-05 | 華為技術有限公司Ｈｕａｗｅｉ Ｔｅｃｈｎｏｌｏｇｉｅｓ Ｃｏ．，Ｌｔｄ． | Data distribution method, data storage method, and relating device and system |
| CN107748702A (en) * | 2015-06-04 | 2018-03-02 | 华为技术有限公司 | Date storage method, restoration methods, relevant apparatus and system |
| CN104932953A (en) * | 2015-06-04 | 2015-09-23 | 华为技术有限公司 | Data distribution method, data storage method, and relevant device and system |
| US10133633B2 (en) | 2015-06-04 | 2018-11-20 | Huawei Technologies Co., Ltd. | Data storage method, data recovery method, related apparatus, and system |
| US9710331B2 (en) | 2015-06-04 | 2017-07-18 | Huawei Technologies Co., Ltd. | Data storage method, data recovery method, related apparatus, and system |
| CN104932953B (en) * | 2015-06-04 | 2017-11-21 | 华为技术有限公司 | A kind of data distributing method, date storage method, relevant apparatus and system |
| CN106469100A (en) * | 2015-08-17 | 2017-03-01 | 华为技术有限公司 | A kind of method of data recovery, the corresponding apparatus and system of method of storage |
| US10810091B2 (en) | 2015-08-17 | 2020-10-20 | Huawei Technologies Co., Ltd. | Data recovery method, data storage method, and corresponding apparatus and system |
| CN106469100B (en) * | 2015-08-17 | 2019-04-05 | 华为技术有限公司 | A kind of method that data are restored, the corresponding apparatus and system of the method for storage |
| CN105159607A (en) * | 2015-08-28 | 2015-12-16 | 浪潮(北京)电子信息产业有限公司 | Discrete storage based high-speed writing method |
| CN105357294B (en) * | 2015-10-31 | 2018-10-02 | 成都华为技术有限公司 | A kind of method and cluster management node of storage data |
| CN105357294A (en) * | 2015-10-31 | 2016-02-24 | 成都华为技术有限公司 | Method for data storage and cluster management node |
| CN107219997A (en) * | 2016-03-21 | 2017-09-29 | 阿里巴巴集团控股有限公司 | A kind of method and device for being used to verify data consistency |
| CN107219997B (en) * | 2016-03-21 | 2020-08-18 | 阿里巴巴集团控股有限公司 | Method and device for verifying data consistency |
| US10725692B2 (en) | 2016-06-28 | 2020-07-28 | Huawei Technologies Co., Ltd. | Data storage method and apparatus |
| WO2018000812A1 (en) * | 2016-06-28 | 2018-01-04 | 华为技术有限公司 | Data storage method and apparatus |
| CN107948233A (en) * | 2016-10-13 | 2018-04-20 | 华为技术有限公司 | The method of processing write requests or read request, interchanger, control node |
| CN107948233B (en) * | 2016-10-13 | 2021-01-08 | 华为技术有限公司 | Method for processing write request or read request, switch and control node |
| CN107957919A (en) * | 2016-10-14 | 2018-04-24 | 北京京东尚科信息技术有限公司 | Data disaster tolerance system, method and apparatus |
| CN106534273B (en) * | 2016-10-31 | 2022-04-15 | 中金云金融（北京）大数据科技股份有限公司 | Block chain metadata storage system and storage method and retrieval method thereof |
| CN106534273A (en) * | 2016-10-31 | 2017-03-22 | 中金云金融（北京）大数据科技股份有限公司 | Block chain metadata storage system, and storage method and retrieval method thereof |
| US10303374B2 (en) | 2016-11-25 | 2019-05-28 | Huawei Technologies Co.,Ltd. | Data check method and storage system |
| CN106776146A (en) * | 2016-12-29 | 2017-05-31 | 华为技术有限公司 | A kind of data verification method, apparatus and system |
| CN107066503A (en) * | 2017-01-05 | 2017-08-18 | 郑州云海信息技术有限公司 | The method and device of magnanimity metadata burst distribution |
| CN108733503B (en) * | 2017-04-24 | 2021-10-01 | 慧与发展有限责任合伙企业 | Storage node, distributed storage system and method for storing data |
| CN108733503A (en) * | 2017-04-24 | 2018-11-02 | 慧与发展有限责任合伙企业 | Data are stored in distributed memory system |
| CN107229535B (en) * | 2017-05-23 | 2020-01-21 | 杭州宏杉科技股份有限公司 | Multi-copy storage method, storage device and data reading method for data block |
| WO2019000949A1 (en) * | 2017-06-28 | 2019-01-03 | 华为技术有限公司 | Metadata storage method and system in distributed storage system, and storage medium |
| US11243706B2 (en) | 2017-06-29 | 2022-02-08 | Huawei Technologies Co., Ltd. | Fragment management method and fragment management apparatus |
| WO2019000950A1 (en) * | 2017-06-29 | 2019-01-03 | 华为技术有限公司 | Fragment management method and fragment management apparatus |
| CN107436733B (en) * | 2017-06-29 | 2020-11-06 | 华为技术有限公司 | Fragment management method and fragment management device |
| CN107436733A (en) * | 2017-06-29 | 2017-12-05 | 华为技术有限公司 | Management by district method and management by district device |
| CN109213431A (en) * | 2017-07-04 | 2019-01-15 | 阿里巴巴集团控股有限公司 | The consistency detecting method and device and electronic equipment of more copy datas |
| CN109213431B (en) * | 2017-07-04 | 2022-05-13 | 阿里巴巴集团控股有限公司 | Consistency detection method and device for multi-copy data and electronic equipment |
| CN107943421A (en) * | 2017-11-30 | 2018-04-20 | 成都华为技术有限公司 | A kind of subregion partitioning method and device based on distributed memory system |
| CN110278222A (en) * | 2018-03-15 | 2019-09-24 | 华为技术有限公司 | The method, system and relevant device of data management in distributed file storage system |
| CN110278222B (en) * | 2018-03-15 | 2021-09-14 | 华为技术有限公司 | Method, system and related device for data management in distributed file storage system |
| CN108491290B (en) * | 2018-03-28 | 2021-07-23 | 新华三技术有限公司 | Data writing method and device |
| CN108491290A (en) * | 2018-03-28 | 2018-09-04 | 新华三技术有限公司 | A kind of method for writing data and device |
| CN108491167A (en) * | 2018-03-29 | 2018-09-04 | 重庆大学 | A kind of quick random distribution storage method of industrial process floor data |
| CN108491167B (en) * | 2018-03-29 | 2020-12-04 | 重庆大学 | Industrial process working condition data rapid random distribution storage method |
| WO2019184012A1 (en) * | 2018-03-30 | 2019-10-03 | 华为技术有限公司 | Data writing method, client server, and system |
| US11579777B2 (en) | 2018-03-30 | 2023-02-14 | Huawei Technologies Co., Ltd. | Data writing method, client server, and system |
| CN110557964A (en) * | 2018-03-30 | 2019-12-10 | 华为技术有限公司 | Data writing method, client server and system |
| WO2019183958A1 (en) * | 2018-03-30 | 2019-10-03 | 华为技术有限公司 | Data writing method, client server, and system |
| CN110557964B (en) * | 2018-03-30 | 2021-12-24 | 华为技术有限公司 | Data writing method, client server and system |
| CN109358809B (en) * | 2018-09-28 | 2020-07-24 | 方一信息科技（上海）有限公司 | RAID data storage system and method |
| CN109358809A (en) * | 2018-09-28 | 2019-02-19 | 方信息科技（上海）有限公司 | A kind of RAID data storage system and method |
| CN109413207A (en) * | 2018-12-11 | 2019-03-01 | 深圳市网心科技有限公司 | A kind of file uploading method, system, device and computer readable storage medium |
| CN111381767A (en) * | 2018-12-28 | 2020-07-07 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN111381767B (en) * | 2018-12-28 | 2024-03-26 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| TWI690192B (en) * | 2019-01-15 | 2020-04-01 | 臺灣網路認證股份有限公司 | System for providing signature entities to sign electronic document in order for generating signed document and method thereof |
| CN111831297A (en) * | 2019-04-17 | 2020-10-27 | 中兴通讯股份有限公司 | Zero-difference upgrading method and device |
| WO2021046693A1 (en) * | 2019-09-09 | 2021-03-18 | 华为技术有限公司 | Data processing method in storage system, device, and storage system |
| CN110908835B (en) * | 2019-11-11 | 2022-07-12 | 华中科技大学 | Data redundancy method and system supporting private label in distributed system |
| CN110908835A (en) * | 2019-11-11 | 2020-03-24 | 华中科技大学 | Data redundancy method and system supporting private label in distributed system |
| CN112799584A (en) * | 2019-11-13 | 2021-05-14 | 杭州海康威视数字技术股份有限公司 | Data storage method and device |
| CN112799584B (en) * | 2019-11-13 | 2023-04-07 | 杭州海康威视数字技术股份有限公司 | Data storage method and device |
| CN111399766A (en) * | 2020-01-08 | 2020-07-10 | 华为技术有限公司 | Data storage method, data reading method, device and system in storage system |
| CN112558875A (en) * | 2020-12-14 | 2021-03-26 | 北京百度网讯科技有限公司 | Data verification method and device, electronic equipment and storage medium |
| WO2022199155A1 (en) * | 2021-03-24 | 2022-09-29 | 华为技术有限公司 | Data transmission system and method, and network device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103699494B (en) | 2017-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103699494A (en) | Data storage method, data storage equipment and distributed storage system | |
| US10387249B2 (en) | Migrating data slices within a dispersed storage network | |
| CN109213618B (en) | Method, apparatus and computer program product for managing a storage system | |
| CN102981927B (en) | Distributed raid-array storage means and distributed cluster storage system | |
| US9442800B2 (en) | Data migration between a raid memory and a dispersed storage network memory | |
| US8464137B2 (en) | Probabilistic multi-tier error correction in not-and (NAND) flash memory | |
| CN111149093B (en) | Data encoding, decoding and repairing method of distributed storage system | |
| US20180074900A1 (en) | First responder parities for storage array | |
| KR101572038B1 (en) | Correction of errors in a memory array | |
| EP2908254A1 (en) | Data redundancy implementation method and device | |
| US10353787B2 (en) | Data stripping, allocation and reconstruction | |
| KR20110102798A (en) | System and method for optimizing redundancy restoration in distributed data layout environments | |
| CN104598162A (en) | Method and system for writing data across storage devices | |
| CN102799533B (en) | Method and apparatus for shielding damaged sector of disk | |
| US10552062B2 (en) | System and method for storing very large key value objects | |
| CN105573680A (en) | Storage method and device for replicated data | |
| US9600189B2 (en) | Bank-level fault management in a memory system | |
| US11314594B2 (en) | Method, device and computer program product for recovering data | |
| CN112749039A (en) | Method, apparatus and program product for data writing and data recovery | |
| US20160342508A1 (en) | Identifying memory regions that contain remapped memory locations | |
| CN109144766B (en) | Data storage and reconstruction method and device and electronic equipment | |
| CN111857560A (en) | Method, apparatus and computer program product for managing data | |
| US20190114221A1 (en) | Coding technique | |
| US9450617B2 (en) | Distribution and replication of erasure codes | |
| US11481275B2 (en) | Managing reconstruction of a malfunctioning disk slice |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20170315 Termination date: 20211206 |