CN110874283B - Data storage method and device based on erasure codes and electronic equipment - Google Patents
Data storage method and device based on erasure codes and electronic equipment Download PDFInfo
- Publication number
- CN110874283B CN110874283B CN201811012704.5A CN201811012704A CN110874283B CN 110874283 B CN110874283 B CN 110874283B CN 201811012704 A CN201811012704 A CN 201811012704A CN 110874283 B CN110874283 B CN 110874283B
- Authority
- CN
- China
- Prior art keywords
- data
- memory
- stored
- preset
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
- G06F11/1044—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices with specific ECC/EDC distribution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1008—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
- G06F11/1072—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices in multilevel memories
Abstract
The embodiment of the invention provides an erasure code-based data storage method and device and electronic equipment. The method comprises the following steps: writing data to be stored into a first memory; when the data to be stored in the first memory reaches the data volume of the data blocks with the target quantity, calculating a first check value corresponding to the data to be stored in the first memory; the XOR operation result of the first check value and the current content in the second memory is stored in the second memory in a covering mode, the data to be stored in the first memory is transferred to preset storage equipment, and the data to be stored is returned to be written into the first memory space; and when the data to be stored based on the current content in the second memory reaches the data storage amount corresponding to the preset EC type, taking the current content in the second memory as a check value of the stripe block specified by the preset EC type and transferring the check value to the storage device. Compared with the prior art, the scheme provided by the embodiment of the invention can reduce the memory space occupied in the storage process.
Description
Technical Field
The invention relates to the technical field of cloud storage, in particular to a data storage method and device based on erasure codes and electronic equipment.
Background
Erasure Coding (EC) is a method of data protection that partitions data into fragments, expands, codes, and stores redundant data blocks in different locations. From a data function perspective, the protection provided by erasure codes can be represented by the following simple formula: n is k + m, where the variable "k" represents the value of original data, the variable "m" represents the value of added additional data for providing protection, and the variable "n" represents the total value of data created after the erasure coding process, and the added additional data for providing protection may be simply referred to as a check value for convenience of description.
In the prior art, in order to implement erasure code-based data storage, a first memory for storing data to be stored and a second memory for storing a check value of the data to be stored are applied in advance, where the size of the first memory is not less than a data storage amount corresponding to an EC type to be utilized. The specific storage process is as follows:
writing data to be stored acquired from a data source into a first memory; when the data to be stored in the first memory reaches the data storage amount, calculating a check value corresponding to the data to be stored in the first memory based on the generating matrix matched with the preset EC type, and storing the check value in the second memory; and then, transferring the data to be stored in the first memory and the check value in the second memory to a preset storage device to finish the storage of one stripe block.
It can be seen that the memory amount of the first memory on which the data storage depends must be not less than the data storage amount corresponding to the EC type, which results in a large memory space occupied in the data storage process. Especially, when the data storage capacity corresponding to the EC type is large or multiple paths of data to be stored are processed simultaneously, the problem of large occupied memory space is more obvious.
Disclosure of Invention
The embodiment of the invention aims to provide an erasure code-based data storage method, an erasure code-based data storage device and electronic equipment, so as to reduce the memory space occupied in the storage process.
The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a data storage method based on erasure codes, where the method includes:
writing data to be stored into a first memory which is applied in advance; the target number of the data blocks which can be accommodated by the first memory is smaller than the total number of the data blocks specified by a preset EC type;
when the data to be stored in the first memory reaches the data volume of the data blocks with the target number, calculating a first check value corresponding to the data to be stored in the first memory based on the generator matrix matched with the preset EC type;
covering and storing the calculated exclusive or operation result of the first check value and the current content in the second memory into the second memory, transferring the data to be stored in the first memory into preset storage equipment, and returning to the step of writing the data to be stored into the first memory space;
and when the data to be stored based on the current content in the second memory reaches the data storage amount corresponding to the preset EC type, taking the current content in the second memory as a check value of the stripe block specified by the preset EC type and transferring the check value to the storage device to finish the storage of one stripe block.
As an implementation manner of the embodiment of the present invention, the step of calculating a first check value corresponding to data to be stored in the first memory based on the generator matrix matched with the preset EC type includes:
according to the block size of the data block, dividing the data block to be stored in the first memory to obtain at least one data block;
and calculating a first check value corresponding to the at least one data block based on the generator matrix matched with the preset EC type.
As an implementation manner of the embodiment of the present invention, the step of calculating the first check value corresponding to the at least one data block based on the generator matrix matched with the preset EC type includes:
calculating a first check value corresponding to the at least one data block through a predetermined calculation formula based on the generator matrix matched with the preset EC type;
wherein the predetermined calculation formula is:
wherein, CmM is more than or equal to 0 and less than or equal to M-1 for a first check value in the mth check block corresponding to the data to be stored in the first memory, M is the number of the check blocks specified by the EC type,
is an exclusive or operation;
Dithe data block corresponding to the ith data block in the stripe block specified by the preset EC type is selected from the at least one data block; xm,i、Xm,i+1、…、Xm,i+Q-1Matched to the EC typeAnd forming elements in the matrix, wherein Q is the target quantity.
As an implementation manner of the embodiment of the present invention, the method further includes:
when the data to be stored in the first memory does not reach the data volume of the data blocks with the target quantity, but the sum of the data to be stored based on the current content in the second memory and the data volume of the data to be stored in the first memory reaches the data storage volume, calculating a first check value corresponding to the data to be stored in the first memory based on a generating matrix matched with the preset EC type;
the XOR operation result of the first check value obtained by calculation and the current content in the second memory is stored in the second memory in a covering mode;
and transferring the data to be stored in the first memory to a preset storage device, and transferring the current content in the second memory to the storage device as a check value of the stripe block specified by the preset EC type, so as to complete the storage process of one stripe block.
As an implementation manner of the embodiment of the present invention, the total number of the data blocks is an integer multiple of the target number; the memory amount of the first memory is: the product of the amount of data per data block and the target number.
In a second aspect, an embodiment of the present invention provides an erasure code-based data storage apparatus, where the apparatus includes:
the data writing module is used for writing data to be stored into a first memory which is applied in advance; the target number of the data blocks which can be accommodated by the first memory is smaller than the total number of the data blocks specified by a preset EC type;
the first calculation module is used for calculating a first check value corresponding to the data to be stored in the first memory based on the generator matrix matched with the preset EC type when the data to be stored in the first memory reaches the data volume of the data blocks with the target number;
the calculation result processing module is used for storing the calculated exclusive or operation result of the first check value and the current content in the second memory into the second memory in a covering manner, transferring the data to be stored in the first memory into a preset storage device, and returning the data to the data writing module;
and the first data storage module is used for taking the current content in the second memory as a check value of the stripe block specified by the preset EC type and transferring the check value to the storage device to finish the storage of one stripe block when the data to be stored based on the current content in the second memory reaches the data storage amount corresponding to the preset EC type.
As an implementation manner of the embodiment of the present invention, the check value calculating module includes:
the data dividing submodule is used for dividing the data to be stored in the first memory into data blocks according to the block size of the data blocks to obtain at least one data block;
and the check value operator module is used for calculating a first check value corresponding to the at least one data block based on the generator matrix matched with the preset EC type.
As an implementation manner of the embodiment of the present invention, the check value operator module is specifically configured to:
calculating a first check value corresponding to the at least one data block through a predetermined calculation formula based on the generator matrix matched with the preset EC type;
wherein the predetermined calculation formula is:
wherein, CmM is more than or equal to 0 and less than or equal to M-1 for a first check value in the mth check block corresponding to the data to be stored in the first memory, M is the number of the check blocks specified by the EC type,
is an exclusive or operation;
Dithe data block corresponding to the ith data block in the stripe block specified by the preset EC type is selected from the at least one data block; xm,i、Xm,i+1、…、Xm,i+Q-1And Q is the target number for the element in the generator matrix matched with the EC type.
As an implementation manner of the embodiment of the present invention, the apparatus further includes:
a second calculation module, configured to calculate, when the data to be stored in the first memory does not reach the data amount of the data blocks of the target number, but a sum of the data to be stored based on current content in the second memory and the data amount of the data to be stored in the first memory reaches the data storage amount, a first check value corresponding to the data to be stored in the first memory based on a generation matrix matched with the preset EC type;
the XOR operation module is used for covering and storing the XOR operation result of the calculated first check value and the current content in the second memory into the second memory;
and the second data storage module is used for transferring the data to be stored in the first memory to a preset storage device, and transferring the current content in the second memory to the storage device as a check value of the stripe block specified by the preset EC type, so as to complete the storage process of one stripe block.
As an implementation manner of the embodiment of the present invention, the total number of the data blocks is an integer multiple of the target number;
the memory amount of the first memory is: the product of the amount of data per data block and the target number.
In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;
a memory for storing a computer program;
a processor, configured to implement any of the method steps of the erasure code-based data storage method provided in the first aspect when executing a program stored in the memory.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements any of the method steps of the erasure code based data storage method provided in the first aspect.
As can be seen from the above, in the scheme provided in the embodiment of the present invention, the target number of the data blocks that can be accommodated by the first memory for writing the data to be stored is less than the total number of the data blocks that is specified by the preset EC type, and when the check value of the stripe block that is specified by the EC type is used, the total number of the data blocks that is specified by the EC type may be calculated in segments according to the target data of the data blocks that can be accommodated by the first memory, and the check value of the stripe block is obtained through an xor operation. This reduces the memory space occupied during storage. Obviously, when the data storage amount corresponding to the EC type is large or multiple paths of data to be stored are processed simultaneously, the memory space occupied in the storage process can be greatly reduced, so that the utilization rate of a Central Processing Unit (CPU) can be improved. In addition, when the obtained data to be stored reaches the data storage amount corresponding to the preset EC type, the data to be stored does not need to be stored in the storage device, but the data to be stored can be stored in the storage device when less data to be stored are obtained, so that the probability of losing or making mistakes of the obtained data to be stored can be reduced, and the reliability of the data to be stored in the storage device is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram illustrating a prior art process for calculating a check value stored based on an erasure code;
fig. 2 is a schematic flowchart of a data storage method based on erasure codes according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a specific example of an erasure code-based data storage method according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of an erasure code-based data storage device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Currently, in a data storage method based on a check code, since the memory amount of a first memory on which data storage depends must not be less than the data storage amount corresponding to an EC type, the memory space occupied in the data storage process is large. Especially, when the data storage capacity corresponding to the EC type is large or multiple paths of data to be stored are processed simultaneously, the problem of large occupied memory space is more obvious. In order to solve the problems in the prior art, embodiments of the present invention provide an erasure code-based data storage method and apparatus, and an electronic device.
In order to facilitate understanding of the solutions provided by the embodiments of the present invention, the following first introduces related technical terms related to the embodiments of the present invention.
The EC type can be expressed by the formula: k + m, where k represents the number of data blocks included in each stripe block, in which data to be stored is stored, when the data to be stored is stored in the form of stripe blocks in a specified storage device; m represents the number of check blocks included in each stripe block when data to be stored is stored in a designated storage device in the form of stripe blocks, the data blocks store check values corresponding to the data to be stored in the data blocks, and the check values are calculated based on the data to be stored in the data blocks and a generator matrix obtained based on a preset stripe EC type.
The data amount of each data block is the same as that of each parity block, and the sum of the data amount of the data to be stored and the parity value included in each stripe block may be referred to as a stripe depth. The data storage amount corresponding to the EC type is the product of the total number of data blocks specified by the preset EC type and the data amount of each data block. The data amount of each data block can be determined according to the requirements in the actual application and/or the device type in the actual application.
For example, if the EC type of a stripe is 4+2, it indicates that when the data to be stored is stored in the designated storage device in the form of a stripe, each stripe includes 4 data blocks for storing the data to be stored and 2 parity blocks for storing parity values of the data to be stored.
According to the EC type k + m of the stripe, a generator matrix used when calculating the check value corresponding to the stripe block may be constructed. The generator matrix is formed by splicing a k × k unit matrix and a k × m construction matrix, as shown in the leftmost matrix of fig. 1.
As shown in fig. 1, when the EC type is k + m, the leftmost matrix is a generator matrix matched with the EC type based on the EC type and a correlation algorithm; intermediate D0-Dk-1After the data to be stored written in the first memory is divided according to the block size of the data block specified by the preset EC type, the data block corresponding to the data block in the stripe block specified by the preset EC type is obtained, and the k data blocks form a matrix of k × 1, which may be referred to as a data matrix.
Further, by operating on the above-mentioned generator matrix and data matrix, the rightmost matrix in fig. 1 can be obtained, which can be called data/check matrix, whereinC in the data/check matrix0-Cm-1And the check value of the data to be stored corresponding to the check block in the stripe block specified by the preset EC type is obtained.
In the example shown in fig. 1, it can be calculated that:
…
wherein the content of the first and second substances,
is an exclusive or operation.
For example, assuming that the type EC type of the stripe is 4+2, it can be calculated as:
next, a data storage method based on erasure codes according to an embodiment of the present invention is described.
The erasure code-based data storage method provided by the embodiment of the invention can be applied to any electronic device, for example, a mobile phone, a tablet computer, a notebook computer, a desktop computer, and the like, and is not particularly limited herein, and will be referred to as an electronic device hereinafter.
It should be noted that, before executing the erasure code-based data storage method provided by the embodiment of the present invention, the electronic device may obtain the preset EC type. Therefore, the electronic device may obtain the generator matrix matched with the preset EC type based on the preset EC type and the related algorithm. In addition, the electronic device may further obtain a preset block size of each data block, so that a data storage amount corresponding to a preset EC type may be obtained.
In addition, in the process of executing the erasure code-based data storage method provided by the embodiment of the present invention, the electronic device needs to store the acquired data to be stored and the calculated check value in the memory, so that the electronic device may apply in advance for a first memory for caching the data to be stored and a second memory for caching the check value.
The target data of the data blocks that can be accommodated by the first memory is smaller than the total number of the data blocks specified by the preset EC type, the memory amount of the second memory is not smaller than the product of the total number of the check blocks specified by the preset EC type and the data amount of each check block, and the initial data content currently stored by the second memory is empty, that is, the applied second memory does not initially store the data content. Generally, in order to reduce the memory space occupied during the data storage process, the memory amount of the second memory may be equal to the product of the total number of parity chunks specified by the aforementioned predetermined EC type and the data amount of each parity chunk.
Fig. 2 is a schematic flowchart of a data storage method based on erasure codes according to an embodiment of the present invention, and as shown in fig. 2, the method may include the following steps:
s201: writing data to be stored into a first memory which is applied in advance;
after the preparation work is completed, the data to be stored, which can be obtained by the electronic device, is written into the first memory applied in advance. Wherein, the present invention does not limit the source of the data to be stored.
Since the target data of the data blocks that can be accommodated by the first memory is smaller than the total number of the data blocks specified by the preset EC type, the memory amount of the first memory is smaller than the data storage amount corresponding to the preset EC type.
S202: when the data to be stored in the first memory reaches the data volume of the data blocks with the target number, calculating a first check value corresponding to the data to be stored in the first memory based on a generating matrix matched with a preset EC type;
the generating matrix matched with the preset EC type is the generating matrix matched with the preset EC type, which is obtained by the electronic equipment based on the preset EC type and a related algorithm.
S203: covering and storing the calculated exclusive or operation result of the first check value and the current content in the second memory into the second memory, transferring the data to be stored in the first memory into a preset storage device, and returning to execute the step S201;
wherein the exclusive-OR operation is applied to a logical operation, which is exclusive OR, abbreviated to xor, and denoted by mathematical symbol "
"XOR algorithm is:
the truth table of the exclusive-or logic is shown in table 1, i.e., if A, B values are not the same, the exclusive-or result P is 1. If A, B the values are the same, the XOR result P is 0.
TABLE 1
It should be noted that, in step S203, the transferring the data to be stored in the first memory to the preset storage device includes: and storing the data to be stored in the first memory in preset storage equipment, and emptying the first memory. The storage device may be any device that can store data to be stored, such as a flash disk, a magnetic disk, a cloud disk, and the like, and the embodiment of the present invention is not particularly limited.
S204: and when the data to be stored based on the current content in the second memory reaches the data storage amount corresponding to the preset EC type, taking the current content in the second memory as a check value of the stripe block specified by the preset EC type and transferring the check value to the storage device to finish the storage of one stripe block.
After writing the data to be stored into the first memory each time, the electronic device may determine whether the data to be stored in the first memory reaches the data amount of the target number of data blocks, and determine whether to execute the step S202 according to the determination result.
Specifically, when it is determined that the data amount of the data blocks in the first memory reaches the target amount, the electronic device may calculate a first check value corresponding to the data to be stored in the first memory based on a generator matrix matched with the preset EC type.
And then, the calculated exclusive or operation result of the first check value and the current content in the second memory is stored in the second memory in a covering manner, the data to be stored in the first memory is transferred to a preset storage device, and the step S201 is executed in a returning manner.
Thus, the electronic device may execute the steps S201 to S203 in a loop until the data to be stored, based on the current content in the second memory, reaches the data storage amount corresponding to the preset EC type, and the electronic device may execute the step S204, and use the current content in the second memory as the check value of the stripe block specified by the preset EC type and store the check value in the storage device, thereby completing the storage of one stripe block.
In step S203, the electronic device may, while overwriting and storing the calculated xor operation result in the second memory, transfer the data to be stored in the first memory to a preset storage device, that is, overwrite and store the calculated xor operation result in the second memory and transfer the data to be stored in the first memory to the preset storage device; the electronic device may also overwrite and store the calculated xor operation result in the second memory, and then transfer the data to be stored in the first memory to a preset storage device; the electronic device may also first transfer the data to be stored in the first memory to a preset storage device, and then store the calculated exclusive or operation result in the second memory in a covering manner. This is all reasonable.
Optionally, in a specific implementation manner, in the step S202, a manner of calculating a first check value corresponding to data to be stored in the first memory based on the generator matrix matched with the preset EC type may include:
step A1: according to the block size of the data block, dividing the data block to be stored in the first memory to obtain at least one data block;
specifically, when the data to be stored in the first memory reaches the data amount of the data blocks of the target number, that is, when the first memory is full, the electronic device may divide the data blocks according to the writing order of the data to be stored based on the block size of each data block obtained in the preparation work, so as to obtain at least one data block.
Step A2: and calculating a check value corresponding to at least one data block based on the generator matrix matched with the preset EC type.
After obtaining the at least one data block, the electronic device may calculate a check value corresponding to the at least one data block based on the generator matrix matched with the preset EC type.
Specifically, in the step a2, based on the generator matrix matched with the preset EC type, the manner of calculating the check value corresponding to at least one data block may be:
calculating a check value corresponding to at least one data block through a preset calculation formula based on a generator matrix matched with a preset EC type;
wherein the predetermined calculation formula is:
wherein, CmM is more than or equal to 0 and less than or equal to M-1 for a first check value in the mth check block corresponding to the data to be stored in the first memory, M is the number of the check blocks specified by the EC type,
is an exclusive or operation;
Dithe method comprises the steps that a data block corresponding to the ith data block in a stripe block specified by a preset EC type is selected from at least one data block; xm,i、Xm,i+1、…、Xm,i+Q-1Q is the target number for the element in the generator matrix that the EC type matches.
It should be noted that, on the premise that the target data of the data blocks that can be accommodated by the first memory is smaller than the total number of the data blocks specified by the preset EC type, the target data of the data blocks that can be accommodated by the first memory may be set according to an actual situation, which is not limited in the embodiment of the present invention.
Optionally, in a specific implementation manner, the total number of data blocks specified by the preset EC type is an integer multiple of a target number of data blocks that can be accommodated by the first memory; the memory amount of the first memory is: the product of the amount of data per data block and the target number.
In this implementation, when the total number of data blocks specified by the preset EC type is an integer multiple of the target number of data blocks that can be accommodated by the first memory, assuming that the integer multiple is N, when it is determined that the data to be stored in the first memory reaches the data amount of the data blocks of the target number N times, after the step S202-the step S203, the data to be stored based on the current content in the second memory reaches the data storage amount corresponding to the preset EC type. Further, the electronic device may perform the step S204 to complete the storage of one stripe block.
Correspondingly, when the total number of the data blocks specified by the preset EC type is not an integer multiple of the target number of the data blocks that can be accommodated by the first memory, after the electronic device has performed the steps S201 to S203 for several times, and writes a certain amount of data to be stored into the first memory again, although the data to be stored in the first memory does not reach the data amount of the data blocks of the target number at this time, the sum of the data to be stored based on the current content in the second memory and the data amount of the data to be stored in the first memory at this time reaches the data storage amount corresponding to the preset EC type, the electronic device still needs to calculate the first check value corresponding to the data to be stored in the first memory at this time.
Therefore, optionally, in another specific implementation manner, a data storage method based on erasure codes provided by the embodiment of the present invention may further include:
step B1: and when the data to be stored in the first memory does not reach the data volume of the data blocks with the target quantity, but the sum of the data to be stored based on the current content in the second memory and the data volume of the data to be stored in the first memory reaches the data storage volume, calculating a first check value corresponding to the data to be stored in the first memory based on a generating matrix matched with the preset EC type.
Step B2: and covering and storing the calculated exclusive OR operation result of the first check value and the current content in the second memory into the second memory.
It should be noted that, in the step B1, the manner of calculating, by the electronic device, the first check value corresponding to the data to be stored in the first memory based on the generator matrix matched with the preset EC type may be the same as that in the step S202; in the step B2, the manner of overwriting and storing the calculated exclusive or operation result of the first check value and the current content in the second memory may be the same as that in the step S203, and is not described herein again.
Step B3: and transferring the data to be stored in the first memory to a preset storage device, and transferring the current content in the second memory to the storage device as a check value of the stripe block specified by the preset EC type to complete the storage process of one stripe block.
Obviously, after the electronic device has finished performing the step B2, if the data to be stored based on the current content in the second memory reaches the data storage amount corresponding to the preset EC type, the electronic device may continue to perform the step B3, and forward the data to be stored in the first memory to the preset storage device, and forward the current content in the second memory to the storage device as the check value of the stripe block specified by the preset EC type, thereby completing the storage process of one stripe block.
As can be seen from the above, in the scheme provided in the embodiment of the present invention, the target number of the data blocks that can be accommodated by the first memory for writing the data to be stored is less than the total number of the data blocks that is specified by the preset EC type, and when the check value of the stripe block that is specified by the EC type is used, the total number of the data blocks that is specified by the EC type may be calculated in segments according to the target data of the data blocks that can be accommodated by the first memory, and the check value of the stripe block is obtained through an xor operation. This reduces the memory space occupied during storage. Obviously, when the data storage amount corresponding to the EC type is large or multiple paths of data to be stored are processed simultaneously, the memory space occupied in the storage process can be greatly reduced, so that the utilization rate of a Central Processing Unit (CPU) can be improved. In addition, when the obtained data to be stored reaches the data storage amount corresponding to the preset EC type, the data to be stored does not need to be stored in the storage device, but the data to be stored can be stored in the storage device when less data to be stored are obtained, so that the probability of losing or making mistakes of the obtained data to be stored can be reduced, and the reliability of the data to be stored in the storage device is improved.
In order to facilitate the data storage method based on erasure codes provided in the above embodiments of the present invention, the method is exemplified below.
Suppose that: and if the preset EC type is 4+2, the stripe block specified by the preset EC type includes 4 data blocks and 2 parity blocks, where the data blocks may be represented by D0, D1, D2, and D3, respectively, and the parity blocks may be represented by P0 and P1. In this way, for one stripe block, the preset storage device includes storage spaces corresponding to 4 data blocks D0, D1, D2, and D3 for storing data to be stored, and storage spaces corresponding to 2 parity blocks P0 and P1 for storing parity values.
Based on the preset EC type and the related algorithm, a generating matrix matched with the preset EC type can be obtained, and the generating matrix is formed by splicing a 4 x 4 unit matrix and a2 x 4 construction matrix. That is, when k is 4 and m is 2, the matrix format shown at the leftmost side in fig. 1 is the representation format of the generator matrix.
The target number of data blocks that can be accommodated by the pre-applied first memory is 1, that is, the memory amount of the first memory is the data amount of one data block, and the data content currently stored in the pre-applied second memory is empty.
Then, as shown in fig. 3, the storage process of one stripe block is specifically explained:
(1) after the data to be stored is obtained, writing the data to be stored into a preset first memory, and when the data volume of the data to be stored written into the first memory reaches the data volume of 1 data block, dividing the data to be stored to obtain a data block, where the data block corresponds to a data block D0 in a stripe block specified by a preset EC type.
In this way, the first check value corresponding to the data block may be calculated based on the generator matrix matched with the preset EC type, and specifically, the first check value corresponding to the data block may be as follows:
P10＝X0,0*D0，P11＝X1,0d0, wherein X0,0And X1,0Elements in the construction matrix in the generation matrix are described above.
At this time, if the data content currently stored in the second memory is empty, the result of the xor operation between the first check value and the current content in the second memory is the above-mentioned P10 and P11, and further, P10 and P11 may be overlaid on the second memory, so that the current content in the second memory becomes P10 and P11.
Meanwhile, the data to be stored in the first memory may be transferred to the storage space corresponding to the data block D0 in the preset storage device.
(2) After the data to be stored in the first memory is transferred to the preset storage device, the first memory is emptied, the obtained data to be stored may be continuously written into the first memory again, and when the data amount of the data to be stored written into the first memory reaches the data amount of 1 data block again, the data block corresponds to the data block D1 in the stripe block specified by the preset EC type.
In this way, the first check value corresponding to the data block may be calculated based on the generator matrix matched with the preset EC type, and specifically, the first check value corresponding to the data block may be as follows:
P20’＝X0,1*D1，P21’＝X1,1d1, wherein X0,1And X1,1Elements in the construction matrix in the generation matrix are described above.
At this time, if the data content currently stored in the second memory is P10 and P11, the xor operation result of P20 'and P21' and the xor operation result of P10 and P11 obtained by calculation may be covered in the second memory, so that the current content in the second memory becomes the xor budget result obtained by calculation, and the data to be stored in the first memory is transferred to the storage space corresponding to the data block D1 in the preset storage device again.
Wherein, the XOR budget result is shown as the following formula:
(3) After the data to be stored written in the first memory in (2) is transferred to the preset storage device, the first memory is emptied, so that the obtained data to be stored can be written into the first memory again, and when the data amount of the data to be stored written in the first memory reaches the data amount of 1 data block again, the data block corresponds to the data block D2 in the stripe block specified by the preset EC type.
In this way, the first check value corresponding to the data block may be calculated based on the generator matrix matched with the preset EC type, and specifically, the first check value corresponding to the data block may be as follows:
P30’＝X0,2*D2，P31’＝X1,2d2, wherein X0,2And X1,2Elements in the construction matrix in the generation matrix are described above.
At this time, if the data content currently stored in the second memory is P20 and P21, the xor operation result of P30 'and P31' and the xor operation result of P20 and P21 obtained by calculation may be covered in the second memory, so that the current content in the second memory becomes the xor budget result obtained by calculation, and the data to be stored in the first memory is transferred to the storage space corresponding to the data block D2 in the preset storage device again. Wherein, the XOR budget result is shown as the following formula:
(4) after the data to be stored written in the first memory in (3) is transferred to the preset storage device, the first memory is emptied, and the data to be stored which is continuously acquired can be written into the first memory again, and when the data amount of the data to be stored written in the first memory reaches the data amount of 1 data block again, the data block corresponds to the data block D3 in the stripe block specified by the preset EC type.
In this way, the first check value corresponding to the data block may be calculated based on the generator matrix matched with the preset EC type, and specifically, the first check value corresponding to the data block may be as follows:
P40’＝X0,3*D3，P41’＝X1,3d3, wherein X0,3And X1,3Elements in the construction matrix in the generation matrix are described above.
At this time, if the data content currently stored in the second memory is P30 and P31, the xor operation result of P40 'and P41' and the xor operation result of P30 and P31 obtained by calculation may be covered in the second memory, so that the current content in the second memory becomes the xor budget result obtained by calculation, and the data to be stored in the first memory is transferred to the storage space corresponding to the data block D3 in the preset storage device again. Wherein, the XOR budget result is shown as the following formula:
(5) at this time, the current contents in the second memory are P40 and P41, and according to the above (1) - (4), in the process of calculating P40 and P41, the data to be stored based on reaches the data storage amount corresponding to the preset EC type. In this way, the current contents P40 and P41 in the second memory can be used as the check values of the stripe blocks specified by the preset EC type, and further, the current contents P40 and P41 can be transferred to the storage spaces corresponding to the check blocks P0 and P1 in the preset storage device.
At this time, one of the stripe blocks specified by the EC type described above, which includes the data to be stored in the storage spaces corresponding to the data blocks D0, D1, D2, and D3 and the parity values P40 and P41 in the storage spaces corresponding to the parity blocks P0 and P1, is completely stored. The parity check values corresponding to the parity check blocks P0 and P1 in the stripe block specified by the EC type are respectively:
corresponding to the erasure code-based data storage method provided by the embodiment of the invention, the embodiment of the invention also provides an erasure code-based data storage device.
Fig. 4 is a schematic structural diagram of an erasure code-based data storage device according to an embodiment of the present invention, and as shown in fig. 4, the erasure code-based data storage device may include the following modules:
a data writing module 410, configured to write data to be stored into a first memory that is applied for in advance;
the target number of the data blocks which can be accommodated by the first memory is smaller than the total number of the data blocks specified by the preset EC type;
a first calculating module 420, configured to calculate, when data to be stored in the first memory reaches a data amount of a data block of a target number, a first check value corresponding to the data to be stored in the first memory based on a generator matrix matched with a preset EC type;
a calculation result processing module 430, configured to store the calculated exclusive or operation result of the first check value and the current content in the second memory in a covering manner, transfer the data to be stored in the first memory to a preset storage device, and return the data to the data writing module 410;
the first storage module 440 is configured to, when the data to be stored based on the current content in the second memory reaches the data storage amount corresponding to the preset EC type, use the current content in the second memory as a check value of a stripe block specified by the preset EC type and forward the check value to the storage device, so as to complete storage of one stripe block.
As can be seen from the above, in the scheme provided in the embodiment of the present invention, the target number of the data blocks that can be accommodated by the first memory for writing the data to be stored is less than the total number of the data blocks that is specified by the preset EC type, and when the check value of the stripe block that is specified by the EC type is used, the total number of the data blocks that is specified by the EC type may be calculated in segments according to the target data of the data blocks that can be accommodated by the first memory, and the check value of the stripe block is obtained through an xor operation. This reduces the memory space occupied during storage. Obviously, when the data storage amount corresponding to the EC type is large or multiple paths of data to be stored are processed simultaneously, the memory space occupied in the storage process can be greatly reduced, so that the utilization rate of a Central Processing Unit (CPU) can be improved. In addition, when the obtained data to be stored reaches the data storage amount corresponding to the preset EC type, the data to be stored does not need to be stored in the storage device, but the data to be stored can be stored in the storage device when less data to be stored are obtained, so that the probability of losing or making mistakes of the obtained data to be stored can be reduced, and the reliability of the data to be stored in the storage device is improved.
Optionally, in a specific implementation manner, the first calculating module 420 may include:
a data block division submodule (not shown in fig. 4) configured to perform data block division on data to be stored in the first memory according to a block size of a data block to obtain at least one data block;
and the check value operator module (not shown in fig. 4) is configured to calculate a first check value corresponding to the at least one data block based on the generator matrix matched with the preset EC type.
Optionally, in a specific implementation manner, the check value operator module (not shown in fig. 4) may be specifically configured to:
calculating a check value corresponding to at least one data block through a preset calculation formula based on a generator matrix matched with a preset EC type;
wherein, the predetermined calculation formula is:
wherein, CmM is more than or equal to 0 and less than or equal to M-1 for a first check value in the mth check block corresponding to the data to be stored in the first memory, M is the number of the check blocks specified by the EC type,
is an exclusive or operation;
Dithe method comprises the steps that a data block corresponding to the ith data block in a stripe block specified by a preset EC type is selected from at least one data block; xm,i、Xm,i+1、…、Xm,i+Q-1Q is the target number for the element in the generator matrix that the EC type matches.
Optionally, in a specific implementation manner, the erasure code-based data storage device may further include:
a second calculating module (not shown in fig. 4) configured to calculate, when the data amount of the data block of which the number of data to be stored in the first memory does not reach the target number and the sum of the data amount of the data to be stored based on the current content in the second memory and the data amount of the data to be stored in the first memory reaches the data storage amount, a first check value corresponding to the data to be stored in the first memory based on a generator matrix matched with the preset EC type;
an exclusive-or operation module (not shown in fig. 4) configured to store the calculated exclusive-or operation result of the first check value and the current content in the second memory in a covering manner;
a second data storage module (not shown in fig. 4), configured to forward the data to be stored in the first memory to a preset storage device, and forward the current content in the second memory to the storage device as a check value of the stripe block specified by the preset EC type, so as to complete a storage process of the stripe block.
Optionally, in a specific implementation manner, the total number of the data blocks is an integer multiple of the target number; the memory amount of the first memory is: the product of the amount of data per data block and the target number.
An embodiment of the present invention further provides an electronic device, as shown in fig. 5, which includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, where the processor 501, the communication interface 502 and the memory 503 complete mutual communication through the communication bus 504,
a memory 503 for storing a computer program;
the processor 501 is configured to implement the erasure code-based data storage method provided in the embodiment of the present invention when executing the program stored in the memory 503.
Specifically, the erasure code-based data storage method includes:
writing data to be stored into a first memory which is applied in advance; the target number of the data blocks which can be accommodated by the first memory is smaller than the total number of the data blocks specified by the preset EC type;
when the data to be stored in the first memory reaches the data volume of the data blocks with the target number, calculating a first check value corresponding to the data to be stored in the first memory based on a generating matrix matched with a preset EC type;
covering and storing the calculated exclusive or operation result of the first check value and the current content in the second memory into the second memory, transferring the data to be stored in the first memory into preset storage equipment, and returning to the step of writing the data to be stored into the first memory space;
and when the data to be stored based on the current content in the second memory reaches the data storage amount corresponding to the preset EC type, taking the current content in the second memory as a check value of the stripe block specified by the preset EC type and transferring the check value to the storage device to finish the storage of one stripe block.
It should be noted that other implementation manners of the erasure code based data storage method implemented by the processor 501 executing the program stored in the memory 503 are the same as the erasure code based data storage method provided in the foregoing method embodiment, and are not described herein again.
As can be seen from the above, in the scheme provided in the embodiment of the present invention, the target number of the data blocks that can be accommodated by the first memory for writing the data to be stored is less than the total number of the data blocks that is specified by the preset EC type, and when the check value of the stripe block that is specified by the EC type is used, the total number of the data blocks that is specified by the EC type may be calculated in segments according to the target data of the data blocks that can be accommodated by the first memory, and the check value of the stripe block is obtained through an xor operation. This reduces the memory space occupied during storage. Obviously, when the data storage amount corresponding to the EC type is large or multiple paths of data to be stored are processed simultaneously, the memory space occupied in the storage process can be greatly reduced, so that the utilization rate of a Central Processing Unit (CPU) can be improved. In addition, when the obtained data to be stored reaches the data storage amount corresponding to the preset EC type, the data to be stored does not need to be stored in the storage device, but the data to be stored can be stored in the storage device when less data to be stored are obtained, so that the probability of losing or making mistakes of the obtained data to be stored can be reduced, and the reliability of the data to be stored in the storage device is improved.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
The embodiment of the present invention further provides a computer-readable storage medium, which is a storage medium in a server, and a computer program is stored in the storage medium, and when the computer program is executed by a processor, the data storage method based on erasure codes provided by the embodiment of the present invention is implemented.
Specifically, the erasure code-based data storage method includes:
writing data to be stored into a first memory which is applied in advance; the target number of the data blocks which can be accommodated by the first memory is smaller than the total number of the data blocks specified by the preset EC type;
when the data to be stored in the first memory reaches the data volume of the data blocks with the target number, calculating a first check value corresponding to the data to be stored in the first memory based on a generating matrix matched with a preset EC type;
covering and storing the calculated exclusive or operation result of the first check value and the current content in the second memory into the second memory, transferring the data to be stored in the first memory into preset storage equipment, and returning to the step of writing the data to be stored into the first memory space;
and when the data to be stored based on the current content in the second memory reaches the data storage amount corresponding to the preset EC type, taking the current content in the second memory as a check value of the stripe block specified by the preset EC type and transferring the check value to the storage device to finish the storage of one stripe block.
It should be noted that other implementation manners of the erasure code-based data storage method implemented when the computer program is executed by the processor are the same as the erasure code-based data storage method provided in the foregoing method embodiment, and are not described herein again.
As can be seen from the above, in the scheme provided in the embodiment of the present invention, the target number of the data blocks that can be accommodated by the first memory for writing the data to be stored is less than the total number of the data blocks that is specified by the preset EC type, and when the check value of the stripe block that is specified by the EC type is used, the total number of the data blocks that is specified by the EC type may be calculated in segments according to the target data of the data blocks that can be accommodated by the first memory, and the check value of the stripe block is obtained through an xor operation. This reduces the memory space occupied during storage. Obviously, when the data storage amount corresponding to the EC type is large or multiple paths of data to be stored are processed simultaneously, the memory space occupied in the storage process can be greatly reduced, so that the utilization rate of a Central Processing Unit (CPU) can be improved. In addition, when the obtained data to be stored reaches the data storage amount corresponding to the preset EC type, the data to be stored does not need to be stored in the storage device, but the data to be stored can be stored in the storage device when less data to be stored are obtained, so that the probability of losing or making mistakes of the obtained data to be stored can be reduced, and the reliability of the data to be stored in the storage device is improved.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, the electronic device embodiment and the computer-readable storage medium embodiment, since they are substantially similar to the method embodiment, the description is relatively simple, and the relevant points can be referred to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (12)
1. An erasure code based data storage method, the method comprising:
writing data to be stored into a first memory which is applied in advance; the target number of the data blocks which can be accommodated by the first memory is smaller than the total number of the data blocks specified by a preset EC type;
when the data to be stored in the first memory reaches the data volume of the data blocks with the target number, calculating a first check value corresponding to the data to be stored in the first memory based on the generator matrix matched with the preset EC type;
covering and storing the calculated exclusive or operation result of the first check value and the current content in a second memory into the second memory, transferring the data to be stored in the first memory into a preset storage device, and returning to the step of writing the data to be stored into the first memory space which is applied in advance;
and when the data to be stored based on the current content in the second memory reaches the data storage amount corresponding to the preset EC type, taking the current content in the second memory as a check value of the stripe block specified by the preset EC type and transferring the check value to the storage device to finish the storage of one stripe block.
2. The method according to claim 1, wherein the step of calculating the first check value corresponding to the data to be stored in the first memory based on the generator matrix matched with the preset EC type includes:
according to the block size of the data block, dividing the data block to be stored in the first memory to obtain at least one data block;
and calculating a first check value corresponding to the at least one data block based on the generator matrix matched with the preset EC type.
3. The method according to claim 2, wherein the step of calculating the first check value corresponding to the at least one data block based on the generator matrix matched with the preset EC type includes:
calculating a first check value corresponding to the at least one data block through a predetermined calculation formula based on the generator matrix matched with the preset EC type;
wherein the predetermined calculation formula is:
wherein the content of the first and second substances,
a first memory for storing data to be storedmM is more than or equal to 0 and less than or equal to M-1, M is the number of the check blocks appointed by the EC type,
is an exclusive or operation;
4. The method according to any one of claims 1-3, further comprising:
when the data to be stored in the first memory does not reach the data volume of the data blocks with the target quantity, but the sum of the data to be stored based on the current content in the second memory and the data volume of the data to be stored in the first memory reaches the data storage volume, calculating a first check value corresponding to the data to be stored in the first memory based on a generating matrix matched with the preset EC type;
the XOR operation result of the first check value obtained by calculation and the current content in the second memory is stored in the second memory in a covering mode;
and transferring the data to be stored in the first memory to a preset storage device, and transferring the current content in the second memory to the storage device as a check value of the stripe block specified by the preset EC type, so as to complete the storage process of one stripe block.
5. The method of any of claims 1-3, wherein the total number of data blocks is an integer multiple of the target number;
the memory amount of the first memory is: the product of the amount of data per data block and the target number.
6. An erasure code-based data storage device, the device comprising:
the data writing module is used for writing data to be stored into a first memory which is applied in advance; the target number of the data blocks which can be accommodated by the first memory is smaller than the total number of the data blocks specified by a preset EC type;
the first calculation module is used for calculating a first check value corresponding to the data to be stored in the first memory based on the generator matrix matched with the preset EC type when the data to be stored in the first memory reaches the data volume of the data blocks with the target number;
the calculation result processing module is used for storing the calculated exclusive or operation result of the first check value and the current content in the second memory into the second memory in a covering manner, transferring the data to be stored in the first memory into a preset storage device, and returning the data to the data writing module;
and the first data storage module is used for taking the current content in the second memory as a check value of the stripe block specified by the preset EC type and transferring the check value to the storage device to finish the storage of one stripe block when the data to be stored based on the current content in the second memory reaches the data storage amount corresponding to the preset EC type.
7. The apparatus of claim 6, wherein the first computing module comprises:
the data dividing submodule is used for dividing the data to be stored in the first memory into data blocks according to the block size of the data blocks to obtain at least one data block;
and the check value operator module is used for calculating a first check value corresponding to the at least one data block based on the generator matrix matched with the preset EC type.
8. The apparatus of claim 7, wherein the check value operator module is specifically configured to:
calculating a first check value corresponding to the at least one data block through a predetermined calculation formula based on the generator matrix matched with the preset EC type;
wherein the predetermined calculation formula is:
wherein the content of the first and second substances,
a first memory for storing data to be storedmM is more than or equal to 0 and less than or equal to M-1, M is the number of the check blocks appointed by the EC type,
is an exclusive or operation;
9. The apparatus according to any one of claims 6-8, further comprising:
a second calculation module, configured to calculate, when the data to be stored in the first memory does not reach the data amount of the data blocks of the target number, but a sum of the data to be stored based on current content in the second memory and the data amount of the data to be stored in the first memory reaches the data storage amount, a first check value corresponding to the data to be stored in the first memory based on a generation matrix matched with the preset EC type;
the XOR operation module is used for covering and storing the XOR operation result of the calculated first check value and the current content in the second memory into the second memory;
and the second data storage module is used for transferring the data to be stored in the first memory to a preset storage device, and transferring the current content in the second memory to the storage device as a check value of the stripe block specified by the preset EC type, so as to complete the storage process of one stripe block.
10. The apparatus according to any of claims 6-8, wherein the total number of data blocks is an integer multiple of the target number;
the memory amount of the first memory is: the product of the amount of data per data block and the target number.
11. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1 to 5 when executing a program stored in the memory.
12. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of the claims 1-5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811012704.5A CN110874283B (en) | 2018-08-31 | 2018-08-31 | Data storage method and device based on erasure codes and electronic equipment |
| PCT/CN2019/103823 WO2020043203A1 (en) | 2018-08-31 | 2019-08-30 | Erasure code-based data storage method, device, and electronic apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811012704.5A CN110874283B (en) | 2018-08-31 | 2018-08-31 | Data storage method and device based on erasure codes and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110874283A CN110874283A (en) | 2020-03-10 |
| CN110874283B true CN110874283B (en) | 2021-07-20 |
Family
ID=69642863
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811012704.5A Active CN110874283B (en) | 2018-08-31 | 2018-08-31 | Data storage method and device based on erasure codes and electronic equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110874283B (en) |
| WO (1) | WO2020043203A1 (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2004029856A1 (en) * | 2002-09-23 | 2004-04-08 | Siemens Aktiengesellschaft | Method for securely transmitting data, particularly for transmission via an air interface |
| CN101178693A (en) * | 2007-12-14 | 2008-05-14 | 沈阳东软软件股份有限公司 | Data cache method and system |
| CN107094186A (en) * | 2017-06-15 | 2017-08-25 | 深圳市云舒网络技术有限公司 | A kind of correcting and eleting codes optimization method of distributed memory system |
| CN107544863A (en) * | 2017-06-26 | 2018-01-05 | 新华三技术有限公司 | A kind of date storage method and device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101840377A (en) * | 2010-05-13 | 2010-09-22 | 上海交通大学 | Data storage method based on RS (Reed-Solomon) erasure codes |
| CN105338035B (en) * | 2014-08-07 | 2018-12-28 | 苏宁易购集团股份有限公司 | Method for secure storing and device based on RS correcting and eleting codes |
| CN105335150B (en) * | 2014-08-13 | 2019-03-19 | 苏宁易购集团股份有限公司 | The quick decoding method and system of correcting and eleting codes data |
| CN107872489B (en) * | 2016-09-28 | 2020-06-05 | 杭州海康威视数字技术股份有限公司 | File slice uploading method and device and cloud storage system |
| CN107656832B (en) * | 2017-09-18 | 2019-10-25 | 华中科技大学 | A kind of correcting and eleting codes method of low data reconstruction expense |
-
2018
- 2018-08-31 CN CN201811012704.5A patent/CN110874283B/en active Active
-
2019
- 2019-08-30 WO PCT/CN2019/103823 patent/WO2020043203A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2004029856A1 (en) * | 2002-09-23 | 2004-04-08 | Siemens Aktiengesellschaft | Method for securely transmitting data, particularly for transmission via an air interface |
| CN101178693A (en) * | 2007-12-14 | 2008-05-14 | 沈阳东软软件股份有限公司 | Data cache method and system |
| CN107094186A (en) * | 2017-06-15 | 2017-08-25 | 深圳市云舒网络技术有限公司 | A kind of correcting and eleting codes optimization method of distributed memory system |
| CN107544863A (en) * | 2017-06-26 | 2018-01-05 | 新华三技术有限公司 | A kind of date storage method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020043203A1 (en) | 2020-03-05 |
| CN110874283A (en) | 2020-03-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11681554B2 (en) | Logical address distribution in multicore memory system | |
| US10810060B2 (en) | Event management for embedded systems | |
| TW201705002A (en) | Data storage device and operating method thereof | |
| CN107135006A (en) | Error-Correcting Circuit and error correcting method | |
| US20130179647A1 (en) | Storage device and data management method thereof | |
| KR20200043676A (en) | Memory system and operating method thereof | |
| US10162750B2 (en) | System address reconstruction | |
| US20160334999A1 (en) | Reduction of maximum latency using dynamic self-tuning for redundant array of independent disks | |
| CN107544863B (en) | Data storage method and device | |
| KR20210146513A (en) | Artificial intelligence calculation semiconductor device and storage device comprising the same | |
| CN109725850A (en) | Storage system and storage equipment | |
| CN107710170A (en) | Reduce the write-in amplification in object storage | |
| KR20170114438A (en) | Data storage device and operating method thereof | |
| US9594629B2 (en) | Data error correction from cached error correction information | |
| CN110874283B (en) | Data storage method and device based on erasure codes and electronic equipment | |
| CN111143146B (en) | Method and system for predicting health state of storage device | |
| CN112507323A (en) | Model training method and device based on unidirectional network and computing equipment | |
| US9478289B1 (en) | Semiconductor memory device and operating method thereof | |
| KR20210112557A (en) | Apparatus and method for accurate prediction of power required to operation of sequential commands performed in memory device | |
| CN108665940B (en) | ECC encoding circuit, decoding circuit and memory controller | |
| CN112115001A (en) | Data backup method and device, computer storage medium and electronic equipment | |
| CN113778346B (en) | Data reading method, device, equipment and storage medium | |
| US11928077B2 (en) | Data processing circuit, data storage device including the same, and operating method thereof | |
| CN110941567B (en) | Memory controller and operating method thereof | |
| US20230070503A1 (en) | Data protection method for memories and memory device thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |