CN117950916A - High-reliability data backup method and system - Google Patents
High-reliability data backup method and system Download PDFInfo
- Publication number
- CN117950916A CN117950916A CN202410346077.8A CN202410346077A CN117950916A CN 117950916 A CN117950916 A CN 117950916A CN 202410346077 A CN202410346077 A CN 202410346077A CN 117950916 A CN117950916 A CN 117950916A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- groups
- stored
- check
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 238000013500 data storage Methods 0.000 claims abstract description 26
- 238000011084 recovery Methods 0.000 claims abstract description 21
- 230000008569 process Effects 0.000 claims abstract description 18
- 239000011159 matrix material Substances 0.000 claims description 45
- 238000012937 correction Methods 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims description 3
- 239000013598 vector Substances 0.000 description 15
- 238000010586 diagram Methods 0.000 description 7
- 230000007246 mechanism Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000013179 statistical model Methods 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000017105 transposition Effects 0.000 description 1
Landscapes
- Techniques For Improving Reliability Of Storages (AREA)
Abstract
The invention provides a high-reliability data backup method and a system, wherein the high-reliability data backup method comprises the following steps: acquiring data information and data content of data to be stored, wherein the data content comprises a plurality of groups of information data; predicting the temperature type of the data to be stored according to the data information; setting coding parameters of data to be stored according to the temperature type; encoding the plurality of groups of information data according to the encoding parameters to obtain a plurality of groups of encoded data; storing the plurality of sets of encoded data in a plurality of storage nodes; reading a plurality of groups of coded data stored in a plurality of storage nodes; if information data is lost in the process of reading the coded data, carrying out data recovery by utilizing constraint relations among a plurality of groups of coded data so as to obtain a plurality of groups of information data; and outputting a plurality of groups of information data in the read plurality of groups of coded data. The invention can adjust the length of the check data according to the data temperature, improves the reliability of data storage, has higher efficiency of a storage system, and is very suitable for practical application.
Description
Technical Field
The invention relates to the technical field of data storage, in particular to a high-reliability data backup method and system.
Background
With the rapid development of information technology, the data storage volume is increased explosively, so that higher requirements are put on the performance of the data backup system. Data stored in the data storage system has different access frequencies in a life cycle, data with a high access frequency is called hot data, and data with a low access frequency is called cold data. The data stored in the data backup system has different data temperatures, and data with high temperatures is often of high importance. During the data lifecycle, the data temperature may change. In order to improve the system performance, it is necessary to effectively identify the data temperature and design a corresponding storage method according to the data temperature.
For data backup systems, the data storage nodes are inevitably affected by factors to fail, resulting in loss of stored data, and reduced system reliability. How to design an effective method to repair the data of the failed storage node is very important to improve the reliability of the data system. Fault tolerance techniques are an important means of improving the data reliability of data backup systems. In the fault-tolerant technology commonly used at present, erasure codes can achieve better compromise between storage reliability and storage efficiency. The extremely large distance separable (Maximum Distance Separable, MDS) code is the most widely used type of erasure codes, has optimal recovery capability, and can be flexibly configured.
In view of the foregoing, in order to improve the performance of the data backup system, it is needed to provide a data backup method with high storage efficiency and high reliability.
Disclosure of Invention
The invention aims to provide a high-reliability data backup method and a system, which at least solve the problem that the data backup method in the prior art does not store correspondingly according to the data temperature, so that the storage performance is poor.
In order to achieve the above object, according to one aspect of the present invention, there is provided a high reliability data backup method comprising: acquiring data information and data content of data to be stored, wherein the data content comprises a plurality of groups of information data; predicting the temperature type of the data to be stored according to the data information; setting coding parameters of data to be stored according to the temperature type; encoding the plurality of groups of information data according to the encoding parameters to obtain a plurality of groups of encoded data; storing the plurality of sets of encoded data in a plurality of storage nodes; reading a plurality of groups of coded data stored in a plurality of storage nodes; if information data is lost in the process of reading the coded data, carrying out data recovery by utilizing constraint relations among a plurality of groups of coded data so as to obtain a plurality of groups of information data; and outputting a plurality of groups of information data in the read plurality of groups of coded data.
Further, the data information includes address information and access information of the data to be stored; each group of information data comprises a plurality of blocks, and the length of each block of information data is a fixed value.
Further, the temperature type is predicted according to the access information, and the temperature type comprises a plurality of types which are preset.
Further, encoding the plurality of sets of information data according to the encoding parameters to obtain the plurality of sets of encoded data includes: constructing a generating matrix according to the coding parameters; calculating to obtain a plurality of groups of check data according to the generation matrix and the plurality of groups of information data; and combining the plurality of groups of check data and the plurality of groups of information data into a plurality of groups of encoded data.
Further, the encoding parameters include an information length of the data to be stored and a check length, the information length is the number of blocks included in the plurality of groups of information data, and the check length is the number of blocks included in the plurality of groups of check data.
Further, the information lengths of the data to be stored in different temperature types are the same, and the higher the temperature value of the temperature type is, the larger the verification length of the corresponding data to be stored is.
Further, constructing the generator matrix from the encoding parameters includes: pre-constructing a matrix G, wherein each element in the matrix G is an element in a finite field GF (256), and each element is represented by a value in 0,1,2, …, 255; and constructing a generating matrix corresponding to the temperature type according to the coding parameters, wherein the generating matrices of different temperature types are all submatrices of a matrix G, the dimension of the generating matrix is N multiplied by K, wherein K is the information length of the data to be stored corresponding to the temperature type, and N is the sum of the information length and the check length of the data to be stored corresponding to the temperature type.
Further, storing the plurality of sets of encoded data in the plurality of storage nodes includes: storing each group of coded data into a data block in each storage node, wherein the data block comprises check data and information data, the information data is stored in the information node, and the check data is stored in the check node; the storage node comprises an information node and a check node.
Further, when the number of the lost data blocks is not more than the number of the blocks of the check data in the case of reading the plurality of sets of encoded data stored in the plurality of storage nodes, the original information data is restored by error correction decoding.
According to another aspect of the present invention, there is provided a high reliability data backup system, which is applied to a high reliability data backup method, the high reliability data backup system comprising: the system comprises a data acquisition module, a temperature type prediction module, a data coding module, a data storage module, a data reading module, a data recovery module and a data output module; the data acquisition module is used for acquiring data information and data content of the data to be stored; the temperature type prediction module is used for predicting the temperature type of the data to be stored according to the data information; the data coding module is used for constructing a generating matrix according to the coding parameters, calculating a plurality of groups of check data according to the generating matrix and a plurality of groups of information data, and combining the plurality of groups of check data and the plurality of groups of information data into a plurality of groups of coded data; the data storage module is used for storing a plurality of groups of coded data in a plurality of storage nodes; the data reading module is used for reading a plurality of groups of coded data stored in the storage nodes; the data recovery module is used for judging whether information data is lost in the data reading process, and if the information data is lost, carrying out data recovery by utilizing the constraint relation among a plurality of groups of coded data so as to obtain a plurality of groups of information data; the data output module is used for outputting the read multiple groups of information data.
The technical scheme of the invention provides a high-reliability data backup method and a system, wherein the high-reliability data backup method comprises the following steps: acquiring data information and data content of data to be stored, wherein the data content comprises a plurality of groups of information data; predicting the temperature type of the data to be stored according to the data information; setting coding parameters of data to be stored according to the temperature type; encoding the plurality of groups of information data according to the encoding parameters to obtain a plurality of groups of encoded data; storing the plurality of sets of encoded data in a plurality of storage nodes; reading a plurality of groups of coded data stored in a plurality of storage nodes; if information data is lost in the process of reading the coded data, carrying out data recovery by utilizing constraint relations among a plurality of groups of coded data so as to obtain a plurality of groups of information data; and outputting a plurality of groups of information data in the read plurality of groups of coded data. The data backup method provided by the invention can adjust the length of the check data according to the data temperature, improves the reliability of data storage, has higher efficiency of a storage system, and is very suitable for practical application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:
FIG. 1 is a data schematic diagram of different data temperatures stored by an alternative storage system according to an embodiment of the present invention;
FIG. 2 is a flow chart of a data backup method of an alternative data backup system in accordance with an embodiment of the present invention;
FIG. 3 is a flow chart of a data storage method of an alternative data backup system in accordance with an embodiment of the present invention;
FIG. 4 is a schematic diagram of an alternative pre-constructed matrix G for encoding according to an embodiment of the invention;
FIG. 5 is a schematic diagram of an alternative data backup system data storage process in accordance with an embodiment of the present invention;
FIG. 6 is a schematic diagram of an alternative data backup system data reading process in accordance with an embodiment of the present invention;
FIG. 7 is a flowchart of an alternative data backup system data recovery method in accordance with an embodiment of the present invention;
FIG. 8 is a schematic diagram of an alternate matrix G generated with dimensions of a 6×4 submatrix G 3 in accordance with an embodiment of the present invention;
FIG. 9 is a schematic diagram of an alternative matrix G K according to an embodiment of the invention;
FIG. 10 is a block diagram of an alternative high reliability data backup system in accordance with an embodiment of the present invention.
Detailed Description
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.
As shown in fig. 1, the data stored by the data storage system changes in temperature during the life cycle. The importance of the data for different temperatures is different, and the importance of data with high temperatures is generally high. Therefore, it is necessary to efficiently recognize the data temperature and to take the corresponding storage method and storage medium according to the data temperature in order to optimize the overall performance of the storage system.
As shown in fig. 2, the present invention provides a high reliability data backup method, which includes: acquiring data information and data content of data to be stored, wherein the data content comprises a plurality of groups of information data; predicting the temperature type of the data to be stored according to the data information; setting coding parameters of data to be stored according to the temperature type; encoding the plurality of groups of information data according to the encoding parameters to obtain a plurality of groups of encoded data; storing the plurality of sets of encoded data in a plurality of storage nodes; reading a plurality of groups of coded data stored in a plurality of storage nodes; if information data is lost in the process of reading the coded data, carrying out data recovery by utilizing constraint relations among a plurality of groups of coded data so as to obtain a plurality of groups of information data; and outputting a plurality of groups of information data in the read plurality of groups of coded data.
The high-reliability data backup method of the invention can be divided into three parts for detailed description, and specifically comprises a data storage method of a data backup system, a data reading method of the data backup system and a data recovery method of the data backup system. The data storage method of the data backup system in the method of the invention is shown in fig. 3, and the data storage process comprises the following steps:
s201: acquiring data information and data content of data to be stored;
S202: predicting the temperature type of the data;
s203: determining coding parameters of the data;
S204: constructing a generation matrix for encoding;
s205: calculating the check data to form encoded data;
s206: storing the encoded data.
Each step in the data storage process is described in detail below.
S201: acquiring data information and data content of data to be stored
The data information of the data to be stored is acquired and comprises address information of the data and access information of the data. The address information of the data indicates a storage location of the data in the storage system. The access information of the data refers to the access information of the data in a set time, and includes, but is not limited to, information such as the number of times of accessing the data and the interval between the last access time and the data storage time. The set time is a predefined length of time including, but not limited to, months, quarters, or a combination thereof. The access information of the data can be obtained by counting the accesses to the data address.
The data content of the acquired data to be stored is the divided data to be stored. Assuming that each set of information data contains K blocks, the length of each block of information data is a fixed value. Data partitioning is well known in the data storage arts and will not be described in detail. For ease of implementation, data storage and reading is typically in bytes (8 bits), i.e., each block of information data is 1 byte in length. At this time, the 8-bit binary vector of each block of information data corresponds to one element in the finite field GF (256). GF (256) is a spread field of the binary field GF (2), the elements of which can be represented by 0,1,2, …, 255. GF (256) may be obtained by an eight degree primitive polynomial construction over GF (2), the specific construction method being well known in the art and will not be described in detail.
S202: predicting temperature type of data
The temperature type of the data is predicted according to the access information of the data, and the prediction method can be realized by adopting a statistical model established based on a cache elimination mechanism (such as an LRU mechanism and an LFU mechanism). The temperature types of the data are assumed to be of the L type: t 1, T2, …, TL, the result of the data temperature type prediction is one of T 1, T2, …, TL. Wherein L is a positive integer. Statistical models based on cache elimination mechanisms are well known in the art and will not be described in detail.
The data temperature prediction can also be realized by a machine learning method by utilizing access information of the data. The machine learning adopts a neural network (Neural Network, NN) which consists of an input layer, a hidden layer and an output layer. The nonlinear operation combination level of the neural network (especially the deep neural network) is generally higher, and can be regarded as the simulation of the neural connection structure of the human brain, the hierarchical characteristics are obtained by carrying out layer-by-layer characteristic transformation on the original characteristics extracted by the output layer to a new characteristic space, and finally the output layer realizes the classification of information.
S203: determining coding parameters of data
The coding parameters of the data are determined according to the data temperature type prediction result. The coding parameters of the data comprise the information length and the check length of the data. The information length is the number of blocks of the information data, and the check length is the number of blocks of the check data. As described above, the information length of the data of all the temperature types is K.
For data with temperature type T i, the check length is R i. The higher the temperature value of the temperature type, the greater the corresponding check length. Assuming that the temperature value of the class L data satisfies T 1> T2 > …> TL, the length R 1> R2 > …> RL is checked.
S204: constructing a generator matrix for encoding
As shown in fig. 4, each element in the pre-constructed matrix G is an element in GF (256), each element is represented by a value in 0,1,2, …, 255, and R 1 x K coefficients P j,k(1≤j≤ R1, 1+.k+.k) are selected such that the erasure code generated by the matrix G satisfies the MDS property.
The generator matrix for each temperature type is a sub-matrix of matrix G, the dimension being determined by the encoding parameters. For data of temperature type T i, the dimension of the generator matrix G i is n×k, where n=k+r i. The generation matrix G i enables the defined erasure code to meet the MDS property, that is, when the number of data lost in the process of reading the encoded data does not exceed the number of blocks R i of the check data, the original R i information data, i=1, 2, …, L, can be recovered by an error correction method.
S205: calculating the check data to form the encoded data
The check data is calculated using the information data and the generator matrix. Assuming that the data temperature type is T i, calculating R i blocks of check data by using K blocks of information data to obtain N=K+R i blocks of encoded data. Information data d= [ D 1,D2,…,DK]T, parity data p= [ P 1,P2,…,PRi]T, encoded data c= [ D; P ] satisfy
Gi*D = C
Where, represents the matrix multiplication operation and T represents the matrix transposition.
The matrix operation can obtain that the calculation formula of the check data P is that
P1 = P1,1D1 + P1,2D2 + … + P1,KDK
P2 = P2,1D1 + P2,2D2 + … + P2,KDK
…
PRi = PRi,1D1 + PRi,2D2 + … + PRi,KDK
S206: storing the encoded data.
And storing a plurality of groups of coded data in a plurality of storage nodes, wherein the specific process of storing is shown in fig. 5, the coded data is stored in blocks, and each storage node stores one data block. For data of temperature type T i, n=k+r i storage nodes are required, where K information nodes store information data and R i check nodes store check data.
The method of the present invention further includes a data reading method of the data backup system, as shown in fig. 6, the data storage node may fail (marked x node in fig. 6) so as to cause the loss of the storage data. The data reading process reads the data block of each storage node to obtain a read data vector. For data of temperature type T i, the data vector length n=k+r i is read. If a certain node data block is not lost, the read result is the value of that data block, otherwise the read result is the lost value (. Is the read value of the label x node in fig. 6.
The method of the invention also comprises a data recovery method of the data backup system, as shown in fig. 7, and the data recovery process comprises the following steps:
S601: acquiring a read data vector;
S602: judging whether data loss exists or not;
S603: recovering the read data vector;
S604: and outputting the information data.
Each step in the data storage process is described in detail below.
S601: acquiring read data vectors
For data of temperature type T i, assume that the read data vector is X, its length is n=k+r i, each component is an element in GF (256) or is a missing value.
S602: judging whether there is data loss
And judging whether information data loss exists in the read vector. For data of temperature type T i, if a certain component of the first K components of the read vector is a missing value. If there is a data loss, go to step S603, otherwise, go to step S604.
S603: restoring read data vectors
For data of temperature type T i, the generator matrix is G i. Assuming that X K is a vector of length K consisting of non-missing values in X, G K is a submatrix of dimension K X K for matrix G i and X K. The data recovery process is carried out by X K and calculating the first K components, and the calculation method is that
D´ = (GK)-1 * XK
Wherein, (G K)-1 is the inverse of matrix G K. Matrix inversion over the finite field is known in the art and will not be described in detail, the first K components of X are then updated with the calculated D'.
S604: outputting information data
The first K components of the output vector X are the error correction recovered information data.
In order to more clearly illustrate the data storage, data reading, and data recovery processes of the data temperature-based data backup system set forth herein, the methods set forth herein are further described below in connection with examples.
For the data storage process, it is assumed that each set of information data contains k=4 blocks, and the length of each block of information data is 1 byte. The temperature types of the data are assumed to be of 3 types: t 1>T2>T3. For the data of the three temperature types, the check length is R 1=8,R2=4,R3 =2.
Step S201 acquires data information and data content of data to be stored. The data information is information for predicting the temperature of the data, and the data content is assumed to be d= [45,12,201,160].
Step S202 predicts the temperature type of the data. And predicting the temperature of the data by adopting an LRU mechanism according to the access information of the data to obtain the temperature type T 3.
Step S203 determines the encoding parameters of the data D, the information length k=4, and the check length R 3 =2.
Step S204 the generating matrix with temperature type T 3 is G 3, and as shown in FIG. 8, the matrix G generates a 6×4 submatrix G 3 with dimensions, i.e. coefficients
P1,1 = 8, P1,2 = 48, P1,3 = 224, P1,4 = 231
P2,1 = 6, P2,2 = 28, P2,3 = 120, P2,4 = 237
The structured erasure codes can be made to satisfy MDS properties.
Step S205 calculates that the check data constitutes encoded data. The calculation method of the verification data P is as follows
P1 = P1,1D1 + P1,2D2 + P1,3D3 + P1,4D4 = 47
P2 = P2,1D1 + P2,2D2 + P2,3D3 + P2,4D4 = 39
The data after the constituent encoding is c= [45,12,201,160,47,39] T.
Step S206 stores the encoded data. Storing the information data and the check data respectively, wherein four information nodes store data 45, 12, 201 and 160 respectively; the two check nodes store data 47, 39 respectively.
For the data reading process, assume that the 1 st information node and the 3 rd information node fail.
Step S601 obtains the read data vector x= [.
Step S602 determines that there is a data loss in the data vector acquired in step S601.
Step S603 restores the read data vector. Construct X K=[12,160,47,39]T. Accordingly, the matrix G K is constructed as shown in fig. 9.
The method for calculating the K components before the X K and the calculation is as follows
D´ = (GK)-1 * XK
=[45,12,201,160]T
The first K components of X are updated with D', yielding x= [45,12,201,160,47,39] T.
Step S604 outputs the read data [45,12,201,160] T.
As shown in fig. 10, the present invention further provides a high reliability data backup system, which is applied to a high reliability data backup method, the high reliability data backup system comprising: a data acquisition module 10, a temperature type prediction module 20, a data encoding module 30, a data storage module 40, a data reading module 50, a data recovery module 60, and a data output module 70; the data acquisition module 10 is used for acquiring data information and data content of data to be stored; the temperature type prediction module 20 is used for predicting the temperature type of the data to be stored according to the data information; the data encoding module 30 is configured to construct a generating matrix according to the encoding parameters, calculate a plurality of sets of check data according to the generating matrix and the plurality of sets of information data, and combine the plurality of sets of check data and the plurality of sets of information data into a plurality of sets of encoded data; the data storage module 40 is configured to store multiple sets of encoded data in multiple storage nodes; the data reading module 50 is configured to read a plurality of sets of encoded data stored in a plurality of storage nodes; the data recovery module 60 is configured to determine whether information data is lost in the data reading process, and if the information data is lost, perform data recovery by using constraint relationships between multiple groups of encoded data to obtain multiple groups of information data; the data output module 70 is used for outputting the read multiple sets of information data.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A high reliability data backup method, comprising:
acquiring data information and data content of data to be stored, wherein the data content comprises a plurality of groups of information data;
Predicting the temperature type of the data to be stored according to the data information;
setting coding parameters of the data to be stored according to the temperature type;
encoding a plurality of groups of information data according to the encoding parameters to obtain a plurality of groups of encoded data;
Storing a plurality of groups of the coded data in a plurality of storage nodes;
reading a plurality of groups of the coded data stored in a plurality of storage nodes;
If the information data is lost in the process of reading the coded data, carrying out data recovery by utilizing constraint relations among a plurality of groups of the coded data so as to obtain a plurality of groups of the information data;
and outputting a plurality of groups of information data in the read plurality of groups of coded data.
2. The high reliability data backup method according to claim 1, wherein the data information includes address information and access information of the data to be stored; each group of information data comprises a plurality of blocks, and the length of each block of information data is a fixed value.
3. The high reliability data backup method according to claim 2, wherein the temperature type is predicted based on the access information, and the temperature type includes a plurality of types set in advance.
4. The high reliability data backup method of claim 1 wherein said encoding a plurality of sets of said information data according to said encoding parameters to obtain a plurality of sets of encoded data comprises:
constructing a generating matrix according to the coding parameters;
calculating to obtain a plurality of groups of check data according to the generation matrix and a plurality of groups of information data;
And combining a plurality of groups of check data and a plurality of groups of information data into a plurality of groups of coded data.
5. The high reliability data backup method according to claim 4, wherein the encoding parameters include an information length of the data to be stored and a check length, the information length being a number of blocks included in a plurality of sets of the information data, and the check length being a number of blocks included in a plurality of sets of the check data.
6. The high reliability data backup method according to claim 5, wherein the information lengths of the data to be stored of the different temperature types are all the same, and the higher the temperature value of the temperature type, the greater the verification length of the corresponding data to be stored.
7. The high reliability data backup method of claim 6 wherein said constructing a generator matrix from said encoding parameters comprises:
pre-constructing a matrix G, each element of the matrix G being an element in a finite field GF (256), each element being represented by a value of 0,1,2, …, 255;
And constructing a generating matrix corresponding to the temperature type according to the coding parameters, wherein the generating matrices of different temperature types are submatrices of the matrix G, and the dimension of the generating matrix is NxK, wherein K is the information length of the data to be stored corresponding to the temperature type, and N is the sum of the information length and the check length of the data to be stored corresponding to the temperature type.
8. The high reliability data backup method of claim 4 wherein storing the plurality of sets of encoded data in a plurality of storage nodes comprises:
Storing each group of coded data into a data block at each storage node, wherein the data block comprises the check data and the information data, the information data is stored in an information node, and the check data is stored in a check node;
wherein the storage node comprises the information node and the check node.
9. The high reliability data backup method according to claim 8, wherein when the number of the lost data blocks is not more than the number of the blocks of the check data in the reading of the plurality of sets of the encoded data stored in the plurality of storage nodes, the original information data is restored by error correction decoding.
10. A high-reliability data backup system applied to the high-reliability data backup method as recited in any one of claims 1 to 9, characterized by comprising:
the data acquisition module (10) is used for acquiring data information and data content of data to be stored;
a temperature type prediction module (20), wherein the temperature type prediction module (20) is used for predicting the temperature type of the data to be stored according to the data information;
The data encoding module (30) is used for constructing a generating matrix according to the encoding parameters, calculating a plurality of groups of check data according to the generating matrix and a plurality of groups of information data, and combining a plurality of groups of check data and a plurality of groups of information data into a plurality of groups of encoding data;
A data storage module (40), the data storage module (40) being configured to store a plurality of sets of the encoded data in a plurality of storage nodes;
A data reading module (50), wherein the data reading module (50) is used for reading a plurality of groups of coded data stored in a plurality of storage nodes;
The data recovery module (60), the said data recovery module (60) is used for judging whether there is the said information data to lose in the data reading process, if the said information data is lost, utilize the constraint relation among the said code data of multiple groups to carry on the data recovery in order to get multiple groups of said information data;
and the data output module (70) is used for outputting the read multiple groups of information data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410346077.8A CN117950916A (en) | 2024-03-26 | 2024-03-26 | High-reliability data backup method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410346077.8A CN117950916A (en) | 2024-03-26 | 2024-03-26 | High-reliability data backup method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117950916A true CN117950916A (en) | 2024-04-30 |
Family
ID=90793090
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410346077.8A Pending CN117950916A (en) | 2024-03-26 | 2024-03-26 | High-reliability data backup method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117950916A (en) |
-
2024
- 2024-03-26 CN CN202410346077.8A patent/CN117950916A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11531593B2 (en) | Data encoding, decoding and recovering method for a distributed storage system | |
| US8928503B2 (en) | Data encoding methods, data decoding methods, data reconstruction methods, data encoding devices, data decoding devices, and data reconstruction devices | |
| JP3978195B2 (en) | Method and system for minimizing the length of a defect list in a storage device | |
| Shahabinejad et al. | An efficient binary locally repairable code for hadoop distributed file system | |
| US20090083590A1 (en) | System and method for determining the fault-tolerance of an erasure code | |
| CN113297000B (en) | RAID (redundant array of independent disks) coding circuit and coding method | |
| Wang et al. | MDR codes: A new class of RAID-6 codes with optimal rebuilding and encoding | |
| CN112000512B (en) | Data restoration method and related device | |
| CN105808170B (en) | A kind of RAID6 coding methods that can repair single disk error | |
| US20160285476A1 (en) | Method for encoding and decoding of data based on binary reed-solomon codes | |
| CN113297001B (en) | RAID (redundant array of independent disks) coding and decoding method and coding and decoding circuit | |
| Venkatesan et al. | Effect of codeword placement on the reliability of erasure coded data storage systems | |
| CN113296999A (en) | RAID6 encoding method and encoding circuit | |
| WO2020029418A1 (en) | Method for constructing repair binary code generator matrix and repair method | |
| JP2020046871A (en) | Memory system | |
| CN115454712B (en) | Check code recovery method, system, electronic equipment and storage medium | |
| CN210110352U (en) | ECC device for correcting multi-bit errors in NAND Flash | |
| Song et al. | A Low complexity design of reed solomon code algorithm for advanced RAID system | |
| CN117950916A (en) | High-reliability data backup method and system | |
| WO2020029423A1 (en) | Construction method and repair method for repairing binary array code check matrix | |
| CN106788454B (en) | Construction method of local unequal codes | |
| CN106911793B (en) | I/O optimized distributed storage data repair method | |
| CN112000509B (en) | Erasure code encoding method, system and device based on vector instruction | |
| CN114138543A (en) | Data strip coding method, system, device and medium | |
| KR101923116B1 (en) | Apparatus for Encoding and Decoding in Distributed Storage System using Locally Repairable Codes and Method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |