CN110502365A - The method, apparatus and computer equipment they data storage and restored - Google Patents
The method, apparatus and computer equipment they data storage and restored Download PDFInfo
- Publication number
- CN110502365A CN110502365A CN201910624964.6A CN201910624964A CN110502365A CN 110502365 A CN110502365 A CN 110502365A CN 201910624964 A CN201910624964 A CN 201910624964A CN 110502365 A CN110502365 A CN 110502365A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- slice
- data slice
- check sheet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000013500 data storage Methods 0.000 title claims abstract description 22
- 238000011084 recovery Methods 0.000 claims abstract description 48
- 238000003860 storage Methods 0.000 claims abstract description 39
- 230000002159 abnormal effect Effects 0.000 claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 8
- 230000001174 ascending effect Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 7
- 238000012360 testing method Methods 0.000 description 6
- 238000012795 verification Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 150000001875 compounds Chemical class 0.000 description 3
- 230000008439 repair process Effects 0.000 description 3
- 241001269238 Data Species 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 239000003795 chemical substances by application Substances 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 210000004556 brain Anatomy 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Storage Device Security (AREA)
Abstract
This application discloses the method, apparatus and computer equipment a kind of data storage and restore, are related to data processing field, can solve when restoring abnormal data file, restore the problem that time-consuming, at high cost.Wherein method includes: the multiple data blocks for obtaining and being evenly dividing by raw data file；Data block coding is processed into multiple data slices and check sheet based on correcting and eleting codes；The raw data file is stored using the data slice and the check sheet；If it is determined that the raw data file missing, then restore the raw data file using the data slice for meeting preset condition and check sheet decoding.The application is suitable for piecemeal storage and the automatic recovery of missing data to data file.
Description
Technical field
This application involves data processing field more particularly to a kind of storage of data and the method, apparatus and computer restored
Equipment.
Background technique
As computer is in the universal of various industries, mass data file is required by computer server
Reason, storage, will be directly concerning to enterprises if occurring abnormal in storing process or losing for the higher data files of some values
Industry or individual economic benefit, serious person even can concerning arrive social benefit.Processor as server brain is in inside and outside
It under the influence of environmental factor, inevitably will appear exception, in the case where processor exception, how to go to ensure the data in server
File does not occur abnormal or does not lose, that will be most important problem.
At present in the industry mainly by the data file in dynamic monitoring memory space, when judge data file storage exception
When, copy data file is uploaded again, and then replaces the data file of missing.
However the mode that above-mentioned data are restored needs to copy taking copy data file and carrying out when determining that storage is abnormal
It passes, when transmitting data file amount is big, copy and transmission process can be taken a long time, and be easy to delay the operation process of business, and make
The cost restored at data is larger.
Summary of the invention
In view of this, this application discloses the method, apparatus and computer equipment a kind of data storage and restore, main mesh
Be solve when restoring abnormal data file, recovery the problem that time-consuming, at high cost.
According to the one aspect of the application, a kind of method data storage and restored is provided, this method comprises:
Obtain the multiple data blocks being evenly dividing by raw data file；
Data block coding is processed into multiple data slices and check sheet based on correcting and eleting codes；
The raw data file is stored using the data slice and the check sheet；
If it is determined that the raw data file missing, then utilize the data slice and the check sheet for meeting preset condition
Decoding restores the raw data file.
According to further aspect of the application, a kind of device data storage and restored is provided, which includes:
Module is obtained, for obtaining the multiple data blocks being evenly dividing by raw data file；
Processing module, for data block coding to be processed into multiple data slices and check sheet based on correcting and eleting codes；
Memory module, for storing the raw data file using the data slice and the check sheet；
Recovery module is used for if it is determined that raw data file missing, then utilize the data for meeting preset condition
Piece and check sheet decoding restore the raw data file.
According to the another aspect of the application, a kind of non-volatile readable storage medium is provided, calculating is stored thereon with
Machine program realizes above-mentioned method data storage and restored when described program is executed by processor.
According to another aspect of the application, a kind of computer equipment is provided, including non-volatile readable storage medium,
Processor and it is stored in the computer program that can be run on non-volatile readable storage medium and on a processor, the processor
Above-mentioned method data storage and restored is realized when executing described program.
By above-mentioned technical proposal, a kind of data storage provided by the present application and the method, apparatus and computer restored are set
Standby, compared with currently with the mode of copy data file replacement missing data file, the application can draw raw data file
It is divided into multiple data blocks, and coded treatment is carried out to data block based on correcting and eleting codes technology, acquires multiple data slices and verification
Data slice and check sheet are stored in different data centers by piece, obtain raw data file by sequentially reading data slice,
When determining that data slice in a certain data center and check sheet lack, using the normal data slice of remainder data and school
It tests piece and carries out automatic decoding reparation to it, and then realize the recovery to data file.The entire recovery process of this programme is unusual
Efficiently, the data block of missing can be detected in time and repair, to not delay business process.It is only needed when data file is restored
Missing data piece is restored, does not need to replace entire data file, therefore the cost of data recovery can be effectively reduced, and can keep away
Exempt from as data recovery time is long and caused by economic loss.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present application, constitutes part of this application, this Shen
Illustrative embodiments and their description please do not constitute the improper restriction to locally applying for explaining the application.In the accompanying drawings:
Fig. 1 shows the flow diagram of a kind of data storage provided by the embodiments of the present application and the method restored；
Fig. 2 shows another data storage provided by the embodiments of the present application and the flow diagrams for the method restored；
Fig. 3 shows the structural schematic diagram of a kind of data storage provided by the embodiments of the present application and the device restored；
Fig. 4 shows the structural schematic diagram of another data storage provided by the embodiments of the present application and the device restored.
Specific embodiment
The application is described in detail hereinafter with reference to attached drawing and in conjunction with the embodiments.It should be noted that in the feelings not conflicted
Under condition, the features in the embodiments and the embodiments of the present application can be combined with each other.
For the problem that time-consuming, at high cost when restoring missing data file, is restored at present, the embodiment of the present application is provided
A kind of method data storage and restored, as shown in Figure 1, this method comprises:
101, the multiple data blocks being evenly dividing by raw data file are obtained.
For the present embodiment, in specific application scenarios, in order to realize the piecemeal processing to raw data file, and
To the code storage of data block, therefore need that raw data file is uniformly cut into multiple equal-sized data blocks in advance, point
Cutting quantity can be determined according to practical application scene.For example, the raw data file of a 24MB, can draw according to the actual situation
It is divided into the data block of two 12MB, raw data file can be got by being successively read two data blocks.
102, data block coding is processed into multiple data slices and check sheet based on correcting and eleting codes.
For the present embodiment, in specific application scenarios, raw data file is being divided according to data arrangement sequence
After the identical original data block of N number of size, the identical data of N number of size are generated to initial data block coding using correcting and eleting codes
Piece and M verification data slices identical with data slice size, wherein the original that raw data file is divided based on correcting and eleting codes
Reason are as follows: the Erasure Code based on Cauchy matrix is applied primarily to the invertibity of Cauchy matrix, in the data of correcting and eleting codes storage
Pass the data for mainly restoring to lose using coding, decoding with recovery process.
103, raw data file is stored using data slice and check sheet.
It,, can will be original in order to realize the secure storage to data file in specific application scenarios for the present embodiment
Data block and verification data block are respectively stored in different positions, such as different disks, different memory nodes.Such subregion
The mode of storage can be avoided data and be stored in a position, cause total data to lack since single memory space damages
The problem of, and when single memory node breaks down or is wrong, it can clearly determine the data slice or verification of shortage of data
Piece, moreover it is possible to data recovery be carried out to missing file using data slice complete in other memory spaces and check sheet.
104, if it is determined that raw data file lacks, then restored using data slice and the check sheet decoding for meeting preset condition
Raw data file.
Wherein, preset condition is the Tolerate and redundance that the data slice of shortage of data and the quantity of check sheet should meet correcting and eleting codes
Degree, Tolerate and redundance degree can correspond to the quantity of check sheet, when the data slice of shortage of data and the quantity of check sheet are less than or equal to appearance
When wrong redundancy, the normal data slice of availability data and check sheet realize the recovery to missing data；When the number of shortage of data
When spending according to the quantity of piece and check sheet greater than Tolerate and redundance, illustrate that the normal data slice of data and check sheet cannot achieve to scarce
The decoding for losing data restores, that is, the data for having exceeded correcting and eleting codes restore condition.
For example, being k block source data (data block) by primary data information (pdi) cutting, then pass through correcting and eleting codes Encoder
It generates n block encoded data (data block of coding), wherein include (n-k) block check block in encoded data, finally unite
One storage transmission；As long as k` >=k block encoded data can be received, it will be able to calculate all source
data。
Raw data file, can be divided into multiple data blocks by middle method data storage and restored through this embodiment,
And coded treatment is carried out to data block based on correcting and eleting codes technology, multiple data slices and check sheet are acquired, by data slice and school
It tests piece and is stored in different data centers, obtain raw data file by sequentially reading data slice, when a certain data of judgement
When data slice and check sheet in center lack, it is carried out certainly using the normal data slice of remainder data and check sheet
Dynamic decoding is repaired, and then realizes the recovery to data file.The entire recovery process of this programme is very efficient, can examine in time
The data block of missing is surveyed and repairs, to not delay business process.Data file restore when only need to missing data piece into
Row restores, and does not need to replace entire data file, therefore the cost of data recovery can be effectively reduced, and is avoided that since data are restored
Time it is long and caused by economic loss.
Further, as the refinement and extension of above-described embodiment specific embodiment, in order to completely illustrate the present embodiment
In specific implementation process, the method for providing the storage of another data and restoring, as shown in Fig. 2, this method comprises:
201, the multiple data blocks being evenly dividing by raw data file are obtained.
For example, the raw data file of a 36MB, can be evenly dividing into the data block of three 12MB according to the actual situation,
And these three data blocks are numbered according to putting in order for raw data file, when reading raw data file, can press
According to the sequence read block that number is ascending, and then arrive raw data file.
202, data block is encoded using correcting and eleting codes, raw data file is divided into greatly according to the first division rule
Small identical first data slice and the first check sheet.
Wherein, the first division rule presets impartial quantity for raw data file coding to be divided into, and generation and data
Block equivalent amount and identical first data slice of size, and generate setting quantity and the first data slice size identical first
Check sheet.In specific application scenarios, the first division rule can set the division number of the first data slice and the first data block
Amount.
For example, obtaining the first division rule are as follows: raw data file coding is divided into 2 the first data slices and 1 the
One check sheet.If raw data file is 24MB, then the first data of 2 12MB are generated according to the first division rule codified
First check sheet of piece, 1 12MB.The first data slice by reading 2 12MB can get raw data file.First
The effect of check sheet is the recovery for losing data to the first data slice, the fault-tolerant energy of the quantity representative correcting and eleting codes of check sheet
Power, i.e. maximum redundancy degree.In the present embodiment, it can only at most allow one of them first data slice that loss of data, ability occurs
Guarantee that the data lost can be restored completely.
203, secondary coding is carried out to the first data slice using correcting and eleting codes, each first data slice is divided according to second and is advised
Then it is divided into identical second data slice of size and the second check sheet.
Wherein, the second division rule are as follows: secondary coding division is carried out to each first data slice after division, i.e., by first
Data slice coding is evenly dividing as the second data slice of predetermined quantity and the second check sheet.In specific application scenarios, second
Division rule can set the quantity that single first data slice is divided into the second data slice and the second check sheet.
For example, obtaining the second division rule are as follows: each first data slice coding is divided into 4 the second data slices and 2
Second check sheet.If raw data file is 24MB, the first data for generating 2 12MB are encoded according to the first division rule
First check sheet of piece, 1 12MB.Can then continue that the first check sheet of two 12MB is respectively divided according to the second division rule
For the second data slice of 4 3MB and the second check sheet of 2 same sizes, the second data slice by reading 4 12MB is
Corresponding first data sheet data can be got, raw data file can be obtained by sequentially reading 8 the second data slices, the
The effect of two check sheets is the recovery for losing data to the second data slice, the fault-tolerant energy of the quantity representative correcting and eleting codes of check sheet
Power, i.e. maximum redundancy degree.In the present embodiment, it can only at most allow the second data slice of two of them that loss of data, ability occurs
Guarantee that the data lost can be restored completely.
204, each first data slice is stored in the first different data centers, each first check sheet is stored in not
The second same data center.
For the present embodiment, in specific application scenarios, in order to avoid data are lost simultaneously, and realize to data file
Secure storage, the first data slice and the first check sheet can be respectively stored in different positions.For example, the text of a 24MB
Part carries out level-one division using the first division rule, is divided into the first data slice of 2 12MB and the first school of 1 12MB
Piece is tested, 2 the first data slices are deposited in into the first data center of the first data center of A and B respectively later, 1 first is verified
Piece deposits in the second data center of C.
205, it is single that the second data slice and the second check sheet are respectively corresponded to the different storages being stored in the first data center
Member.
Based on the embodiment of embodiment step 204, by the first verification of the first data slice of 2 12MB and 1 12MB
After piece deposits in the second data center of the first data center of A, the first data center of B and C respectively, for each first data center
The first data slice of 12MB of interior storage can carry out second level division based on the second division rule, be cut into the second number of 4 3MB respectively
According to the second check sheet of piece and 2 3MB, the second data slice is lost simultaneously in order to prevent, can be by the second data slice and the second school
Test the different storage locations that piece is respectively stored in the first data center of the first data center of corresponding A and B.
206a, raw data file is obtained by reading all first data slices.
For the present embodiment, in specific application scenarios, embodiment step 206a be can specifically include: obtain each the
First serial number of one data slice, wherein the corresponding first data slice assembly raw data file of the first serial number puts in order；According to
The first ascending serial number reads the first data slice, to obtain raw data file.
For example, after raw data file is evenly dividing into four first data slices a, b, c, d based on the first division rule,
When wanting to obtain raw data file, it can be read out by obtaining first serial number of this four the first data slices, such as be obtained
The first serial number for getting a, b, c, d this four the first data slices is respectively 4,2,3,1, then can have according to the first serial number small to big
Sequence reads the first data slice d, the first data slice b, the first data slice c, the first data slice a to get raw data file.
207a, if it is determined that exist missing the first data slice, it is determined that raw data file missing.
For the present embodiment, in specific application scenarios, all first data slices, which add up, represents initial data text
Part, when in the way of the first data slice of reading to obtain raw data file, if it is determined that there is the first data slice of missing
When, complete raw data file can not be got by reading the first data slice, that is, can determine that raw data file lacks.
208a, determine that there are the first data centers of target of the first data slice shortage of data.
Correspondingly, when determining there are when the first data slice missing, needing to obtain the first number of missing in advance for the present embodiment
According to the first data center of target where piece, as data center's point to be restored.
The second check sheet of the second data slice of all targets and target in 209a, extraction the first data center of target.
In specific application scenarios, the first data center of target there are the first data slice shortage of data is being determined
Afterwards, the first data slice lacked in the first data center of target can be restored, specifically may include two kinds of reset modes.In
The first reset mode to the first data slice is introduced in the present embodiment step first, i.e., using in the first data center of target
Second data slice and the second check sheet are restored, therefore need to extract all targets second in the first data center of target first
The second check sheet of data slice and target.And then by executing embodiment step 210a to embodiment step 211a, to realize to scarce
Lose the recovery of the first data slice.
210a, if it is determined that target the second data sheet data is complete, then read the second data slice of target according to the second serial number, with
Just the first data slice of target of shortage of data is obtained.
Correspondingly, using in the first data center of target the second data slice and the second check sheet come restore missing first number
When according to piece, and including two kinds of restoration scenarios, the present embodiment step provides wherein the first restoration scenario, that is, determines all targets
In the second complete situation of data sheet data, can by sequentially read all second data slices in the first data center of target come
Obtain the first data slice of target of shortage of data.
211a, if it is determined that the second data slice of target there are shortage of data and meet the second data restore condition, then utilize number
The second data slice of target for restoring shortage of data according to complete the second data slice of target and the decoding of the second check sheet of target, according to the
Two serial numbers read the second data slice of target after restoring, to obtain the first data slice of target.
Wherein, the second data restore the quantity for the second check sheet of the second data slice of target and target that condition is shortage of data
Less than or equal to the second preset threshold.The maximum redundancy degree that second preset threshold corresponding data restores represents the number of target second
According to the maximum fault-tolerant number of piece, the second preset threshold can correspond to the quantity of the second check sheet of target, such as the second check sheet of target
Quantity is 2, then can set the second preset threshold as 2, when the second check sheet of the second data slice of target and target of shortage of data
When quantity is less than or equal to 2, complete the second data slice of target of availability data and the decoding of the second check sheet of target restore data and lack
The second data slice of target of mistake can be by sequentially reading the first data of target after completing the data to the second data slice and restoring
All second data slices in center obtain the first data slice of target of shortage of data.
For the present embodiment, as with embodiment step 210a it is arranged side by side using the second data slice and the second check sheet come extensive
Another restoration scenario of the first data slice is lacked again, i.e., judgement the second data slice of target there are shortage of data and meets the second number
According to the condition of recovery, then complete the second data slice of target of availability data and the decoding of the second check sheet of target restore shortage of data
The second data slice of target, after all target the second data sheet datas restore complete, by sequentially reading in the first data of target
Intracardiac all second data slices obtain the first data slice of target of shortage of data.
212a, if it is determined that the second data slice of target there are shortage of data and do not meet the second data restore condition, then obtain
Complete first data slice of data and the first check sheet.
It is introduced in the present embodiment step corresponding with embodiment step 209a extensive to second for lacking the first data slice
Compound formula utilizes the first verification in complete first data slice of other first intra-data center datas and the second data center
Piece restores the first data slice of the first data center of target.Because obtaining the data in other data centers than obtaining same data
Time-consuming for data in center, therefore in order to improve the efficiency of data recovery, embodiment can be preferentially executed as a preferred method,
The recovery operation of step 209a, when determining that the second data slice in target data center has missing and decodes irrecoverable,
The data recovery operation of embodiment step 213a to embodiment step 214a can be executed in turn.
213a restores condition if it is determined that the first data slice of target meets the first data, then utilizes complete first data of data
Piece and the decoding of the first check sheet restore the first data slice of target.
Wherein, the first data restore the quantity for the first check sheet of the first data slice of target and target that condition is shortage of data
Less than or equal to the first preset threshold.The maximum redundancy degree that first preset threshold corresponding data restores, that is, represent the first data slice
Maximum fault-tolerant number, the first preset threshold can correspond to the quantity of the first check sheet, if the quantity of the second check sheet of target is 1,
The second preset threshold can then be set as 1, when the quantity of the second check sheet of the second data slice of target and target of shortage of data is less than
Or when being equal to 1, complete first data slice of availability data and the decoding of the first check sheet restore the first data slice of target.
214a restores condition if it is determined that the first data slice of target does not meet the first data, then output data restores abnormal report
Alert information.
For the present embodiment, if it is determined that the first data slice of target and the first check sheet of target of shortage of data are unsatisfactory for first
Data restore condition cannot be real by above two data reconstruction method that is, when restoring the first data slice of shortage of data
Now to the recovery of the first data slice, then alarm signal can be further exported, play the purpose of prompt, so that staff takes plan
Slightly timely repaired.
The embodiment step 206b arranged side by side, original to obtain by reading all second data slices with embodiment step 206a
Data file.
For the present embodiment, in specific application scenarios, embodiment step 206b be can specifically include: obtain each the
Second serial number of two data slices, wherein the corresponding second data slice assembly raw data file of the second serial number puts in order；According to
The second ascending serial number reads the second data slice, to obtain raw data file.
For example, after raw data file is evenly dividing into two the first data slices A, B based on the first division rule, then will
First data slice A is divided into four second data slices a1, a2, a3, a4 according to the second division rule coding.By the first data slice B
Four second data slices b1, b2, b3, b4 are divided into according to the second division rule coding.When wanting to obtain raw data file,
Can be read out according to second serial number of this 8 the second data slices, such as get a1, a2, a3, a4, b1, b2, b3, b4 this 8
First serial number of a first data slice, which is respectively 4,2,3,1,5,6,8,7, to be passed through according to the ascending sequence of the first serial number
Read the second data slice a4, the second data slice a2, the second data slice a3, the second data slice a1, the second data slice b1, the second data
Piece b2, the second data slice b4, the second data slice b3 get raw data file.
207b, if it is determined that exist missing the second data slice, it is determined that raw data file missing.
For the present embodiment, in specific application scenarios, all second data slices, which add up, represents initial data text
Part, when in the way of the second data slice of reading to obtain raw data file, if it is determined that there is the second data slice of missing
When, complete raw data file can not be got by reading the second data slice, that is, can determine that raw data file lacks.
208b, determine that there are the first data centers of target of the second data slice shortage of data.
Correspondingly, when determining there are when the second data slice missing, needing to obtain the second number of missing in advance for the present embodiment
According to the first data center of target where piece, as data center's point to be restored.
209b, complete the second data slice of target of the first intra-data center data of target and the second check sheet of target are obtained.
In specific application scenarios, the first data center of target there are the second data slice shortage of data is being determined
Afterwards, the second data slice lacked in the first data center of target can be restored, specifically may include two kinds of reset modes.In
The first way restored to the second data slice of missing is introduced in the present embodiment step first, i.e., in the first data center of sharp target
Complete the second data slice of target of data and the second check sheet of target are restored, therefore need to extract first in the first data of target
Complete the second data slice of target of intracardiac all data and the second check sheet of target.And then by executing embodiment step
210b, to realize the recovery to the second data slice of missing.
210b, if it is determined that the second data slice of target of shortage of data meet the second data restore condition, then using data it is complete
Whole the second data slice of target and the decoding of the second check sheet of target restore the second data slice of target of shortage of data.
Wherein, the second data restore the quantity for the second check sheet of the second data slice of target and target that condition is shortage of data
Less than or equal to the second preset threshold.The maximum redundancy degree that second preset threshold corresponding data restores represents the number of target second
According to the maximum fault-tolerant number of piece, the second preset threshold can correspond to the quantity of the second check sheet of target, such as the second check sheet of target
Quantity is 2, then can set the second preset threshold as 2, when the second check sheet of the second data slice of target and target of shortage of data
When quantity is less than or equal to 2, complete the second data slice of target of availability data and the decoding of the second check sheet of target restore data
The second data slice of target of missing.
211b, if it is determined that the second data slice of target do not meet the second data restore condition, then obtain in the first data of target
Intracardiac the first data slice of target.
It is introduced in the present embodiment step corresponding with embodiment step 209b extensive to second for lacking the second data slice
Compound formula realizes the recovery to the second data slice of missing using the first data slice of target in the first data center of target,
Therefore need to extract the first data slice of target in the first data center of target first.And then by executing embodiment step 212b
To embodiment step 213b, to realize the recovery to the second data slice of missing.
212b, if it is determined that target the first data sheet data is complete, then the first data slice of target is drawn again using error correcting code
It is divided into the second data slice and the second check sheet for meeting the second division rule, so as to the second data of target of replacement data missing
Piece.
Correspondingly, when restoring to lack the second data slice using the first data slice in the first data center of target, and wrap
Two kinds of restoration scenarios are included, the present embodiment step provides the first restoration scenario, i.e. judgement the first data of target sheet data is complete
In the case where, the first data slice of target can be repartitioned by error correcting code the second data slice for meeting the second division rule and
Second check sheet, and then the second check sheet of the second data slice of target or target of replacement data missing.
213b, if it is determined that target the first data slice shortage of data and meet the first data restore condition, then using data it is complete
The first whole data slice and the decoding of the first check sheet restore the first data slice of target of shortage of data.
Wherein, the first data restore the quantity for the first check sheet of the first data slice of target and target that condition is shortage of data
Less than or equal to the first preset threshold.The maximum redundancy degree that first preset threshold corresponding data restores, that is, represent the first data slice
Maximum fault-tolerant number, the first preset threshold can correspond to the quantity of the first check sheet, if the quantity of the second check sheet of target is 1, then
The second preset threshold can be set as 1, when the quantity of the second check sheet of the second data slice of target and target of shortage of data is less than etc.
When 1, complete first data slice of availability data and the decoding of the first check sheet restore the first data slice of target.It can pass through later
The first data slice of target after recovery is repartitioned into the second data slice and the second school for meeting the second division rule by error correcting code
Test piece, and then the second check sheet of the second data slice of target or target of replacement data missing.
214b, if it is determined that target the first data slice shortage of data and not meeting the first data and restoring condition, then output data
Restore abnormal warning message.
For the present embodiment, if it is determined that the first data slice of target of shortage of data, which is unsatisfactory for the first data, restores condition,
When restoring the second data slice of shortage of data, it can not achieve by above two data reconstruction method to the second data slice
Restore, then can further export alarm signal, play the purpose of prompt, so that staff takes strategy timely to be robbed
It repairs.
It, can also will be upper in order to provide the preferred embodiment of a kind of progress data storage and recovery in specific application scenarios
It states the mode that two kinds were read and restored raw data file to combine, be obtained by reading the first data slice and the second data slice
Raw data file；It, can be by reading first data center when determining the first data slice missing in the first data center
Interior whole complete second data slice obtains data；Equally, when determining the second data slice missing in the first data center,
Data can be obtained by reading the first data slice in first data center；After reading data success, then based on complete
The first data slice, the first check sheet and complete second data slice of data, the second check sheet realize to the extensive of missing data piece
It is multiple.
In the present embodiment, it is based primarily upon the invertibity of Cauchy matrix, loss is stored and restored by encoding, decoding
Data.Wherein, using the method for storing normal second data slice and the second check sheet decoding the first data slice of recovery, specifically may be used
It include: to constitute decoding column vector using normal second data slice of storage and the second check sheet；Will in generator matrix decoding column to
Corresponding row composition inverse of a matrix matrix is measured to be determined as decoding generator matrix；By calculating decoding generator matrix and decoding column vector
Product obtain the second data slice of shortage of data.Correspondingly, using normal first data slice and the first check sheet is stored
The method that decoding restores the first data slice and the first check sheet of shortage of data, specifically can include: normal first number will be stored
Decoding column vector is constituted according to piece and the first check sheet；Inverse of a matrix matrix is constituted by row corresponding to column vector is decoded in generator matrix
It is determined as decoding generator matrix；The first of shortage of data is obtained by calculating decoding generator matrix and decoding the product of column vector
Data slice.
Raw data file can tentatively be evenly dividing as multiple data by the method for storing and restoring by above-mentioned data
Block carries out coded treatment to data block based on correcting and eleting codes technology, acquires multiple first data slices and the first check sheet, later
The first data slice is encoded based on correcting and eleting codes technology again, each first data slice is again divided into multiple second data slices
And second check sheet, each first data slice is stored in the first different data centers, each first check sheet is stored in
The second different data centers；Second data slice and the second check sheet are respectively corresponded into the difference being stored in the first data center
Storage unit.Raw data file can be read by two ways, the first reading manner: by sequentially reading each first
The first data slice in data center obtains raw data file.Second of reading manner: by sequentially reading each first
Data slice corresponding second data slice obtains raw data file.Two kinds of reading manners correspond to two kinds of data reset modes again,
The first reset mode:, can be by owning in the first data center where reading when determining has the first data slice of missing
Complete second data slice to obtain former first data slice, and then realizes that the replacement to the first data slice of missing restores.In addition,
The second data slice where determining in the first data center equally exist shortage of data and it is irrecoverable when, can be complete by data
Whole the first data slice and the first check sheet is decoded recovery.Second of reset mode: when the second number for determining that there is missing
When according to piece, it can be decoded by complete second data slice of other data and the second check sheet in first data center extensive
It is multiple.When that can not realize recovery, then its corresponding first data slice is obtained, re-starts the second data slice using the first data slice
And second check sheet coding divide, thus for restore the missing the second data slice.The two kinds of data provided by this programme
Reading manner, and to two kinds of reset modes of missing data, it can be achieved that duplicate protection to business datum, when in reading data
When shortage of data occurring in the process, reading manner can be replaced in time, the integrality and safety of business datum is effectively ensured.
In addition, this programme specific to the storage and recovery to data slice, and provides the storage and recovery to whole business datum
A variety of reset modes, therefore can guarantee the efficiency that data are restored, recovery time is saved, data Resuming agent is reduced.
Further, the concrete embodiment as method shown in Fig. 1 and Fig. 2, the embodiment of the present application provide a kind of data and deposit
Storage and the device restored, as shown in figure 3, the device includes: to obtain module 31, processing module 32, memory module 33, recovery module
34。
Module 31 is obtained, can be used for obtaining the multiple data blocks being evenly dividing by raw data file；
Processing module 32 can be used for that data block coding is processed into multiple data slices and check sheet based on correcting and eleting codes；
Memory module 33 can be used for utilizing data slice and check sheet storage raw data file；
Recovery module 34 can be used for if it is determined that raw data file lacks, then using the data slice for meeting preset condition and
Check sheet decoding restores raw data file.
In specific application scenarios, in order to which data block coding is processed into multiple data slices and check sheet, processing module
32, it is particularly used in and data block is encoded using correcting and eleting codes, raw data file is divided into according to the first division rule
Identical first data slice of size and the first check sheet；Secondary coding is carried out to the first data slice using correcting and eleting codes, by each the
One data slice is divided into identical second data slice of size and the second check sheet according to the second division rule.
Correspondingly, safety and restorability in order to guarantee data storage, memory module 33, being particularly used in will be each
A first data slice is stored in the first different data centers, each first check sheet is stored in the second different data
The heart；Second data slice and the second check sheet are respectively corresponded to the different storage units being stored in the first data center.
In specific application scenarios, in order to provide the two ways for determining raw data file missing, as shown in figure 4,
The present apparatus further include: determining module 35.
Obtain module 31, it may also be used for obtain raw data file by reading all first data slices；And/or pass through
All second data slices are read to obtain raw data file；
Determining module 35 can be used for if it is determined that there is the first data slice or the second data slice of missing, it is determined that original number
According to missing documents.
Correspondingly, obtaining module 31 in order to obtain raw data file by reading all first data slices, specifically may be used
For obtaining the first serial number of each first data slice, wherein the corresponding first data slice assembly raw data file of the first serial number
Put in order；The first data slice is read according to the first ascending serial number, to obtain raw data file.
Correspondingly, obtaining module 31 in order to obtain raw data file by reading all second data slices, specifically may be used
For obtaining the second serial number of each second data slice, wherein the corresponding second data slice assembly raw data file of the second serial number
Put in order；The second data slice is read according to the second ascending serial number, to obtain raw data file.
In specific application scenarios, in order to determine the first data slice there are when shortage of data, using meeting default item
Data slice and the check sheet decoding of part restore raw data file, and recovery module 34 is particularly used in and determines that there are the first data
The first data center of target of sheet data missing；Extract the second data slice of all targets and target in the first data center of target
Second check sheet；If it is determined that target the second data sheet data is complete, then the second data slice of target is read according to the second serial number, so as to
Obtain the first data slice of target of shortage of data；If it is determined that the second data slice of target is there are shortage of data and to meet the second data extensive
Multiple condition then restores the target the of shortage of data using complete the second data slice of target of data and the decoding of the second check sheet of target
Two data slices read the second data slice of target after restoring according to the second serial number, to obtain the first data slice of target, wherein
The quantity that second data restore the second check sheet of the second data slice of target and target that condition is shortage of data is less than or equal to the
Two preset thresholds；If it is determined that the second data slice of target is there are shortage of data and does not meet the second data recovery condition, then number is obtained
According to complete first data slice and the first check sheet；If it is determined that the first data slice of target, which meets the first data, restores condition, then it is sharp
Restore the first data slice of target with complete first data slice of data and the decoding of the first check sheet, wherein the first data restore item
Part is that the quantity of the first check sheet of the first data slice of target and target of shortage of data is less than or equal to the first preset threshold；If sentencing
First data slice that sets the goal does not meet the first data and restores condition, then output data restores abnormal warning message.
Correspondingly, in order to determine the second data slice there are when shortage of data, using the data slice for meeting preset condition and
Check sheet decoding restores raw data file, and recovery module 34 is particularly used in and determines that there are the second data slice shortage of data
The first data center of target；It obtains complete the second data slice of target of the first intra-data center data of target and target second verifies
Piece；If it is determined that the second data slice of target of shortage of data, which meets the second data, restores condition, then the complete target of data the is utilized
Two data slices and the decoding of the second check sheet of target restore the second data slice of target, wherein the second data are restored condition and lacked for data
The quantity of the second check sheet of the second data slice of target and target of mistake is less than or equal to the second preset threshold；If it is determined that target second
Data slice does not meet the second data and restores condition, then obtains the first data slice of target in the first data center of target；If it is determined that
Target the first data sheet data is complete, then repartitions into the first data slice of target using error correcting code and meet the second division rule
The second data slice and the second check sheet, to replace the second data slice of target；If it is determined that target the first data slice shortage of data
And meet the first data and restore condition, then restore shortage of data using complete first data slice of data and the decoding of the first check sheet
The first data slice of target, wherein the first data restore condition be shortage of data the first school of the first data slice of target and target
The quantity for testing piece is less than or equal to the first preset threshold；If it is determined that target the first data slice shortage of data and not meet the first data extensive
Multiple condition, then output data restores abnormal warning message.
It should be noted that each functional unit involved by a kind of data storage provided in this embodiment and the device restored
Other corresponding descriptions, can be referring to figs. 1 to the corresponding description in Fig. 2, and details are not described herein.
Based on above-mentioned method as depicted in figs. 1 and 2, correspondingly, the embodiment of the present application also provides a kind of storage medium,
On be stored with computer program, which realizes above-mentioned data storage as depicted in figs. 1 and 2 and restores when being executed by processor
Method.
Based on this understanding, the technical solution of the application can be embodied in the form of software products, which produces
Product can store in a non-volatile memory medium (can be CD-ROM, USB flash disk, mobile hard disk etc.), including some instructions
With so that computer equipment (can be personal computer, server or the network equipment an etc.) execution the application is each
The method of implement scene.
Based on above-mentioned method as shown in Figure 1 and Figure 2 and Fig. 3, virtual bench embodiment shown in Fig. 4, in order to realize
Above-mentioned purpose, the embodiment of the present application also provides a kind of computer equipments, are specifically as follows personal computer, server, network
Equipment etc., the entity device include storage medium and processor；Storage medium, for storing computer program；Processor is used for
The method that computer program is executed to realize above-mentioned data storage as depicted in figs. 1 and 2 and recovery.
Optionally, which can also include user interface, network interface, camera, radio frequency (Radio
Frequency, RF) circuit, sensor, voicefrequency circuit, WI-FI module etc..User interface may include display screen
(Display), input unit such as keyboard (Keyboard) etc., optional user interface can also connect including USB interface, card reader
Mouthful etc..Network interface optionally may include standard wireline interface and wireless interface (such as blue tooth interface, WI-FI interface).
It will be understood by those skilled in the art that computer equipment structure provided in this embodiment is not constituted and is set to the entity
Standby restriction may include more or fewer components, perhaps combine certain components or different component layouts.
It can also include operating system, network communication module in non-volatile readable storage medium.Operating system is video
The program of the entity device hardware and software resource of shot cuts supports message handling program and other softwares and/or program
Operation.Network communication module for realizing the communication between component each inside non-volatile readable storage medium, and with this
It is communicated between other hardware and softwares in entity device.
Through the above description of the embodiments, those skilled in the art can be understood that the application can borrow
It helps software that the mode of necessary general hardware platform is added to realize, hardware realization can also be passed through.Pass through the skill of application the application
Art scheme, compared with currently available technology, raw data file can tentatively be evenly dividing as multiple data blocks by the application, be based on
Correcting and eleting codes technology carries out coded treatment to data block, acquires multiple first data slices and the first check sheet, is based on again later
Correcting and eleting codes technology encodes the first data slice, and each first data slice is again divided into multiple second data slices and second
Each first data slice is stored in the first different data centers by check sheet, each first check sheet is stored in different
Second data center；It is single that second data slice and the second check sheet are respectively corresponded to the different storages being stored in the first data center
Member.Raw data file can be read by two ways, the first reading manner: by sequentially reading in each first data
Intracardiac the first data slice obtains raw data file.Second of reading manner: by sequentially reading each first data slice
Corresponding second data slice obtains raw data file.Two kinds of reading manners correspond to two kinds of data reset modes again, the first
Reset mode:, can be by all complete in the first data center where reading when determining has the first data slice of missing
Second data slice to obtain former first data slice, and then realizes that the replacement to the first data slice of missing restores.In addition, when determining
The second data slice in the first data center of place equally exist shortage of data and it is irrecoverable when, data complete can be passed through
One data slice and the first check sheet are decoded recovery.Second of reset mode: when determining has the second data slice of missing,
Recovery can be decoded by complete second data slice of other data and the second check sheet in first data center.When can not
When realizing recovery, then its corresponding first data slice is obtained, re-starts the second data slice and the second school using the first data slice
The coding for testing piece divides, thus for the second data slice for restoring the missing.The two kinds of data reading modes provided by this programme,
And to two kinds of reset modes of missing data, it can be achieved that duplicate protection to business datum, when being sent out in data read process
When raw shortage of data, reading manner can be replaced in time, the integrality and safety of business datum is effectively ensured.In addition, this
Scheme is by the storage and recovery to whole business datum, specific to the storage and recovery to data slice, and provides a variety of extensive
Compound formula, therefore can guarantee the efficiency that data are restored, recovery time is saved, data Resuming agent is reduced.
It will be appreciated by those skilled in the art that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, module in attached drawing or
Process is not necessarily implemented necessary to the application.It will be appreciated by those skilled in the art that the mould in device in implement scene
Block can according to implement scene describe be distributed in the device of implement scene, can also carry out corresponding change be located at be different from
In one or more devices of this implement scene.The module of above-mentioned implement scene can be merged into a module, can also be into one
Step splits into multiple submodule.
Above-mentioned the application serial number is for illustration only, does not represent the superiority and inferiority of implement scene.Disclosed above is only the application
Several specific implementation scenes, still, the application is not limited to this, and the changes that any person skilled in the art can think of is all
The protection scope of the application should be fallen into.
Claims (10)
1. a kind of method data storage and restored characterized by comprising
Obtain the multiple data blocks being evenly dividing by raw data file；
Data block coding is processed into multiple data slices and check sheet based on correcting and eleting codes；
The raw data file is stored using the data slice and the check sheet；
If it is determined that the raw data file missing, then decoded using the data slice for meeting preset condition and the check sheet
Restore the raw data file.
2. the method according to claim 1, wherein described be processed into data block coding based on correcting and eleting codes
Multiple data slices and check sheet, specifically include:
The data block is encoded using correcting and eleting codes, the raw data file is divided into greatly according to the first division rule
Small identical first data slice and the first check sheet；
Secondary coding is carried out to first data slice using correcting and eleting codes, each first data slice is divided according to second and is advised
Then it is divided into identical second data slice of size and the second check sheet.
3. according to the method described in claim 2, it is characterized in that, described store institute using the data slice and the check sheet
Raw data file is stated, is specifically included:
Each first data slice is stored in the first different data centers, each first check sheet is stored in not
The second same data center；
Second data slice and second check sheet are respectively corresponded the difference being stored in first data center to deposit
Storage unit.
4. according to the method described in claim 3, it is characterized in that, using meeting the data slice of preset condition and described
Before check sheet decoding restores the raw data file, specifically further include:
The raw data file is obtained by reading all first data slices；And/or
The raw data file is obtained by reading all second data slices；
If it is determined that there is first data slice or second data slice of missing, it is determined that the raw data file lacks
It loses.
5. according to the method described in claim 4, it is characterized in that, described obtained by reading all first data slices
The raw data file, specifically includes:
Obtain the first serial number of each first data slice, wherein it is assembled that first serial number corresponds to first data slice
The raw data file puts in order；
First data slice is read according to ascending first serial number, to obtain the raw data file；
It is described to obtain the raw data file by reading all second data slices, it specifically includes:
Obtain the second serial number of each second data slice, wherein it is assembled that second serial number corresponds to second data slice
The raw data file puts in order；
Second data slice is read according to ascending second serial number, to obtain the raw data file.
6. according to the method described in claim 5, it is characterized in that, if it is determined that first data slice there are shortage of data, institutes
It states and restores the raw data file using the data slice for meeting preset condition and check sheet decoding, specifically include:
Determine that there are the first data centers of target of the first data slice shortage of data；
Extract the second check sheet of the second data slice of all targets and target in first data center of target；
If it is determined that the second data of target sheet data is complete, then second data of target are read according to second serial number
Piece, to obtain the first data slice of target of shortage of data；
If it is determined that second data slice of target is there are shortage of data and meets the second data recovery condition, then it is complete using data
The second data slice of target and the decoding of the second check sheet of target restore the second data slice of target of the shortage of data, according to described
Second serial number reads the second data slice of the target after restoring, to obtain first data slice of target, wherein described the
The quantity that two data restore the second check sheet of the second data slice of target and target that condition is shortage of data is less than or equal to second
Preset threshold；
If it is determined that second data slice of target is there are shortage of data and does not meet the second data recovery condition, then number is obtained
According to complete first data slice and the first check sheet；
If it is determined that first data slice of target, which meets the first data, restores condition, then complete first data of the data are utilized
Piece and the decoding of the first check sheet restore first data slice of target, wherein it is shortage of data that first data, which restore condition,
The first check sheet of the first data slice of target and target quantity be less than or equal to the first preset threshold；
If it is determined that first data slice of target, which does not meet first data, restores condition, then output data restores abnormal report
Alert information.
7. according to the method described in claim 5, it is characterized in that, if it is determined that second data slice there are shortage of data, institutes
It states and restores the raw data file using the data slice for meeting preset condition and check sheet decoding, specifically include:
Determine that there are the first data centers of target of the second data slice shortage of data；
Obtain complete the second data slice of target of first intra-data center data of target and the second check sheet of target；
If it is determined that the second data slice of target of shortage of data, which meets the second data, restores condition, then the complete mesh of the data is utilized
It marks the second data slice and the decoding of the second check sheet of target restores second data slice of target, wherein second data are restored
Condition is that the quantity of the second check sheet of the second data slice of target and target of shortage of data is less than or equal to the second preset threshold；
If it is determined that second data slice of target, which does not meet second data, restores condition, then first data of target are obtained
The first data slice of target in center；
If it is determined that the first data of target sheet data is complete, then first data slice of target is repartitioned using error correcting code
At the second data slice and the second check sheet for meeting second division rule, to replace second data slice of target；
If it is determined that target the first data slice shortage of data and meeting the first data and restoring condition, then data complete the are utilized
One data slice and first check sheet decoding restore the first data slice of target of the shortage of data, wherein first number
Quantity according to the first check sheet of the first data slice of target and target that the condition of recovery is shortage of data is less than or equal to the first default threshold
Value；
If it is determined that target the first data slice shortage of data and not meeting first data and restoring condition, then output data is extensive
Multiple abnormal warning message.
8. a kind of device data storage and restored characterized by comprising
Module is obtained, for obtaining the multiple data blocks being evenly dividing by raw data file；
Processing module, for data block coding to be processed into multiple data slices and check sheet based on correcting and eleting codes；
Memory module, for storing the raw data file using the data slice and the check sheet；
Recovery module is used for if it is determined that the raw data file lacks, then using the data slice for meeting preset condition and
The check sheet decoding restores the raw data file.
9. a kind of non-volatile readable storage medium, is stored thereon with computer program, which is characterized in that described program is processed
Device is realized the storage of data described in any one of claims 1 to 7 and restored method when executing.
10. a kind of computer equipment, including non-volatile readable storage medium, processor and it is stored in non-volatile readable storage
On medium and the computer program that can run on a processor, which is characterized in that the processor is realized when executing described program
The method they data described in any one of claims 1 to 7 storage and restored.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910624964.6A CN110502365B (en) | 2019-07-11 | 2019-07-11 | Data storage and recovery method and device and computer equipment |
| PCT/CN2019/103524 WO2021003822A1 (en) | 2019-07-11 | 2019-08-30 | Data storage and recovery method and apparatus, and computer device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910624964.6A CN110502365B (en) | 2019-07-11 | 2019-07-11 | Data storage and recovery method and device and computer equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110502365A true CN110502365A (en) | 2019-11-26 |
| CN110502365B CN110502365B (en) | 2024-03-01 |
Family
ID=68585585
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910624964.6A Active CN110502365B (en) | 2019-07-11 | 2019-07-11 | Data storage and recovery method and device and computer equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110502365B (en) |
| WO (1) | WO2021003822A1 (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111240597A (en) * | 2020-01-15 | 2020-06-05 | 书生星际(北京)科技有限公司 | Method, device, equipment and computer readable storage medium for storing data |
| CN111541512A (en) * | 2020-03-13 | 2020-08-14 | 中国科学院深圳先进技术研究院 | Data processing method, terminal device and readable storage medium |
| CN111596861A (en) * | 2020-05-06 | 2020-08-28 | 清华大学 | Data storage method and device and data recovery method and device |
| CN111682874A (en) * | 2020-06-11 | 2020-09-18 | 山东云海国创云计算装备产业创新中心有限公司 | Data recovery method, system, equipment and readable storage medium |
| CN111818124A (en) * | 2020-05-29 | 2020-10-23 | 平安科技（深圳）有限公司 | Data storage method, data storage device, electronic equipment and medium |
| CN112052114A (en) * | 2020-08-27 | 2020-12-08 | 江苏超流信息技术有限公司 | Data storage and recovery method, coder-decoder and coder-decoder system |
| CN112558875A (en) * | 2020-12-14 | 2021-03-26 | 北京百度网讯科技有限公司 | Data verification method and device, electronic equipment and storage medium |
| CN112948335A (en) * | 2021-02-23 | 2021-06-11 | 北京星震同源数字系统股份有限公司 | Data processing method and system |
| CN113419684A (en) * | 2021-07-09 | 2021-09-21 | 深圳大普微电子科技有限公司 | Data processing method, device and equipment and readable storage medium |
| CN117608500A (en) * | 2024-01-23 | 2024-02-27 | 四川省华存智谷科技有限责任公司 | Method for rescuing effective data of storage system when data redundancy is insufficient |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112860475B (en) * | 2021-02-04 | 2023-02-28 | 山东云海国创云计算装备产业创新中心有限公司 | Method, device, system and medium for recovering check block based on RS erasure code |
| CN114884624B (en) * | 2022-07-08 | 2022-12-09 | 广州思德医疗科技有限公司 | Data processing method and device |
| CN115543697B (en) * | 2022-11-30 | 2023-03-10 | 北京网智易通科技有限公司 | Electronic archive storage method and device, electronic equipment and storage medium |
| CN117420969B (en) * | 2023-12-19 | 2024-04-16 | 中电云计算技术有限公司 | Distributed data storage method, device, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101834899A (en) * | 2010-04-29 | 2010-09-15 | 中科院成都信息技术有限公司 | Distributed adaptive coding and storing method |
| CN102546755A (en) * | 2011-12-12 | 2012-07-04 | 华中科技大学 | Data storage method of cloud storage system |
| CN102833040A (en) * | 2012-08-03 | 2012-12-19 | 中兴通讯股份有限公司 | Method and device for decoding, and coding and decoding system |
| CN105007286A (en) * | 2014-04-23 | 2015-10-28 | 苏宁云商集团股份有限公司 | Decoding method, decoding device, and cloud storage method and system |
| CN108170555A (en) * | 2017-12-21 | 2018-06-15 | 浙江大华技术股份有限公司 | A kind of data reconstruction method and equipment |
| CN108334419A (en) * | 2017-12-25 | 2018-07-27 | 华为技术有限公司 | A kind of method and apparatus that data are restored |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104111880B (en) * | 2013-04-16 | 2016-03-02 | 华中科技大学 | A kind of forms data dish inefficacy fast reconstructing method holding three dish inefficacy correcting and eleting codes |
| KR20150061258A (en) * | 2013-11-27 | 2015-06-04 | 한국전자통신연구원 | Operating System and Method for Parity chunk update processing in distributed Redundant Array of Inexpensive Disks system |
-
2019
- 2019-07-11 CN CN201910624964.6A patent/CN110502365B/en active Active
- 2019-08-30 WO PCT/CN2019/103524 patent/WO2021003822A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101834899A (en) * | 2010-04-29 | 2010-09-15 | 中科院成都信息技术有限公司 | Distributed adaptive coding and storing method |
| CN102546755A (en) * | 2011-12-12 | 2012-07-04 | 华中科技大学 | Data storage method of cloud storage system |
| CN102833040A (en) * | 2012-08-03 | 2012-12-19 | 中兴通讯股份有限公司 | Method and device for decoding, and coding and decoding system |
| CN105007286A (en) * | 2014-04-23 | 2015-10-28 | 苏宁云商集团股份有限公司 | Decoding method, decoding device, and cloud storage method and system |
| CN108170555A (en) * | 2017-12-21 | 2018-06-15 | 浙江大华技术股份有限公司 | A kind of data reconstruction method and equipment |
| CN108334419A (en) * | 2017-12-25 | 2018-07-27 | 华为技术有限公司 | A kind of method and apparatus that data are restored |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111240597A (en) * | 2020-01-15 | 2020-06-05 | 书生星际(北京)科技有限公司 | Method, device, equipment and computer readable storage medium for storing data |
| CN111240597B (en) * | 2020-01-15 | 2024-05-17 | 书生星际(北京)科技有限公司 | Method, apparatus, device and computer readable storage medium for storing data |
| CN111541512A (en) * | 2020-03-13 | 2020-08-14 | 中国科学院深圳先进技术研究院 | Data processing method, terminal device and readable storage medium |
| CN111596861A (en) * | 2020-05-06 | 2020-08-28 | 清华大学 | Data storage method and device and data recovery method and device |
| WO2021120585A1 (en) * | 2020-05-29 | 2021-06-24 | 平安科技（深圳）有限公司 | Data storage method, apparatus, device and storage medium |
| CN111818124A (en) * | 2020-05-29 | 2020-10-23 | 平安科技（深圳）有限公司 | Data storage method, data storage device, electronic equipment and medium |
| CN111682874B (en) * | 2020-06-11 | 2022-06-17 | 山东云海国创云计算装备产业创新中心有限公司 | Data recovery method, system, equipment and readable storage medium |
| CN111682874A (en) * | 2020-06-11 | 2020-09-18 | 山东云海国创云计算装备产业创新中心有限公司 | Data recovery method, system, equipment and readable storage medium |
| CN112052114A (en) * | 2020-08-27 | 2020-12-08 | 江苏超流信息技术有限公司 | Data storage and recovery method, coder-decoder and coder-decoder system |
| CN112052114B (en) * | 2020-08-27 | 2024-05-07 | 江苏超流信息技术有限公司 | Data storage and recovery method, coder and decoder and coding and decoding system |
| CN112558875A (en) * | 2020-12-14 | 2021-03-26 | 北京百度网讯科技有限公司 | Data verification method and device, electronic equipment and storage medium |
| CN112948335A (en) * | 2021-02-23 | 2021-06-11 | 北京星震同源数字系统股份有限公司 | Data processing method and system |
| CN113419684A (en) * | 2021-07-09 | 2021-09-21 | 深圳大普微电子科技有限公司 | Data processing method, device and equipment and readable storage medium |
| CN113419684B (en) * | 2021-07-09 | 2023-02-24 | 深圳大普微电子科技有限公司 | Data processing method, device and equipment and readable storage medium |
| CN117608500A (en) * | 2024-01-23 | 2024-02-27 | 四川省华存智谷科技有限责任公司 | Method for rescuing effective data of storage system when data redundancy is insufficient |
| CN117608500B (en) * | 2024-01-23 | 2024-03-29 | 四川省华存智谷科技有限责任公司 | Method for rescuing effective data of storage system when data redundancy is insufficient |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021003822A1 (en) | 2021-01-14 |
| CN110502365B (en) | 2024-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110502365A (en) | The method, apparatus and computer equipment they data storage and restored | |
| US10725692B2 (en) | Data storage method and apparatus | |
| US10481978B2 (en) | Optimal slice encoding strategies within a dispersed storage unit | |
| US20170195420A1 (en) | Detailed memory device statistics with drive write location determination | |
| EP3101530A1 (en) | Data distribution method, data storage method, related apparatus, and system | |
| EP3109757A2 (en) | Data storage method, data recovery method, related apparatus, and system | |
| US20130132770A1 (en) | Method and apparatus for reconstructing redundant array of inexpensive disks, and system | |
| CN105468479B (en) | A kind of disk array RAID bad block processing methods and device | |
| US10530862B2 (en) | Determining slices to rebuild from low-level failures | |
| CN110750213A (en) | Hard disk management method and device | |
| CN114153651B (en) | Data encoding method, device, equipment and medium | |
| CN104468548A (en) | Data verification method, disaster recovery storage and production storage | |
| CN105094684B (en) | The method for reusing and system of problem disk in disc array system | |
| CN110008056A (en) | EMS memory management process, device, electronic equipment and computer readable storage medium | |
| CN115098295A (en) | Data local recovery method, equipment and storage medium | |
| CN111459710A (en) | Erasure code memory recovery method, device and memory system capable of sensing heat degree and risk | |
| CN114415982A (en) | Data storage method, device and equipment and readable storage medium | |
| CN105183583A (en) | Method for data reconstruction of disk array, and disk array system | |
| US9524213B2 (en) | Storage control apparatus, recording medium having stored therein storage control program and storage control method | |
| CN115543693B (en) | Data recovery method and related equipment | |
| CN105117303A (en) | Data recovery method and equipment | |
| Iliadis et al. | Most probable paths to data loss: An efficient method for reliability evaluation of data storage systems | |
| CN106708445A (en) | Link selection method and device | |
| JPWO2014010077A1 (en) | Disk array control device, disk array control method, and disk array control program | |
| CN110502496A (en) | A kind of distributed file system restorative procedure, system, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |