CN107346273A - A kind of data reconstruction method, device and electronic equipment - Google Patents
A kind of data reconstruction method, device and electronic equipment Download PDFInfo
- Publication number
- CN107346273A CN107346273A CN201710446718.7A CN201710446718A CN107346273A CN 107346273 A CN107346273 A CN 107346273A CN 201710446718 A CN201710446718 A CN 201710446718A CN 107346273 A CN107346273 A CN 107346273A
- Authority
- CN
- China
- Prior art keywords
- data block
- path
- fault data
- server
- fault
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiments of the invention provide a kind of data reconstruction method, device and electronic equipment, method includes：Obtain the target data block of destination object；Two adjacent data blocks of target data block are obtained, and detects in two adjacent data blocks and whether there is fault data block；If there is fault data block, all data blocks of destination object are obtained, and fault data block and normal data block are determined from all data blocks of destination object；Can judgement be locked to the path where each fault data block, if can lock, the path where each fault data block is locked successively according to preset order；Using predetermined number normal data block, each fault data block is reverted into normal data block corresponding with the fault data block, normal data block corresponding with the fault data block is sent to server corresponding to the path where each fault data block.When carrying out data recovery using scheme provided in an embodiment of the present invention, realize and save Internet resources and computing resource.
Description
Technical field
The present invention relates to cloud storage technical field, more particularly to a kind of data reconstruction method, device and electronic equipment.
Background technology
In recent years, cloud storage system was developed rapidly, and conventional cloud storage system includes：Microsoft Azure
Storage (Microsoft's cloud storage), Facebook (facebook), Google (Google) and Openstack Swift (high in the clouds storage of increasing income
Project).Wherein, Openstack Swift cloud storage systems are a kind of distributed objects storage systems, and so-called distribution refers to this
Cloud storage system includes multiple servers, and so-called object storage is exactly to be stored data as an object, for example, figure
Sheet data, video data, voice data, document data etc., can serve as object and are stored in Openstack Swift clouds depositing
In any one server of storage system.
For the data degradation for preventing the system failure from bringing, the availability and reliability of data, many cloud storage systems are improved
The storage strategy used unite as replication policy.So-called replication policy is exactly that each part of data are called one by more parts of data storage
Copy.In order to prevent adjacent part server collective failure, the different copies of data are stored in different physical regions not
With the different disk of server.So, even if one of copy is lost, data still have.
The conventional storage strategy of another kind is correcting and eleting codes.For replication policy, correcting and eleting codes are ensureing that data can use
While property and reliability, substantial amounts of memory space is saved.The general principle of correcting and eleting codes is that original data are being divided into k
Data block, then by carrying out computing according to block and a companion matrix to this k, obtain r check block.In this k+r school
Test in block and data block, even if lost wherein any 1 arrives r check block and/or data block, remaining data can be passed through
Block sum check block is recovered.In actual applications, r value is typically less than k, so ensureing the premise of data redundancy
Under, memory space is saved, so as to save cost.
At present, many cloud storage systems all apply correcting and eleting codes as storage strategy, with using correcting and eleting codes storage strategy
Illustrated exemplified by the data recovery procedure of Openstack Swift cloud storage systems, data recovery procedure is：Cloud storage system
In every server by Reconstructor (recovery) processes to the adjacent of each data block for being stored on the server
Data block is scanned, and the adjacent data blocks of each data block refer to belong to same target with the data block, and with the data
Block data block adjacent on Hash ring, if it find that when the adjacent data blocks of data block are fault data block, then can be to failure
Data block is recovered.Server to fault data block when recovering, it is necessary to read k part normal datas from other servers
Block, and then matrix computations are carried out using this k part normal datas block, fault data block is reverted into normal data block, so that will just
Regular data block is sent to the server of storage fault data block, completes the recovery to fault data block.
There is the problem of multiple servers carry out data recovery to a fault data block simultaneously in the above method, for example, such as
In distributed system architecture described in Fig. 1, data block b adjacent data blocks are data block a and data block c, and data block c's is adjacent
Data block is data block b and data block d, and data block a, b, c, d are stored respectively in the server A of distributed system 101, B, C, D
In, if data block c is fault data block, server B and server D can detect that it is fault data block, and then take
Being engaged in device B and server D all may be to data block c progress data recovery, so as to cause the Internet resources of cloud storage system
With the waste of computing resource.
The content of the invention
The purpose of the embodiment of the present invention is to provide a kind of data reconstruction method, device and electronic equipment, saved with realizing
Internet resources and computing resource.Concrete technical scheme is as follows：
In a first aspect, the embodiments of the invention provide a kind of data reconstruction method, applied in distributed memory system
Server, for server every described, methods described includes：
The target data block of destination object is obtained, wherein, the target data block is：All data of the destination object
The data block being stored in block on the server；
Two adjacent data blocks of the target data block are obtained, and detects and whether there is in described two adjacent data blocks
Fault data block；
If there is fault data block, all data blocks of the destination object are obtained, and from the institute of the destination object
Have and fault data block and normal data block are determined in data block；
Can judgement lock to the path where each fault data block, if can lock, by each institute
Path where stating fault data block is locked successively according to preset order；Using the predetermined number normal data block,
Each fault data block is reverted into normal data block corresponding with the fault data block, to each fault data block
Server corresponding to the path at place sends normal data block corresponding with the fault data block.
Optionally, if there is no fault data block, or judgement can not be to the road where each fault data block
Footpath is locked, and methods described also includes：
Any object in addition to the destination object being stored on the server is obtained, as destination object, is returned
The step of performing the target data block for obtaining the destination object being stored on the server.
Optionally, methods described also includes：
Path where each fault data block is unlocked.
Optionally, the step of two adjacent data blocks of the acquisition target data block, including：
Using the Hash ring being stored on the server, the path where each adjacent data blocks is obtained；
Using the path where each adjacent data blocks, it is determined that each service of each adjacent data blocks of storage
Device, and each server is based on, obtain each adjacent data blocks.
Optionally, each fault data block includes unique number, and can the judgement to each fault data
Path where block is locked, including：
Check whether the path where numbering minimum or the largest number of fault data block has been locked；
If be locked, the difference of initial time that the path is locked and current time is obtained, judges the difference
Whether value is more than predetermined threshold value, if it is greater, then judging that the path where each fault data block can be locked；Such as
Fruit is less than or equal to, then judges that the path where each fault data block can not be locked；
If be not locked, judgement can lock to the path where each fault data block.
Optionally, for inspection it is the minimum fault data block situation of numbering, it is described by each fault data block institute
Path locked successively according to preset order, including：
According to the order of the numbering from small to large, the path where each fault data block is added successively
Lock；
What it is for inspection is situation of the numbering maximum number of faults according to block, the road by where each fault data block
Footpath locks successively according to preset order, including：
According to the order of the numbering from big to small, the path where each fault data block is added successively
Lock.
Second aspect, the embodiments of the invention provide a kind of Data Recapture Unit, applied in distributed memory system
Server, for server every described, described device includes：
First obtains module, for obtaining the target data block of destination object, wherein, the target data block is：It is described
The data block being stored in all data blocks of destination object on the server；
Second obtains module, for obtaining two adjacent data blocks of the target data block, and detects described two phases
It whether there is fault data block in adjacent data block；
3rd obtains module, for when the judged result of the described second acquisition module is is, obtaining the destination object
All data blocks, and fault data block and normal data block are determined from all data blocks of the destination object；
Judge module, for judging that the path where each fault data block can be locked, if can add
Lock, then locked in the path where each fault data block according to preset order successively；Utilize predetermined number institute
Normal data block is stated, each fault data block is reverted into normal data block corresponding with the fault data block, to each
Server corresponding to path where the fault data block sends normal data block corresponding with the fault data block.
Optionally, described device also includes：
4th obtains module, when the judged result for obtaining module described second is no, or, the judge module
Judged result for it is no when, obtain and be stored in any object in addition to the destination object on the server, as target
Object, return to the step of performing the target data block for obtaining the destination object being stored on the server.
Optionally, described device also includes：
Unlocked state, for the path where each fault data block to be unlocked.
Optionally, described second module is obtained, including：
First obtains submodule, for using the Hash ring being stored on the server, obtaining each adjacent data
Path where block；
Second obtains submodule, for the path where each adjacent data blocks of utilization, it is determined that storage is each described
Each server of adjacent data blocks, and each server is based on, obtain each adjacent data blocks.
Optionally, each fault data block includes unique number, the judge module, including：
Detection sub-module, for check that numbering is minimum or the largest number of fault data block where path whether by
Lock；
First decision sub-module, for when the testing result of the detection sub-module is is, obtaining the path and being locked
Initial time and current time difference, judge whether the difference is more than predetermined threshold value, if it is greater, then judge can be to each
Path where the individual fault data block is locked；If less than or be equal to, then judge can not be to each failure
Path where data block is locked；
Second decision sub-module, for when the testing result of the detection sub-module is no, judgement can be to each described
Path where fault data block is locked.
Optionally, for inspection it is the minimum fault data block situation of numbering, the judge module, including：
First locks submodule, for the order according to the numbering from small to large, successively by each fault data
Path where block is locked；
What it is for inspection is situation of the numbering maximum number of faults according to block, the judge module, including：
Second locks submodule, for the order according to the numbering from big to small, successively by each fault data
Path where block is locked.
At the another aspect that the present invention is implemented, a kind of computer-readable recording medium is additionally provided, it is described computer-readable
Instruction is stored with storage medium, when run on a computer so that it is extensive that computer performs any of the above-described described data
Compound method.
At the another aspect that the present invention is implemented, the embodiment of the present invention additionally provides a kind of computer program production comprising instruction
Product, when run on a computer so that computer performs any of the above-described described data reconstruction method.
It can be seen that using the embodiment of the present invention, the path of all fault data blocks of one object is entered by a server
Row locks, then other servers hinder data block path and are locked and can not access the fault data block for some reason, therefore can only be by locking
The server in the fault data block path, data recovery is carried out to all fault data blocks of the object, avoids multiple services
Device carries out data recovery to same fault data block simultaneously, has saved Internet resources and computing resource.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described.
Fig. 1 is distributed system architecture figure of the prior art；
Fig. 2 is a kind of schematic flow sheet of data reconstruction method provided in an embodiment of the present invention；
Fig. 3 is the distributed system architecture using the carry out data recovery of data reconstruction method provided in an embodiment of the present invention
Figure；
Fig. 4 is the carry out data recovery operational flow diagram using data reconstruction method provided in an embodiment of the present invention；
Fig. 5 is a kind of structural representation of Data Recapture Unit provided in an embodiment of the present invention；
The structural representation for another Data Recapture Unit that Fig. 6 is provided by the embodiment of the present invention；
The structural representation for another Data Recapture Unit that Fig. 7 is provided by the embodiment of the present invention；
Fig. 8 is the structural representation that the embodiment of the present invention also provides a kind of electronic equipment.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is described.
In order to solve the problems, such as Internet resources in the prior art and computing resource waste, the embodiment of the invention discloses one kind
Data reconstruction method, device and electronic equipment.
Specifically, a kind of application scenarios of data reconstruction method provided in an embodiment of the present invention can be：In server
Recovering process obtains the target data block for the destination object being stored on the server, further, obtains the target data block
Two adjacent data blocks, and detect in two adjacent data blocks and whether there is fault data block, if there is fault data block, then
Obtain all data blocks of the destination object, and determine from all data blocks of the destination object fault data block and normal
Data block；Can judgement lock to the path where each fault data block, if can lock, by each fault data
Path where block is locked successively according to preset order；Using predetermined number normal data block, by each fault data
Block reverts to normal data block corresponding with the fault data block, to server corresponding to the path where each fault data block
Send normal data block corresponding with the fault data block.
A kind of data reconstruction method provided in an embodiment of the present invention is described in detail below.
It should be noted that a kind of data reconstruction method that the embodiment of the present invention is provided, applied to distributed storage system
Server in system.Moreover, it is achieved that a kind of functional software for data reconstruction method that the embodiment of the present invention is provided can be special
The data recovery software of door, or available data, which recovers software or other, has inserting in the software of data recovery function
Part.
Referring to Fig. 2, Fig. 2 is a kind of schematic flow sheet of data reconstruction method provided in an embodiment of the present invention, for distribution
Every server in formula storage system, method comprise the following steps：
S201, obtain the target data block of destination object.
It is understood that the server in distributed memory system can store the data block of each object, also, often
Platform server can obtain the data block of itself storage, can also be sent to other servers and obtain request, to be stored
In the data block of other servers.Each object has multiple data blocks, wherein, target data block is：The institute of the destination object
There is the data block being stored in data block on the server.
S202, two adjacent data blocks of the target data block are obtained, and detect and whether there is in two adjacent data blocks
Fault data block, if there is fault data block, then perform S203.
It is understood that adjacent data blocks refer to the data block adjacent on Hash ring with target data block.Target pair
All data blocks of elephant all correspond to a position on Hash ring respectively, by obtain with target data block on Hash ring it is adjacent
Data block, it is possible to obtain each adjacent data blocks of destination object.Each adjacent data blocks and target data block are all mesh
The data block of object is marked, in practical application, adjacent data blocks can be stored in same server with target data block, can also be with
Target data block is stored respectively in different server.The embodiment of the present invention is not limited the specific store path of adjacent data blocks
It is fixed.
Fault data block refers to because data are destroyed, and cause server can not normal reading of content data block.Tool
Whether the detection data block of body is that the method for fault data block is prior art, and the embodiment of the present invention will not be repeated here.This hair
Bright embodiment is not limited the method specifically detected.It is for instance possible to use Algorithm for Consistency Checking, detects adjacent data blocks
In whether there is fault data block.
S203, all data blocks of the destination object are obtained, and failure is determined from all data blocks of the destination object
Data block and normal data block.
It is understood that for each object, if some data block of the object is fault data block, then
The object there will be likely some multiple fault data blocks, and therefore, adjacent data blocks are as in all data blocks of destination object
A part, if fault data block in adjacent data blocks be present, show in all data blocks of destination object, it is understood that there may be remove
Data block beyond adjacent data blocks is also fault data block, therefore, it is necessary to obtain all data blocks of the destination object, and one by one
Malfunction elimination is carried out, so as to determine fault data block and normal data block from all data blocks of the destination object.
S204, can judgement lock to the path where each fault data block, will each event if can lock
Path where barrier data block is locked successively according to preset order；, will each event using predetermined number normal data block
Barrier data block reverts to normal data block corresponding with the fault data block, to corresponding to the path where each fault data block
Server sends normal data block corresponding with the fault data block.
It should be noted that if the path of fault data block is locked by some server, the only server can
The content of the fault data block is read, i.e., other servers in addition to the server can not be all had access in fault data block
Hold, if the path where hindering data block for some reason is locked by other servers, and cause the server can not read failure data
The content of block, then the server can not also be repaired to the content of the fault data block.Therefore, locking can prevent difference
Server is repaired to the fault data block of same object simultaneously, further, by the road where each fault data block
Footpath is locked successively according to preset order, can prevent different faults data block of the different server simultaneously to same object
The path at place is locked, thus, it is possible to realize all fault data block institutes completed by a server to destination object
Path locked.
For example, the data block that the numbering of destination object is the 0th, 3,6, No. 8 breaks down respectively.If server A is to No. 0
Locked with the path where 6 number blocks, and server B locks to the path where No. 3 and 8 number blocks, then
Both sides can not lock to the path where whole fault data blocks of the destination object.But if according to default suitable
Path locks where ordered pair fault data block, and the above situation would not occur.It is assumed that according to data block numbering from small to large suitable
Path where ordered pair fault data block locks, if server A locks to the path where 0 number block, then clothes
Being engaged in device B can not be to path locking where 0 number block, and therefore, server B abandons all fault data block institutes to the object
Path lock, the path where all fault data blocks of the destination object is locked as server A.
After server locks in the path where each fault data block to destination object, the target pair can be utilized
The predetermined number normal data block of elephant, each fault data block is reverted into normal data corresponding with the fault data block
Block, and then, send normal number corresponding with the fault data block to server corresponding to the path where each fault data block
According to block, the repair process of completion fault data block.
It should be noted that if the legacy data of destination object is divided into k according to block, and utilize this k evidence
Block carries out computing with companion matrix, obtains r check block, then destination object shares k+r data block, is implemented using the present invention
Example, can directly utilize k normal data block, disposably revert to each fault data block corresponding with the fault data block
Normal data block, repaired every time for a fault data block will read k normal data block compared to prior art, application
The embodiment of the present invention, network overhead and time overhead caused by repeatedly reading k normal data block are saved, so as to improve
The performance of data recovery, the resource of distributed memory system is saved so that distributed memory system can be preferably client
The access at end provides service.
It can be seen that using the embodiment of the present invention, the path of all fault data blocks of one object is entered by a server
Row locks, then other servers hinder data block path and are locked and can not access the fault data block for some reason, therefore can only be by locking
The server in the fault data block path, data recovery is carried out to all fault data blocks of the object, avoids multiple services
Device carries out data recovery to same fault data block simultaneously, has saved Internet resources and computing resource.
Fault detect number is carried out to the data block of destination object in order to reduce server, to reduce the net of cloud storage system
The waste of network resource and computing resource, in a kind of specific embodiment, if failure is not present in detection adjacent data blocks
Data block, or judge that the path where each fault data block can not be locked, this method also includes：
Any object in addition to the destination object being stored on the server is obtained, as destination object, return is held
Row S201.
It is understood that multiple objects are stored with every server in distributed memory system, for every clothes
Business device, when the server detects, and fault data block is not present in adjacent data blocks, the server is considered as the institute of destination object
Have and fault data block is not present in data block, then need not carry out fault detect to other data blocks of the destination object, can be with
Any object in addition to the destination object being stored on the server is directly obtained, as destination object.For example, server
In be stored with object A, object B data block, when in the adjacent data blocks of server detected target object A target data block not
Fault data block be present, then any object in addition to destination object A, i.e. object B can be returned and performed as destination object
S201。
Or when server judges that the path where each fault data block can not be locked, then show the clothes
Being engaged in device can not be to each fault data block progress data recovery of the destination object, therefore the server need not be to the destination object
Other data blocks carry out fault detect, any in addition to the destination object being stored on the server can be directly obtained
Object, as destination object.
As can be seen that if fault data block is not present in detection adjacent data blocks, the server is considered as destination object
All data blocks in fault data block is not present, then need not carry out fault detect to other data blocks of the destination object,
Therefore reduce the number that server carries out fault detect to the data block of the destination object, so as to reduce cloud storage system
The waste of Internet resources and computing resource；
Or, if it is decided that the path where each fault data block can not be locked, then show the server without
Method carries out data recovery to each fault data block of the destination object, therefore the server need not be to other numbers of destination object
Fault detect is carried out according to block, therefore reduces the number that server carries out fault detect to the data block of the destination object, so as to subtract
The Internet resources of cloud storage system and the waste of computing resource are lacked.
In order to avoid unnecessary resource occupying, network resource utilization is improved, this method also includes：
Path where each fault data block is unlocked.
Server corresponding to server to the path where each fault data block sends corresponding with the fault data block
After normal data block, the path where each fault data block is unlocked so that other clothes in addition to the server
Content after the fault data block recovery that business device can have access to, so as to avoid the server to resource occupying, carry
High network resource utilization.
In order to accelerate to obtain the speed of adjacent data blocks, the step of obtaining two adjacent data blocks of the target data block,
Following two step can be included：
The first step, using the Hash ring being stored on the server, obtain the path where each adjacent data blocks；
Second step, using the path where each adjacent data blocks, it is determined that storing each service of each adjacent data blocks
Device, and each server is based on, obtain each adjacent data blocks.
In practical application, based on each server, obtaining each adjacent data blocks can be：Send and use to each server
In the request for obtaining each adjacent data blocks, so as to obtain each server and be directed to each adjacent data blocks that the request is sent.
The path where each adjacent data blocks can be quickly obtained due to server by utilizing Hash ring, thus, accelerate
Obtain the speed of adjacent data blocks.
Specifically, in order to prevent different server while the path where the different faults data block of same object be entered
Row locks, and causes the waste of computing resource, each fault data block includes unique number, and can judgement to each number of faults
The step of being locked according to the path where block, the following steps can be included：
Step A, checks whether the path where numbering minimum or the largest number of fault data block has been locked；
Step B, if be locked, the difference of initial time that the path is locked and current time is obtained, is judged
Whether the difference is more than predetermined threshold value, if it is greater, then judging that the path where each fault data block can be locked；Such as
Fruit is less than or equal to, then judges that the path where each fault data block can not be locked；
It is understood that being directed to every server, if the server judges that difference is more than predetermined threshold value, show this
The duration that path is locked by other servers exceedes effective time, and the path can be locked again by the server, according to above
Described, the path where fault data block is locked successively according to preset order, therefore, when numbering is minimum or numbering is maximum
Fault data block where the duration that is locked of path exceed effective time, it is believed that other fault data blocks being locked
The duration being locked exceed effective time, therefore can be determined that can lock to the path where each fault data block；
If the server judges that difference is less than predetermined threshold value, show duration that the path is locked by other servers not less than effective
Duration, the path can not be locked again by the server, and the path where other each fault data blocks can only be also somebody's turn to do by locking
The server in path is locked, therefore can be determined that can not lock to the path where each fault data block.
Predetermined threshold value can be set according to user's request, and the embodiment of the present invention is not done to the concrete numerical value of predetermined threshold value
Limit.For example, predetermined threshold value can be：1st, 2,3,4,5 etc..
Step C, if be not locked, judgement can lock to the path where each fault data block.
As can be seen that if the numbering of destination object is minimum or the largest number of fault data block where path by
Server locks, and locks duration not less than predetermined threshold value, then other servers in addition to the server can not be to each
Path where fault data block is locked, it is thus possible to prevent difference event of the different server simultaneously to same object
Path where barrier data block is locked, and avoids the waste of computing resource.
Specifically, in order to guarantee as same server to the path where all fault data blocks of same object
Locked, what it is for inspection is the minimum fault data block situation of numbering, by the path where each fault data block according to pre-
If order locks successively, Ke Yiwei：According to the order of numbering from small to large, the path where each fault data block is entered successively
Row locks；
What it is for inspection is situation of the numbering maximum number of faults according to block, by the path where each fault data block according to pre-
If order locks successively, Ke Yiwei：According to the order of numbering from big to small, the path where each fault data block is entered successively
Row locks.
It is understood that for inspection be the minimum fault data block situation of numbering, according to numbering from small to large suitable
Sequence, the path where each fault data block is locked successively, so as to if server is to the minimum number of faults of numbering
Locked according to the path where block, then can only be by order of the server according to numbering from small to large, successively by each fault data
Path where block is locked；Similar, what it is for inspection is numbering maximum number of faults according to block situation, according to numbering from greatly to
Small order, the path where each fault data block is locked successively, so as to if server is to the largest number of
Path where fault data block locks, then can only be by order of the server according to numbering from big to small, successively will each event
Path where barrier data block is locked, so as to which ensure that can be faulty to the institute of same object by same server
Path where data block is locked.
The embodiment of the present invention is simply introduced below by an instantiation.
Searching method provided in an embodiment of the present invention is applied to some server in distributed memory system, such as Fig. 3
It is shown, by server B restoring running (Reconstructor) process 2 in distributed system 301, by the Reconstructor
Process 2 performs the data reconstruction method of the embodiment of the present invention, and specific Data Recovery Process is as shown in Figure 4.
The Reconstructor2 processes obtain the target data block of destination object, i.e. data block b, further, obtain mesh
Mark two adjacent nodes of the data block on Hash ring, due to two adjacent nodes correspond to respectively two of target data block it is adjacent
Data block, and then, obtain two adjacent data blocks of target data block, i.e. data block a and data block c, and detect two it is adjacent
It whether there is fault data block in data block；
If there is fault data block, then the path where record fault data block is into set set, and obtains the target
All data blocks of object, and then, fault data block and normal data block are determined from all data blocks of destination object, its
In, a fault data block is often determined, just recorded in the path where it in set set, and entered by the Reconstructor
Can journey 2 judge lock the path where each fault data block for recording in set set；
For example, determination data block a, data block c, data block d are fault data block, and data block a, data block c, number
Can the numbering according to block d be respectively 1,2,4, then specifically judge the mistake that locked to the path where each fault data block
Journey is as follows：
Check whether the path for the fault data block that minimum is numbered in set is locked；
If it is, obtaining the difference of initial time that the path is locked and current time, the difference is when locking
It is long, judge whether the locking duration is more than predetermined threshold value timeout, if it is greater, then judgement can be to each fault data block institute
Path locked, and according to numbering order from small to large, successively by the road where each fault data block in set
Footpath is locked, i.e., is successively locked the path where data block a, data block c, data block d；If less than or wait
In then judgement can not lock to the path where each fault data block.
After being locked successively to malfunctioning node, M normal data block is read by the Reconstructor processes 2, utilizes this
M normal data block carries out matrix computations, and data block a, data block c, data block d reverted into normal data block, and respectively to
Server sends A, C, D and data block a, data block c, data block d corresponding normal data block respectively, finally, by data block a,
Path where data block c, data block d is unlocked.
If there is no fault data block, then acquisition is stored in any right in addition to the destination object on the server
As destination object, returning to the step of performing the target data block for obtaining destination object.
It can be seen that using the embodiment of the present invention, the path of all fault data blocks of one object is entered by a server
Row locks, then other servers hinder data block path and are locked and can not access the fault data block for some reason, therefore can only be by locking
The server in the fault data block path, data recovery is carried out to all fault data blocks of the object, avoids multiple services
Device carries out data recovery to same fault data block simultaneously, has saved Internet resources and computing resource, further, service
Device only needs to read the normal data block once recovered needed for fault data block, disposably can just recover each fault data block
Into normal data block, repaired every time for a fault data block will read normal data block compared to prior art, application
The embodiment of the present invention, network overhead and time overhead caused by repeatedly reading normal data block are saved, so as to improve number
According to the performance of recovery, the resource of distributed memory system has been saved.
Corresponding with above-mentioned data reconstruction method embodiment, the embodiment of the present invention also provides a kind of Data Recapture Unit,
Applied to the server in distributed memory system.
Referring to Fig. 5, Fig. 5 is a kind of structural representation map device of Data Recapture Unit provided in an embodiment of the present invention, for
Every server in distributed memory system, device include：
First obtains module 501, for obtaining the target data block of destination object, wherein, the target data block is：The mesh
Mark the data block being stored in all data blocks of object on the server；
Second obtains module 502, for obtaining two adjacent data blocks of the target data block, and detects two consecutive numbers
According to whether there is fault data block in block；
3rd obtains module 503, for when the judged result of the second acquisition module 502 is is, obtaining the target pair
All data blocks of elephant, and fault data block and normal data block are determined from all data blocks of the destination object；
Judge module 504, for judging that the path where each fault data block can be locked, if can add
Lock, then locked in the path where each fault data block according to preset order successively；Utilize predetermined number normal number
According to block, each fault data block is reverted into normal data block corresponding with the fault data block, to each fault data block institute
Path corresponding to server send corresponding with fault data block normal data block.
It can be seen that when carrying out data recovery using Data Recapture Unit provided in an embodiment of the present invention, by a server pair
The path of all fault data blocks of one object is locked, then other servers hinder data block path and are locked and nothing for some reason
Method accesses the fault data block, therefore can only be by locking the server in the fault data block path, to all number of faults of the object
Data recovery is carried out according to block, multiple servers is avoided and data recovery is carried out to same fault data block simultaneously, saved
Internet resources and computing resource.
Optionally, the second acquisition module 502, including：
First obtains submodule, for using the Hash ring being stored on the server, obtaining each adjacent data blocks institute
Path；
Second obtains submodule, for the path where each adjacent data blocks of utilization, it is determined that storing each adjacent data
Each server of block, and each server is based on, obtain each adjacent data blocks.
Optionally, each fault data block includes unique number, the judge module 504, including：
Detection sub-module, for check that numbering is minimum or the largest number of fault data block where path whether by
Lock；
First decision sub-module, for when the testing result of the detection sub-module is is, obtaining what the path was locked
The difference of initial time and current time, judges whether the difference is more than predetermined threshold value, if it is greater, then judgement can be to each event
Path where barrier data block is locked；If less than or be equal to, then judge can not be to where each fault data block
Path is locked；
Second decision sub-module, for when the testing result of the detection sub-module is no, judgement can be to each number of faults
Locked according to the path where block.
Wherein, for inspection it is the minimum fault data block situation of numbering, the judge module 504, including：
First locks submodule, for the order according to numbering from small to large, successively by where each fault data block
Path is locked；
What it is for inspection is situation of the numbering maximum number of faults according to block, the judge module 504, including：
Second locks submodule, for the order according to numbering from big to small, successively by where each fault data block
Path is locked.
Referring to Fig. 6, the structural representation for another Data Recapture Unit that Fig. 6 is provided by the embodiment of the present invention；This hair
Bright Fig. 6 embodiments increase unlocked state 505 on the basis of embodiment illustrated in fig. 5；
Unlocked state 505, for the path where each fault data block to be unlocked.
Referring to Fig. 7, the structural representation for another Data Recapture Unit that Fig. 7 is provided by the embodiment of the present invention；This hair
For bright Fig. 7 embodiments on the basis of embodiment illustrated in fig. 5, increase the 4th obtains module 506；
4th obtain module 506, for this second obtain module 502 judged result for it is no when, or, the judgement mould
When the judged result of block 504 is no, any object in addition to the destination object being stored on the server is obtained, as mesh
Object is marked, returns to the step of performing the target data block for obtaining the destination object being stored on the server.
The embodiment of the present invention additionally provides a kind of electronic equipment, as shown in figure 8, including processor 801, communication interface 802,
Memory 803 and communication bus 804, wherein, processor 801, communication interface 802, memory 803 is complete by communication bus 804
Into mutual communication,
Memory 803, for depositing computer program；
Processor 801, during for performing the program deposited on memory 803, realize what the embodiment of the present invention was provided
Data reconstruction method, specifically, the data reconstruction method comprises the following steps：
The target data block of destination object is obtained, wherein, the target data block is：In all data blocks of the destination object
The data block being stored on the server；
Two adjacent data blocks of the target data block are obtained, and detects in two adjacent data blocks and whether there is number of faults
According to block；
If there is fault data block, all data blocks of the destination object are obtained, and from all numbers of the destination object
According to determination fault data block and normal data block in block；
Can judgement lock to the path where each fault data block, if can lock, by each number of faults
Locked successively according to preset order according to the path where block；Using predetermined number normal data block, by each number of faults
Normal data block corresponding with the fault data block is reverted to according to block, to service corresponding to the path where each fault data block
Device sends normal data block corresponding with the fault data block.
Optionally, if there is no fault data block, or judge that the path where each fault data block can not be entered
Row locks, and this method also includes：
Any object in addition to the destination object being stored on the server is obtained, as destination object, return is held
Row obtains the step of target data block for the destination object being stored on the server.
Optionally, this method also includes：
Path where each fault data block is unlocked.
Optionally, the step of obtaining two adjacent data blocks of the target data block, including：
Using the Hash ring being stored on the server, the path where each adjacent data blocks is obtained；
Using the path where each adjacent data blocks, it is determined that each server of each adjacent data blocks is stored, and base
In each server, each adjacent data blocks are obtained.
Optionally, each fault data block includes unique number, and can judgement to the path where each fault data block
Locked, including：
Check whether the path where numbering minimum or the largest number of fault data block has been locked；
If be locked, the difference of initial time that the path is locked and current time is obtained, judges the difference
Whether predetermined threshold value is more than, if it is greater, then judging that the path where each fault data block can be locked；If less than
Or be equal to, then judge that the path where each fault data block can not be locked；
If be not locked, judgement can lock to the path where each fault data block.
Optionally, for inspection it is the minimum fault data block situation of numbering, by where each fault data block
Path locks successively according to preset order, including：
According to the order of numbering from small to large, the path where each fault data block is locked successively；
What it is for inspection is situation of the numbering maximum number of faults according to block, and the path by where each fault data block is pressed
Locked successively according to preset order, including：
According to the order of numbering from big to small, the path where each fault data block is locked successively.
The communication bus that above-mentioned electronic equipment is mentioned can be Peripheral Component Interconnect standard (Peripheral Component
Interconnect, abbreviation PCI) bus or EISA (Extended Industry Standard
Architecture, abbreviation EISA) bus etc..The communication bus can be divided into address bus, data/address bus, controlling bus etc..
For ease of representing, only represented in figure with a thick line, it is not intended that an only bus or a type of bus.
The communication that communication interface is used between above-mentioned electronic equipment and other equipment.
Memory can include random access memory (Random Access Memory, abbreviation RAM), can also include
Nonvolatile memory (non-volatile memory), for example, at least a magnetic disk storage.Optionally, memory may be used also
To be at least one storage device for being located remotely from aforementioned processor.
Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit,
Abbreviation CPU), network processing unit (Ne twork Processor, abbreviation NP) etc.；It can also be digital signal processor
(Digital Signal Processing, abbreviation DSP), application specific integrated circuit (Applica tion Specific
Integrated Circuit, abbreviation ASIC), field programmable gate array (Field-Programmable Gate Array,
Abbreviation FPGA) either other PLDs, discrete gate or transistor logic, discrete hardware components.
At the another aspect that the present invention is implemented, a kind of computer-readable recording medium is additionally provided, it is described computer-readable
Instruction is stored with storage medium, when run on a computer so that computer performs above-mentioned data reconstruction method.
At the another aspect that the present invention is implemented, the embodiment of the present invention additionally provides a kind of computer program production comprising instruction
Product, when run on a computer so that computer performs above-mentioned data reconstruction method.
In the above-described embodiments, can come wholly or partly by software, hardware, firmware or its any combination real
It is existing.When implemented in software, can realize in the form of a computer program product whole or in part.The computer program
Product includes one or more computer instructions.When loading on computers and performing the computer program instructions, all or
Partly produce according to the flow or function described in the embodiment of the present invention.The computer can be all-purpose computer, special meter
Calculation machine, computer network or other programmable devices.The computer instruction can be stored in computer-readable recording medium
In, or the transmission from a computer-readable recording medium to another computer-readable recording medium, for example, the computer
Instruction can pass through wired (such as coaxial cable, optical fiber, numeral from a web-site, computer, server or data center
User's line (DSL)) or wireless (such as infrared, wireless, microwave etc.) mode to another web-site, computer, server or
Data center is transmitted.The computer-readable recording medium can be any usable medium that computer can access or
It is the data storage devices such as server, the data center integrated comprising one or more usable mediums.The usable medium can be with
It is magnetic medium, (for example, floppy disk, hard disk, tape), optical medium (for example, DVD) or semiconductor medium (such as solid state hard disc
Solid State Disk (SSD)) etc..
It should be noted that herein, such as first and second or the like relational terms are used merely to a reality
Body or operation make a distinction with another entity or operation, and not necessarily require or imply and deposited between these entities or operation
In any this actual relation or order.Moreover, term " comprising ", "comprising" or its any other variant are intended to
Nonexcludability includes, so that process, method, article or equipment including a series of elements not only will including those
Element, but also the other element including being not expressly set out, or it is this process, method, article or equipment also to include
Intrinsic key element.In the absence of more restrictions, the key element limited by sentence "including a ...", it is not excluded that
Other identical element also be present in process, method, article or equipment including the key element.
Each embodiment in this specification is described by the way of related, identical similar portion between each embodiment
Divide mutually referring to what each embodiment stressed is the difference with other embodiment.Especially for device,
For electronic equipment, computer-readable recording medium, computer program product embodiments, implement because it is substantially similar to method
Example, so description is fairly simple, the relevent part can refer to the partial explaination of embodiments of method.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all
Any modification, equivalent substitution and improvements made within the spirit and principles in the present invention etc., are all contained in protection scope of the present invention
It is interior.
Claims (13)
1. a kind of data reconstruction method, it is characterised in that applied to the server in distributed memory system, for described in every
Server, methods described include：
The target data block of destination object is obtained, wherein, the target data block is：In all data blocks of the destination object
The data block being stored on the server；
Two adjacent data blocks of the target data block are obtained, and detects and whether there is failure in described two adjacent data blocks
Data block；
If there is fault data block, all data blocks of the destination object are obtained, and from all numbers of the destination object
According to determination fault data block and normal data block in block；
Can judgement lock to the path where each fault data block, will each event if can lock
Path where barrier data block is locked successively according to preset order；, will be each using the predetermined number normal data block
The individual fault data block reverts to normal data block corresponding with the fault data block, to where each fault data block
Path corresponding to server send corresponding with fault data block normal data block.
2. according to the method for claim 1, it is characterised in that if there is no fault data block, or judgement can not be right
Path where each fault data block is locked, and methods described also includes：
Any object in addition to the destination object being stored on the server is obtained, as destination object, returns and performs
The step of obtaining the target data block for the destination object being stored on the server.
3. according to the method for claim 1, it is characterised in that methods described also includes：
Path where each fault data block is unlocked.
4. according to the method for claim 1, it is characterised in that two adjacent datas for obtaining the target data block
The step of block, including：
Using the Hash ring being stored on the server, the path where each adjacent data blocks is obtained；
Using the path where each adjacent data blocks, it is determined that each server of each adjacent data blocks is stored,
And each server is based on, obtain each adjacent data blocks.
5. according to the method for claim 1, it is characterised in that each fault data block includes unique number, described
Can judgement lock to the path where each fault data block, including：
Check whether the path where numbering minimum or the largest number of fault data block has been locked；
If be locked, the difference of initial time that the path is locked and current time is obtained, judges that the difference is
It is no to be more than predetermined threshold value, if it is greater, then judging that the path where each fault data block can be locked；It is if small
In or be equal to, then judge the path where each fault data block can not be locked；
If be not locked, judgement can lock to the path where each fault data block.
6. according to the method for claim 5, it is characterised in that what it is for inspection is the minimum fault data block situation of numbering,
The path by where each fault data block locks successively according to preset order, including：
According to the order of the numbering from small to large, the path where each fault data block is locked successively；
What it is for inspection is situation of the numbering maximum number of faults according to block, and the path by where each fault data block is pressed
Locked successively according to preset order, including：
According to the order of the numbering from big to small, the path where each fault data block is locked successively.
7. a kind of Data Recapture Unit, it is characterised in that applied to the server in distributed memory system, for described in every
Server, described device include：
First obtains module, for obtaining the target data block of destination object, wherein, the target data block is：The target
The data block being stored in all data blocks of object on the server；
Second obtains module, for obtaining two adjacent data blocks of the target data block, and detects described two consecutive numbers
According to whether there is fault data block in block；
3rd obtains module, for when the judged result of the described second acquisition module is is, obtaining the institute of the destination object
There is data block, and fault data block and normal data block are determined from all data blocks of the destination object；
Judge module, for judging that the path where each fault data block can be locked, if can lock,
Path where each fault data block is locked successively according to preset order；It is described normal using predetermined number
Data block, each fault data block is reverted into normal data block corresponding with the fault data block, to each event
Server corresponding to path where barrier data block sends normal data block corresponding with the fault data block.
8. device according to claim 7, it is characterised in that described device also includes：
4th obtains module, when the judged result for obtaining module described second is no, or, the judge module is sentenced
Disconnected result for it is no when, obtain any object in addition to the destination object being stored on the server, as destination object,
Return to the step of performing the target data block for obtaining the destination object being stored on the server.
9. device according to claim 7, it is characterised in that described device also includes：
Unlocked state, for the path where each fault data block to be unlocked.
10. device according to claim 7, it is characterised in that described second obtains module, including：
First obtains submodule, for using the Hash ring being stored on the server, obtaining each adjacent data blocks institute
Path；
Second obtains submodule, for the path where each adjacent data blocks of utilization, it is determined that storage is each described adjacent
Each server of data block, and each server is based on, obtain each adjacent data blocks.
11. device according to claim 7, it is characterised in that each fault data block includes unique number, described
Judge module, including：
Detection sub-module, for checking whether the path where numbering minimum or the largest number of fault data block has been added
Lock；
First decision sub-module, for when the testing result of the detection sub-module is is, obtaining that the path is locked
The difference for time and the current time of beginning, judges whether the difference is more than predetermined threshold value, if it is greater, then judgement can be to each institute
Path where stating fault data block is locked；If less than or be equal to, then judge can not be to each fault data
Path where block is locked；
Second decision sub-module, for when the testing result of the detection sub-module is no, judgement can be to each failure
Path where data block is locked.
12. device according to claim 11, it is characterised in that what it is for inspection is the minimum fault data block feelings of numbering
Condition, the judge module, including：
First locks submodule, for the order according to the numbering from small to large, successively by each fault data block institute
Path locked；
What it is for inspection is situation of the numbering maximum number of faults according to block, the judge module, including：
Second locks submodule, for the order according to the numbering from big to small, successively by each fault data block institute
Path locked.
13. a kind of electronic equipment, it is characterised in that including processor, communication interface, memory and communication bus, wherein, processing
Device, communication interface, memory complete mutual communication by communication bus；
Memory, for depositing computer program；
Processor, during for performing the program deposited on memory, realize any described method and steps of claim 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710446718.7A CN107346273B (en) | 2017-06-14 | 2017-06-14 | Data recovery method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710446718.7A CN107346273B (en) | 2017-06-14 | 2017-06-14 | Data recovery method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107346273A true CN107346273A (en) | 2017-11-14 |
| CN107346273B CN107346273B (en) | 2020-09-04 |
Family
ID=60254494
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710446718.7A Active CN107346273B (en) | 2017-06-14 | 2017-06-14 | Data recovery method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107346273B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111104283A (en) * | 2019-11-29 | 2020-05-05 | 浪潮电子信息产业股份有限公司 | Fault detection method, device, equipment and medium of distributed storage system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101276302A (en) * | 2007-03-29 | 2008-10-01 | 中国科学院计算技术研究所 | Magnetic disc fault processing and data restructuring method in magnetic disc array system |
| CN101561773A (en) * | 2009-06-03 | 2009-10-21 | 成都市华为赛门铁克科技有限公司 | Method for recovering disk data and device thereof |
| CN102156928A (en) * | 2011-04-29 | 2011-08-17 | 浪潮通信信息系统有限公司 | Method for system concurrency control through business logic lock |
-
2017
- 2017-06-14 CN CN201710446718.7A patent/CN107346273B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101276302A (en) * | 2007-03-29 | 2008-10-01 | 中国科学院计算技术研究所 | Magnetic disc fault processing and data restructuring method in magnetic disc array system |
| CN101561773A (en) * | 2009-06-03 | 2009-10-21 | 成都市华为赛门铁克科技有限公司 | Method for recovering disk data and device thereof |
| CN102156928A (en) * | 2011-04-29 | 2011-08-17 | 浪潮通信信息系统有限公司 | Method for system concurrency control through business logic lock |
Non-Patent Citations (1)
| Title |
|---|
| 孔华锋: "NAS设备卷管理模块中失效数据恢复问题研究", 《小型微型计算机系统》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111104283A (en) * | 2019-11-29 | 2020-05-05 | 浪潮电子信息产业股份有限公司 | Fault detection method, device, equipment and medium of distributed storage system |
| CN111104283B (en) * | 2019-11-29 | 2022-04-22 | 浪潮电子信息产业股份有限公司 | Fault detection method, device, equipment and medium of distributed storage system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107346273B (en) | 2020-09-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109347787B (en) | Identity information identification method and device | |
| US11303626B2 (en) | Secure token passing via hash chains | |
| CN108062202A (en) | A kind of file block storage method and system | |
| US11210003B2 (en) | Method, device and computer program product for restoring data based on replacing child node identifiers with parent node identifier | |
| CN108108127B (en) | File reading method and system | |
| CN107368507A (en) | A kind of common recognition method and device based on block chain | |
| CN111222176B (en) | Block chain-based cloud storage possession proving method, system and medium | |
| CN110505228B (en) | Edge cloud architecture-based big data processing method, system, medium and device | |
| CN107864676A (en) | System and method for detecting unknown leak in calculating process | |
| CN111625400B (en) | Data redundancy processing method, device and storage medium | |
| CN104346365A (en) | Device and method for determining specific service associated logs | |
| CN110855458A (en) | Configuration command generation method and equipment | |
| US8589894B2 (en) | Self-monitoring object-oriented applications | |
| CN107844542A (en) | A kind of distributed document storage method and device | |
| CN112905542A (en) | File processing method, device, server, equipment and storage medium | |
| CN115408247A (en) | Threat behavior analysis method based on big data and server | |
| CN107346273A (en) | A kind of data reconstruction method, device and electronic equipment | |
| CN113407374A (en) | Fault processing method and device, fault processing equipment and storage medium | |
| US8407175B2 (en) | Method, apparatus and product for SAT solving using templates clauses | |
| CN116938523A (en) | Blockchain system, operating method, electronic device and storage medium | |
| Yajnik et al. | Graceful degradation in algorithm-based fault tolerant multiprocessor systems | |
| CN116361153A (en) | Method and device for testing firmware codes, electronic equipment and storage medium | |
| US20170116073A1 (en) | Detecting single event upsets and stuck-at faults in ram-based data path controllers | |
| CN110166295B (en) | Method for judging whether network topology supports Byzantine fault tolerance or not | |
| US11163642B2 (en) | Methods, devices and computer readable medium for managing a redundant array of independent disks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |