CN103186678B - Data recovery method, device and system - Google Patents
Data recovery method, device and system Download PDFInfo
- Publication number
- CN103186678B CN103186678B CN201310144430.6A CN201310144430A CN103186678B CN 103186678 B CN103186678 B CN 103186678B CN 201310144430 A CN201310144430 A CN 201310144430A CN 103186678 B CN103186678 B CN 103186678B
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- data server
- server
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention discloses a kind of data recovery method, device and system, the method includes: send things number to data server；Receiving the storage result of data server feedback, wherein, the storage result of data server feedback is for representing this data server storage condition to things number；Judge that abnormal data server occurs in storage according to storage result, and the data of this data server are repaired by the data stored according to other data servers.The present invention is by judging abnormal data server occur, and the data of this abnormal data server are repaired by the data stored according to other data servers, can know that abnormal data server occurs exactly by the fewest signaling consumption, avoid meta data server is accessed in a large number, and data consistency can be effectively improved.
Description
Technical field
The present invention relates to computer realm, in particular it relates to a kind of data recovery method, device and system.
Background technology
In current distributed file system framework, including the file system of two kinds of frameworks:
(1) band internal schema (In-band Mode): file system is metadata and data mix and deposit
Storage；
(2) band external schema (Out-band Mode): file system is metadata and data Separate Storage.
Specifically, the storage capacity of the file system of band internal schema is limited by disk size, causes it to gulp down
Ability of telling is limited by magnetic disc i/o and network I/O, is accordingly difficult to meet the needs of application at present.
Application server is directly connected by the file system with external schema with storage device, it is possible to increase data
Transmittability and decrease the time delay of data；Further, owing to only transmitting the first number about fileinfo
According to time just process through meta data server (MDS), it is possible to effectively reduce the centre of data transmission
Link, improves efficiency of transmission, alleviates the load of meta data server.
For the outer file system of current existing band, in order to ensure that the reliability of wherein data division is with consistent
Property, generally use Raid (Redundant Array of Inexpensive Disc) technology.The conventional technology used includes Raid-0
Technology, Raid-5 technology and Raid-6 technology, and Erasure Code (correcting and eleting codes) can be used further
Technology makes system in having a certain degree of Redundant process, reaches the purpose of High Availabitity.Above-mentioned redundancy technique can
To be considered data server (DS) is repartitioned band, therefore, present band external schema
The model of distributed file system is as shown in Figure 1.Seeing Fig. 1, this system includes client, multiple units number
According to server (MDS) and multi-group data server.
At present, mainly data (i.e. data described in literary composition are recovered by following two mode under band external schema
Repair):
The data message of inconsistent node, by meta data server, is saved in metadata by (mode one)
On server, when node occurs abnormal, restored data to unanimously by meta data server；
(mode two), by self safeguarding the information of DS, when DS occurs abnormal, is counted by DS self
Calculate or what other means restores data to unanimously.
But, aforesaid way makes system substantially increase the visit capacity of meta data server for a moment, impact unit number
According to operation.And second aforesaid way exists the problem being difficult to find that consistent point, affect the correct of data recovery
Property, meanwhile, the data recovery operation of aforesaid way two can not be carried out in time, and client can be caused to read
The data of mistake.
, data big to meta data server visit capacity for data recovery scenario in correlation technique are repaired inaccurate
Problem, effective solution is the most not yet proposed.
Summary of the invention
, data big to meta data server visit capacity for data recovery scenario in correlation technique are repaired inaccurate
Problem, the present invention proposes a kind of data recovery method, device and system, it is possible to avoid Metadata Service
Device accesses in a large number, and can be effectively improved data consistency.
The technical scheme is that and be achieved in that:
According to an aspect of the invention, it is provided a kind of data recovery method.
This data recovery method includes:
Things number is sent to data server；
Receiving the storage result of data server feedback, wherein, the storage result of data server feedback is used for
Represent this data server storage condition to things number；
Judge that abnormal data server occurs in storage according to storage result, and deposit according to other data servers
The data of this data server are repaired by the data of storage.
Additionally, this data recovery method farther includes:
The things number being properly received as storage result and is fed back this storage result by data server.
Wherein, judge that storage occurs that abnormal data server includes according to storage result:
If the things number of a data server feedback is less than sending the things number to this data server before,
Then determine that the storage of this data server occurs abnormal.
And, carry out repairing bag to the data of this data server according to the data that other data servers store
Include:
Obtain from other data servers and need the data of storage, according to right corresponding to this in the data of this acquisition
In this storage, the data that things number that abnormal data server does not feeds back is corresponding should occur, this is stored out
The data of now abnormal data server are repaired.
Further, the data of this data server are carried out repairing bag by the data stored according to other data servers
Include:
Obtain from other servers and need the data of storage, and according to the data of this acquisition, this storage is occurred different
The data of normal data server are repaired.
According to another aspect of the present invention, it is provided that a kind of data prosthetic device.
This data prosthetic device includes:
Sending module, for sending things number to data server；
Receiver module, for receiving the storage result of data server feedback, wherein, data server feeds back
Storage result for representing this data server storage condition to things number；
According to storage result, repair module, for judging that abnormal data server occurs in storage, and according to it
The data of this data server are repaired at the data of data server storage by he.
Wherein, the feedback result that receiver module receives is the things number that data server is properly received.
And, repair module sent to these data before being less than in the things number of a data server feedback
In the case of the things number of server, determine that the storage of this data server occurs abnormal.
Further, repair module needs the data of storage, according to this specifically for obtaining from other data servers
The data obtained occur, corresponding to this storage, the things number that abnormal data server does not feeds back corresponding to this
Corresponding data, occur that to this storage the data of abnormal data server are repaired.
According to a further aspect of the invention, it is provided that a kind of data repair system.
This data repair system includes meta data server, multiple data server and according to above-mentioned data
Prosthetic device, wherein, data prosthetic device is arranged in appointment data server, wherein, it is intended that data take
Business device is specified by meta data server.
The data server that the present invention is abnormal by judging appearance, and store according to other data servers
The data of this abnormal data server are repaired by data, it is possible to accurate by the fewest signaling consumption
Really know that abnormal data server occurs, it is to avoid meta data server is accessed in a large number, and energy
Enough it is effectively improved data consistency.
Accompanying drawing explanation
In order to be illustrated more clearly that the embodiment of the present invention or technical scheme of the prior art, below will be to enforcement
In example, the required accompanying drawing used is briefly described, it should be apparent that, the accompanying drawing in describing below is only
Some embodiments of the present invention, for those of ordinary skill in the art, are not paying creative work
Under premise, it is also possible to obtain other accompanying drawing according to these accompanying drawings.
Fig. 1 is the model framework chart of the distributed file system of band external schema in prior art；
Fig. 2 is the flow chart of data recovery method according to embodiments of the present invention；
Fig. 3 is the block diagram of data prosthetic device according to embodiments of the present invention；
Fig. 4 is the signal of meta data server side operation in data repair process according to embodiments of the present invention
Figure；
Fig. 5 and Fig. 6 is the schematic diagram of client operation in data repair process according to embodiments of the present invention；
Fig. 7 is the schematic diagram of data server side operation in data recovery scenario according to embodiments of the present invention；
Fig. 8 is the signal of virtual server reselection operation in data recovery scenario according to embodiments of the present invention
Figure.
Detailed description of the invention
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clearly
Chu, be fully described by, it is clear that described embodiment be only a part of embodiment of the present invention rather than
Whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art obtained all its
His embodiment, broadly falls into the scope of protection of the invention.
According to embodiments of the invention, it is provided that a kind of data recovery method.
As in figure 2 it is shown, data recovery method according to embodiments of the present invention includes:
Step S201, sends things number to data server；
Step S203, receives the storage result of data server feedback, wherein, data server feedback
Storage result is for representing this data server storage condition to things number；
According to storage result, step S205, judges that abnormal data server occurs in storage, and according to other
The data of this data server are repaired by the data of data server storage.
Additionally, this data recovery method farther includes:
The things number being properly received as storage result and is fed back this storage result by data server.
Wherein, judge that storage occurs that abnormal data server includes according to storage result:
If the things number of a data server feedback is less than sending the things number to this data server before,
Then determine that the storage of this data server occurs abnormal.
And, carry out repairing bag to the data of this data server according to the data that other data servers store
Include:
Obtain from other data servers and need the data of storage, according to right corresponding to this in the data of this acquisition
In this storage, the data that things number that abnormal data server does not feeds back is corresponding should occur, this is stored out
The data of now abnormal data server are repaired.
Further, the data of this data server are carried out repairing bag by the data stored according to other data servers
Include:
Obtain from other servers and need the data of storage, and according to the data of this acquisition, this storage is occurred different
The data of normal data server are repaired.
According to embodiments of the invention, additionally provide a kind of data prosthetic device.
As it is shown on figure 3, this data prosthetic device includes:
Sending module 31, for sending things number to data server；
Receiver module 32, for receiving the storage result of data server feedback, wherein, data server
The storage result of feedback is for representing this data server storage condition to things number；
According to storage result, repair module 33, for judging that abnormal data server, and root occurs in storage
The data of this data server are repaired by the data stored according to other data servers.
Wherein, the feedback result that receiver module 32 receives is the things number that data server is properly received.
And, repair module 33 sent to being somebody's turn to do before being less than in the things number of a data server feedback
In the case of the things number of data server, determine that the storage of this data server occurs abnormal.
Further, repair module 33 needs the data of storage, root specifically for obtaining from other data servers
Corresponding to this storage, the thing that abnormal data server does not feeds back occurs corresponding to this according in the data of this acquisition
The data that thing number is corresponding, occur that to this storage the data of abnormal data server are repaired.
According to embodiments of the invention, additionally provide a kind of data repair system.
This data repair system includes meta data server, multiple data server and according to above-mentioned data
Prosthetic device, wherein, data prosthetic device can be arranged in appointment data server, wherein, it is intended that number
Specify by meta data server according to server.In other embodiments of the invention, data prosthetic device is permissible
Outside being arranged on meta data server and data server or be arranged in meta data server.
To arrange by data server below and be described as a example by above-mentioned data prosthetic device.
In the implementation process of the present invention, in order to reduce redundancy during recovery as far as possible, by data, services
Be divided into region by group, such as, Raid-6 pattern be 6 nodes that have of 4+2 be one group of data, that
, metadata has the such combination of how many kinds of, DS just has how many so corresponding subregions.Fig. 2
Also it is with each subregion (also referred herein as region or referred to as group) for single with the scheme described by Fig. 3
Position, describes the operating process for one group of data server.
For each region such by metadata choose a virtual server (Virtual Server,
Referred to as VS), record, on meta data server, when client is to DS write operation, can access VS,
Data are sent to other nodes (data server) by VS.Meanwhile, (data take the node in same group
Business device) all can inform oneself duty by timer access VS.If there is abnormal data clothes occur
Business device, the VS in this server place group is notified that other nodes (other data servers) recover this
Abnormal data server, has reached the purpose of data spontaneous recovery.When VS occurs abnormal, other node meetings
Notice meta data server, is re-elected new VS by meta data server in this group.
Write operation processing procedure includes the operation on meta data server, the operation of client, the behaviour of DS end
Work, the operation of VS end and the recovery of node.Below with reference to accompanying drawings these operations are described.
As shown in Figure 4, the operation of meta data server side includes:
Meta data server can be by data storage method, and it is different that all of node is pressed certain regular partition
Band, and choose the child node in a band as VS for certain band at random；
When, after the message receiving VS fault, judging whether to need weight according to the node state that self preserves
The newly selected VS, if the node of report message is normal, then need not reselect VS；Otherwise, again select
Select VS.After selecting VS success, all nodes being notified that in VS group are to change VS information.
Fig. 5 and Fig. 6 shows the operation of client.
Specifically, as it is shown in figure 5, VS is invisible to client, when client is to file operation, first
First remove to obtain on meta data server layout (data strip to be write)；
Then, as shown in Figure 6, the data of burst can be sent to the DS at layout place, if DS returns
The number returned exceedes and can guarantee that data successfully minimum number, then acknowledged client end, shows this time to operate successfully.
As it is shown in fig. 7, when carrying out data and repairing, the operation of DS side includes:
After DS receives the request message of client, generate a tid (things number the most herein)；
By operation lower wall corresponding for tid, write dish (write DS dish, returned by tid).Meanwhile, DS can determine
Time the tid in oneself handled VS group is sent to VS；
DS can receive the message (this message represents that current DS is the need of carrying out data reparation) from VS,
If desired (that is, current DS is in abnormal state and causes write operation failure), then this DS are repaired
Prepare to receive and recover data, and recover；If have received read data request (that is, to represent that this DS is in
Normal condition), then specify data read-out to be read VS, and be sent to the node specified by VS up；
After DS have received the data that other nodes send, the data of disappearance can be write on dish according to algorithm,
Thus complete data reparation (recovery).
In the present embodiment, VS is dummy node, its be positioned at MDS a DS group select a certain
DS.Function main for VS is to calculate which node data in real time to be broken (occurring abnormal), and notifies other
Node transmits data to specify node.If the data of a certain node received need not recover, then to
This node that need not recover sends the message of a partial data.Meanwhile, VS also can be by all VS groups
The tid being inside complete issues the node in group, allows node daily record be deleted, to reach Free up Memory
Purpose.All nodes in VS group regularly can send all tid of self VS group to VS, and VS is according to receipts
The tid of each VS group interior nodes arrived, it becomes possible to know that the data of certain node are the need of recovery.
As shown in Figure 8, when carrying out VS gravity treatment, operating process is as follows: if the node failure at VS place,
Then send the node in failed VS group and can send a message to meta data server, if weighing through several times
After examination, it is impossible to send successfully, the most do not do any action.If sending successfully, meta data server can be according to joint
Dotted state re-elects VS, and notifies all nodes in former VS group, hereafter, in remaining VS group
Node can send its data to the VS of new election and complete information, new VS node is also to original work
Mode of making equally is operated.
After abnormal nodes is recovered normally, meta data server is notified that all VS of this node place group,
Informing that abnormal nodes has been reached the standard grade, now, VS is according to the information received, and certain several node sends wherein
Read the data of certain affairs, and be sent in that node to be recovered.By consistent, all complete
Affairs, by all nodes in VS notice VS group, are deleted completing affairs.
In sum, by means of the technique scheme of the present invention, by judging that abnormal data, services occurs
Device, and the data of this abnormal data server are repaiied by the data stored according to other data servers
Multiple, it is possible to distributed file system (in a particular embodiment, can be band external schema) can be made to enter
The state being in work that can be the most fast in the case of the extra information record of row, it is to avoid to Metadata Service
Device accesses in a large number, and can be effectively improved data consistency.
The foregoing is only presently preferred embodiments of the present invention, not in order to limit the present invention, all at this
Within bright spirit and principle, any modification, equivalent substitution and improvement etc. made, should be included in this
Within bright protection domain.
Claims (8)
1. a data recovery method, it is characterised in that described method includes:
Things number is sent to data server；
Receive the storage result of described data server feedback, wherein, the storage of data server feedback
Result is for representing this data server storage condition to described things number；
Judge that abnormal data server occurs in storage according to described storage result, and according to other data
The data of this data server are repaired by the data of server storage；
Wherein, the things number being properly received as described storage result and is fed back this and deposits by data server
Storage result.
Data recovery method the most according to claim 1, it is characterised in that according to described storage
Result judges that storage occurs that abnormal data server includes:
If the things number of a data server feedback is less than sending the things to this data server before
Number, it is determined that the storage of this data server occurs abnormal.
Data recovery method the most according to claim 2, it is characterised in that according to other data
The data of server storage carry out reparation to the data of this data server and include:
Obtain from other data servers described and need the data of storage, according to right in the data of this acquisition
In storage, the data that things number that abnormal data server does not feeds back is corresponding should occur, to this storage
Occur that the data of abnormal data server are repaired.
Data recovery method the most according to claim 1, it is characterised in that according to other data
The data of server storage carry out reparation to the data of this data server and include:
Obtain from other servers described and need the data of storage, and according to the data of this acquisition, this is deposited
Storage occurs that the data of abnormal data server are repaired.
5. a data prosthetic device, it is characterised in that described device includes:
Sending module, for sending things number to data server；
Receiver module, for receiving the storage result of described data server feedback, wherein, data take
The storage result of business device feedback is for representing this data server storage condition to described things number；
Repair module, for judging that abnormal data server occurs in storage according to described storage result,
And the data of this data server are repaired by the data stored according to other data servers；
Wherein, the feedback result that described receiver module receives is the things number that data server is properly received.
Data prosthetic device the most according to claim 5, it is characterised in that described repair module
The things number to this data server was sent before being less than in the things number of a data server feedback
In the case of, determine that the storage of this data server occurs abnormal.
Data prosthetic device the most according to claim 6, it is characterised in that described repair module
The data of storage are needed, according to the data of this acquisition specifically for obtaining from other data servers described
In corresponding to storage, the data that things number that abnormal data server does not feeds back is corresponding occur, to this
Storage occurs that the data of abnormal data server are repaired.
8. a data repair system, it is characterised in that include that meta data server, multiple data take
Business device and according to the data prosthetic device according to any one of claim 5-7, wherein, described
Data prosthetic device is arranged in appointment data server, and wherein, described appointment data server is by institute
State meta data server to specify.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310144430.6A CN103186678B (en) | 2013-04-24 | 2013-04-24 | Data recovery method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310144430.6A CN103186678B (en) | 2013-04-24 | 2013-04-24 | Data recovery method, device and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103186678A CN103186678A (en) | 2013-07-03 |
| CN103186678B true CN103186678B (en) | 2016-09-14 |
Family
ID=48677845
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310144430.6A Active CN103186678B (en) | 2013-04-24 | 2013-04-24 | Data recovery method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103186678B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101079758A (en) * | 2007-07-02 | 2007-11-28 | 华为技术有限公司 | Data check method, device and system |
| CN101163010A (en) * | 2007-11-14 | 2008-04-16 | 华为软件技术有限公司 | Method of authenticating request message and related equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7143117B2 (en) * | 2003-09-25 | 2006-11-28 | International Business Machines Corporation | Method, system, and program for data synchronization by determining whether a first identifier for a portion of data at a first source and a second identifier for a portion of corresponding data at a second source match |
-
2013
- 2013-04-24 CN CN201310144430.6A patent/CN103186678B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101079758A (en) * | 2007-07-02 | 2007-11-28 | 华为技术有限公司 | Data check method, device and system |
| CN101163010A (en) * | 2007-11-14 | 2008-04-16 | 华为软件技术有限公司 | Method of authenticating request message and related equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103186678A (en) | 2013-07-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107908494A (en) | Processing method, device, electronic equipment and the storage medium of anomalous event | |
| CN100461125C (en) | Priority scheme for transmitting blocks of data | |
| CN106406758A (en) | Data processing method based on distributed storage system, and storage equipment | |
| CN105187249A (en) | Fault recovery method and device | |
| CN106844108B (en) | A kind of date storage method, server and storage system | |
| CN103354503A (en) | Cloud storage system capable of automatically detecting and replacing failure nodes and method thereof | |
| CN102088490B (en) | Data storage method, device and system | |
| EP1676272A2 (en) | Method of recovering data | |
| EP1678712A2 (en) | Methods of reading and writing data | |
| CN108173672B (en) | Method and device for detecting fault | |
| JP2008538624A (en) | Remote data mirroring system | |
| EP1678616A2 (en) | Methods of reading and writing data | |
| CN103535014B (en) | A kind of network store system, data processing method and client | |
| CN109308227A (en) | Fault detection control method and relevant device | |
| CN104579765A (en) | Disaster tolerance method and device for cluster system | |
| CN104935481A (en) | Data recovery method based on redundancy mechanism in distributed storage | |
| CN107368485A (en) | The management method and Database Systems of a kind of database | |
| CN110351313B (en) | Data caching method, device, equipment and storage medium | |
| CN105487609A (en) | Server | |
| CN106909307A (en) | A kind of method and device for managing dual-active storage array | |
| CN113326006B (en) | Distributed block storage system based on erasure codes | |
| US20230004465A1 (en) | Distributed database system and data disaster backup drilling method | |
| CN104158843B (en) | The storage-unit-failure detection method and device of distributed file storage system | |
| CN110209550A (en) | Fault handling method, device, electronic equipment and the storage medium of storage medium | |
| CN103186678B (en) | Data recovery method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220725 Address after: 100089 building 36, courtyard 8, Dongbeiwang West Road, Haidian District, Beijing Patentee after: Dawning Information Industry (Beijing) Co.,Ltd. Patentee after: DAWNING INFORMATION INDUSTRY Co.,Ltd. Address before: 100193 No. 36 Building, No. 8 Hospital, Wangxi Road, Haidian District, Beijing Patentee before: Dawning Information Industry (Beijing) Co.,Ltd. |