CN107506266A - A kind of data reconstruction method and system - Google Patents
A kind of data reconstruction method and system Download PDFInfo
- Publication number
- CN107506266A CN107506266A CN201710765240.4A CN201710765240A CN107506266A CN 107506266 A CN107506266 A CN 107506266A CN 201710765240 A CN201710765240 A CN 201710765240A CN 107506266 A CN107506266 A CN 107506266A
- Authority
- CN
- China
- Prior art keywords
- data
- restored
- disk
- file
- data block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
This application provides a kind of data reconstruction method and system, when the data disk exception in inner nuclear layer multivolume file system, can be based on the data disk stored in metadata disk, file and the incidence relation of data block determine the data block data block i.e. to be restored stored in abnormal data disk, and the file belonging to the data block is file to be restored, the mark for being then based on file to be restored determines the storage location of file to be restored, and then the data block to be restored of file to be restored is obtained from application layer distributed memory system based on the storage location, the data block of acquisition is write into normal data disk.The application can accomplish the other data recovery of block level using the positional information of file to be restored and the information of relevant data block, the amount of recovery of data can so be greatly reduced, and data recovery is in the processing of file system aspect, the burden of application layer distributed memory system can be mitigated, reduce the influence to front-end business, restorability is improved, shortens recovery time.
Description
Technical field
The present invention relates to technical field of data storage, more particularly to a kind of data reconstruction method and system.
Background technology
At present, application layer distributed memory system is widely used, and application layer distributed memory system is realized in application layer, will
Data are stored in underlying file systems with the file of fixed size, are protected by the way of multinode copy and automatically restoring fault
Demonstrate,prove the security of data.
Under normal circumstances, in application layer distributed memory system, data are evenly distributed on the file of each storage server
In system, for example, in a distributed file system, there are 4 files, each 3 copies of file, distribution situation such as Fig. 1 institutes
Show, in this case, if disk failures in storage server 1, file system can be identified as to visit shown in Fig. 1
Ask, now, for application layer distributed memory system, partial data is in the state of copy missing, in order to ensure data
Security, application layer distributed memory system can trigger data recover function, after fault recovery terminates, file distribution such as Fig. 2
It is shown.
At present, in the application layer distributed memory system of application layer, data recovery is all using file as granularity, by application layer
Distributed memory system is performed, and the file of copy recovery loss is read from other nodes of application layer distributed memory system, because
Substantial amounts of calculating is needed for recovery process, and system resource, including network, CPU, disk etc. are fought for business, therefore, to preceding
Hold service impact larger.
The content of the invention
In view of this, the invention provides a kind of data reconstruction method and system, to solve file of the prior art
Recovery process needs substantial amounts of calculating, and fights for system resource with business, the problem of being had a great influence to front-end business, its technology
Scheme is as follows：
A kind of data reconstruction method, applied to inner nuclear layer multivolume file system, the inner nuclear layer multivolume file system can be with
Application layer distributed memory system carries out data interaction, and the inner nuclear layer multivolume file system includes metadata disk and data magnetic
Disk, the incidence relation of data disk, file and data block is stored with the metadata disk；
The data reconstruction method includes：
Detect the status information of disk in the inner nuclear layer multivolume file system；
When the status information indicates abnormal disk, and the abnormal disk be present in the inner nuclear layer multivolume file system
For data disk when, if normal data disk also be present in the inner nuclear layer multivolume file system, and meet data recovery
Condition, then the data block for determining to store in the abnormal disk by the incidence relation is used as data block to be restored, and passes through
The incidence relation determines file belonging to the data block to be restored as file to be restored；
The identification information of the file to be restored is sent to the application layer distributed memory system, so that the application
Layer distributed memory system determines the storage location information of the file to be restored based on the identification information；
The storage location information that the application layer distributed memory system is sent is received, from the storage location information
The storage server of instruction obtains the data block to be restored in the file to be restored, and the data block write-in of acquisition is described just
Normal data disk.
Described data reconstruction method also includes：
By in the incidence relation of the metadata disk storage, changed with the data disk of the data block associated to be restored
To write the normal data disk of the data block to be restored.
Wherein, when the storage location information instruction of the file to be restored have multiple storage servers be stored with it is described treat it is extensive
During multiple file, the data to be restored in the file to be restored are obtained from the storage server of storage location information instruction
Block, including：
A storage server in the multiple storage servers indicated to the storage location information sends data read request,
Read the data block to be restored in the file to be restored；
When the data block to be restored in the file to be restored reads failure, indicated to the storage location information more
Another storage server in individual storage server sends data read request, reads the data block to be restored in the file to be restored.
Described data reconstruction method also includes：
When the abnormal disk is the metadata disk, or, the abnormal disk is the data disk and institute
State when other normal data disks being not present in inner nuclear layer multivolume file system, by the shape of the inner nuclear layer multivolume file system
State is arranged to inaccessible state.
Described data reconstruction method also includes：
When receiving the data write request that the application layer distributed memory system is sent, the data write request is judged
Whether corresponding data disk is normal；
When data disk exception corresponding to the data write request, judge that the inner nuclear layer multivolume file system is current
Whether state is data recovery state；
When the current state of the inner nuclear layer multivolume file system is data recovery state, the data write request is judged
Whether corresponding data block is the data block to be restored；
When data block is the data block to be restored corresponding to the data write request, the data write request is hung
Rise；
When the data block to be restored is recovered to complete, the data write request is waken up to carry out data write operation.
Described data reconstruction method also includes：
When receiving the data read request that the application layer distributed memory system is sent, the data read request is judged
Whether corresponding data disk is normal；
When data disk exception corresponding to the data read request, judge that the inner nuclear layer multivolume file system is current
Whether state is data recovery state；
When the current state of the inner nuclear layer multivolume file system is data recovery state, the data read request is judged
Whether corresponding data block is data block to be restored；
When data block corresponding to the data read request is the data block to be restored, terminate the data read request,
And the configured information of failure is read to the application layer distributed memory system returned data.
A kind of data recovery system, including：Inner nuclear layer multivolume file system and application layer distributed memory system；
The inner nuclear layer multivolume file system includes metadata disk and data disk, is stored with the metadata disk
The incidence relation of data disk, file and data block；
The inner nuclear layer multivolume file system, the state for detecting disk in the inner nuclear layer multivolume file system are believed
Breath；When the status information indicates abnormal disk in the inner nuclear layer multivolume file system be present, and the abnormal disk is number
During according to disk, if normal data disk also be present in the inner nuclear layer multivolume file system, and meet data recovery condition,
The data block for then determining to store in the abnormal disk by the incidence relation is used as data block to be restored, and passes through the pass
Connection relation determines file belonging to the data block to be restored as file to be restored；By the identification information of the file to be restored
Send to the application layer distributed memory system；
The application layer distributed memory system, for determining the storage of the file to be restored based on the identification information
Positional information, and the storage location information is sent to the inner nuclear layer multivolume file system；
The inner nuclear layer multivolume file system, is additionally operable to receive that the application layer distributed memory system sends described deposits
Positional information is stored up, the data to be restored in the file to be restored are obtained from the storage server of storage location information instruction
Block, and the data block of acquisition is write into the normal data disk.
Wherein, the inner nuclear layer multivolume file system, specifically for the multiple storages indicated to the storage location information
A storage server in server sends data read request, reads the data block to be restored in the file to be restored, works as institute
When stating the data block to be restored reading failure in file to be restored, to multiple storage servers of storage location information instruction
In another storage server send data read request, read the data block to be restored in the file to be restored.
The inner nuclear layer multivolume file system, it is additionally operable to when the number for receiving the application layer distributed memory system transmission
During according to write request, judge whether data disk corresponding to the data write request is normal, when number corresponding to the data write request
During according to disk exception, judge whether the current state of the inner nuclear layer multivolume file system is data recovery state, when described interior
When the current state of stratum nucleare multivolume file system is data recovery state, whether data block corresponding to the data write request is judged
For the data block to be restored, when data block is the data block to be restored corresponding to the data write request, by the data
Write request is hung up, and when the data block to be restored is recovered to complete, wakes up the data write request to carry out data write operation.
The inner nuclear layer multivolume file system, it is additionally operable to when the number for receiving the application layer distributed memory system transmission
During according to read request, judge whether data disk is normal corresponding to the data read request；When number corresponding to the data read request
During according to disk exception, judge whether the current state of the inner nuclear layer multivolume file system is data recovery state；When described interior
When the current state of stratum nucleare multivolume file system is data recovery state, whether data block corresponding to the data read request is judged
For data block to be restored；When data block corresponding to the data read request is the data block to be restored, terminate the data
Read request, and the configured information to fail is read to the application layer distributed memory system returned data.
Above-mentioned technical proposal has the advantages that：
It is distributed using inner nuclear layer multivolume file system as application layer in data reconstruction method provided by the invention and system
The rear end storage of storage system, when the data disk in inner nuclear layer multivolume file system causes to be distributed on the data disk extremely
Dropout of data block when, can based on data disk, file and the data block stored in metadata disk incidence relation determine
The data block that is stored in abnormal data disk data block i.e. to be restored, and file belonging to the data block is text to be restored
Part, the mark for being then based on the file to be restored determine storage position of the file to be restored in application layer distributed memory system
Put, and then the data block to be restored in file to be restored obtained from application layer distributed memory system based on the storage location,
And the data block of acquisition is write into normal data disk.As can be seen here, the present invention using file to be restored positional information and
The information of relevant data block can accomplish the other data recovery of block level, can so greatly reduce the amount of recovery of data, and
Data recovery is handled in file system aspect, can be mitigated the burden of application layer distributed memory system, be reduced to front-end business
Influence, improve restorability, shorten recovery time.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
The embodiment of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
The accompanying drawing of offer obtains other accompanying drawings.
Fig. 1 is the structure of application layer distributed memory system and file distribution schematic diagram in the prior art；
Fig. 2 is file distribution after the application layer distributed memory system trigger data recovery function in Fig. 1 makes fault recovery
Schematic diagram；
Fig. 3 is the schematic flow sheet of data reconstruction method provided in an embodiment of the present invention；
Fig. 4 is the structural representation of data recovery system provided in an embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, rather than whole embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other under the premise of creative work is not made
Embodiment, belong to the scope of protection of the invention.
It is more applied to inner nuclear layer multivolume file system, inner nuclear layer the embodiments of the invention provide a kind of data reconstruction method
Disk file system can carry out data interaction with application layer distributed memory system, and inner nuclear layer multivolume file system includes metadata magnetic
Disk and data disk, the incidence relation of data disk, file and data block are stored with metadata disk, referring to Fig. 3, showing
The schematic flow sheet of the data reconstruction method is gone out, can include：
Step S301：Detect the status information of disk in inner nuclear layer multivolume file system.
Wherein, the status information of disk is used to indicate that disk is normal or abnormal.
Step S302：When status information indicates abnormal disk be present in inner nuclear layer multivolume file system, and abnormal disk is
During data disk, if normal data disk also be present in inner nuclear layer multivolume file system, and meet data recovery condition, then
The data block for determining to store in abnormal disk by the incidence relation of data disk, file and data block is used as data to be restored
Block, and determine that the file belonging to data block to be restored is used as file to be restored by the incidence relation.
Wherein, the incidence relation of data disk, file and data block is used for indicating which stores in each data disk
Which data block of a little files.Table 1 below shows the example of the incidence relation of data disk, file and data block：
The incidence relation of the data disk of table 1, file and data block
Exemplary, exception, which occurs, for data disk 1 to access, now, based on the data disk shown in upper table 1, file
And the incidence relation of data block, it may be determined that file to be restored includes file 1, file 2 and file 3, the data block specifically recovered
For the data block 1 and data block 5 in file 1, the data block 1 in file 2, and, the data block 1 in file 3.
It should be noted that if abnormal disk is metadata disk, the state of inner nuclear layer multivolume file system is set
Inaccessible state is set to, if likewise, normal data disk is not present in inner nuclear layer multivolume file system, also by kernel
The state of layer multivolume file system is arranged to inaccessible state.
Wherein, data recovery condition can support data recovery for the space of inner nuclear layer multivolume file system spare.
Step S303：The identification information of file to be restored is sent to application layer distributed memory system, so that application layer
Distributed memory system determines the storage location information of file to be restored based on identification information.
In a kind of possible implementation, application layer distributed memory system is stored with the mark letter of the file of its storage
Breath and the incidence relation of file storage location information, after file to be restored is determined, the identification information of file to be restored is sent out
Application layer distributed memory system is delivered to, application layer distributed memory system can utilize the mark of file to be restored, based on text
The identification information of part and the incidence relation of file storage location information, determine the storage location information of file to be restored.
Exemplary, identification information and the file storage location information of the file of application layer distributed memory system storage
Incidence relation is as shown in table 2：
The identification information of the file of table 2 and the incidence relation of file storage location information
| File identification information | The storage location information of file |
| aa | Storage server 1 |
| bb | Storage server 2 |
| cc | Storage server 1 |
| dd | Storage server 3 |
Assuming that the file identification of file to be restored is aa, then application layer distributed memory system passes through shown in inquiry table 2
Incidence relation, the storage location information that would know that file aa to be restored are storage server 1, are determining file aa's to be restored
After storage location information, the storage location information is sent to inner nuclear layer multivolume file system.
Step S304：The storage location information that application layer distributed memory system is sent is received, is referred to from storage location information
The storage server shown obtains the data block to be restored in file to be restored, and the data block of acquisition is write into normal data magnetic
Disk.
Data reconstruction method provided in an embodiment of the present invention, deposited inner nuclear layer multivolume file system as application layer distribution
The rear end storage of storage system, when the data disk in inner nuclear layer multivolume file system causes to be distributed on the data disk extremely
During dropout of data block, it can be determined based on the incidence relation of data disk, file and the data block stored in metadata disk different
The data block stored in normal data disk data block i.e. to be restored, and file belonging to the data block is text to be restored
Part, the mark for being then based on the file to be restored determine storage position of the file to be restored in application layer distributed memory system
Put, and then the data block to be restored in file to be restored obtained from application layer distributed memory system based on the storage location,
And the data block of acquisition is write into normal data disk.As can be seen here, data reconstruction method profit provided in an embodiment of the present invention
It can accomplish the other data recovery of block level with the positional information of file to be restored and the information of relevant data block, so can be big
The big amount of recovery for reducing data, and data recovery is handled in file system aspect, can mitigate application layer distributed storage system
The burden of system, the influence to front-end business is reduced, improve restorability, shorten recovery time.
It should be noted that file to be restored there may be multiple duplicates of the document in application layer distributed memory system,
Now, the storage location information instruction of the file to be restored received has multiple storage servers to be stored with file to be restored, base
In this, in the data reconstruction method that above-described embodiment provides, obtained from the storage server of storage location information instruction to be restored
The process of data block to be restored in file can include：
A storage server in the multiple storage servers indicated to storage location information sends data read request, reads
Data block to be restored in file to be restored；When the data block to be restored in file to be restored reads failure, to storage location
Another storage server in multiple storage servers of information instruction sends data read request, reads in file to be restored and treats
Recover data block.
Exemplary, the storage location information instruction of file to be restored has multiple storage servers to be stored with text to be restored
Part, respectively storage server 1, storage server 2 and storage server 3, now, a memory can for example it store thereto
Server 1 sends data read request, and the data block to be restored of file to be restored is read from storage server 1, is lost if read
Lose, then send data read request to storage server 2, the data block to be restored of file to be restored is read from storage server 2.
On the basis of the data reconstruction method that the embodiments of the present invention provide, now by an instantiation to above-mentioned side
Method illustrates：
The state of disk in inner nuclear layer multivolume file system periodic scan system, when discovery data disk can not visit extremely
Ask, and when data recovery is supported in the space of system spare enough, trigger data recovery process.
Specifically, inner nuclear layer multivolume file system is based on data disk, file and the data block stored in metadata disk
Incidence relation determine data block to be restored and the file (file i.e. to be restored) belonging to data block to be restored, by text to be restored
The identification information of data block to be restored in the identification information of part and file to be restored forms list to be restored, then will treat extensive
Multiple list reports to application layer distributed memory system, and application layer distributed memory system is treated in listed files to be restored
The identification information of recovery file determines the storage location information of the duplicate of the document of file to be restored, the storage position that then will be determined
Confidence breath feeds back to inner nuclear layer multivolume file system, and inner nuclear layer multivolume file system is recovered to treat one by one extensive based on storage location information
The data block to be restored of file to be restored in multiple list, it is every recovered a data block to be restored when, by it from row to be restored
Deleted in table, when data block to be restored is not present in list to be restored, show that data recovery procedure terminates.
Wherein, the process recovered to each data block, the copy as based on file to be restored are distributed in application layer
The data block to be restored in storage location acquisition of information file to be restored in formula storage system, and the data block of acquisition is write
The process of normal data disk.
In the above-described embodiments, the incidence relation of data disk, file and data block is stored in metadata disk, i.e.,
Equivalent to the index information for storing data block, after by the data block of abnormal disk storage to other normal disks, data
The storage location of block is changed, now, in order to ensure subsequently can correctly have access to these data blocks, it is necessary to corresponding modification
The index information of these data blocks.Based on this, the method that above-described embodiment provides can also include：By metadata disk storage
In incidence relation, it is revised as writing the normal data magnetic of data block to be restored with the data disk of data block associated to be restored
Disk.
By taking the incidence relation shown in table 1 as an example, it is assumed that data disk 1 can not access there occurs abnormal, and carrying out, data are extensive
When multiple, the data block in data disk 1 is write into normal data disk 3, now, the incidence relation shown in table 1 is updated,
After renewal as shown in table 3：
The incidence relation of data disk, file and data block after the renewal of table 3
It should be noted that sometimes, it is understood that there may be the data disk in inner nuclear layer multivolume file system occurs different
After often, before data recovery, distributed memory system sends the situation of data write request, pin to inner nuclear layer multivolume file system
To such case, the data reconstruction method that above-described embodiment provides can also include：
When receiving the data write request of application layer distributed memory system transmission, number corresponding to data write request is judged
It is whether normal according to disk；When data disk exception corresponding to data write request, judge that inner nuclear layer multivolume file system is current
Whether state is data recovery state；When the current state of inner nuclear layer multivolume file system is data recovery state, number is judged
Whether it is data block to be restored according to data block corresponding to write request；When data block corresponding to data write request is data block to be restored
When, data write request is hung up, trigger the data write request corresponding to data block preferentially recover；When data block to be restored is recovered
During completion, data write request is waken up to carry out data write operation.
In addition, when the current state of inner nuclear layer multivolume file system is not data recovery state, show inner nuclear layer polydisc text
Part system does not have started carry out data recovery, now, terminates data write request, and return to number to application layer distributed memory system
According to the configured information for reading failure, so that application layer distributed memory system reads other copies of this document.Being write when data please
Data disk corresponding to asking where data block is normal data disk, or, when data block corresponding to data read request is not
During data block to be restored, data read operation is normally carried out.
Similarly, sometimes, after exception occurs for the data disk in inner nuclear layer multivolume file system, data recovery
Before, distributed memory system sends the situation of data read request to inner nuclear layer multivolume file system, above-mentioned for such case
The data reconstruction method that embodiment provides can also include：
When receiving the data read request of application layer distributed memory system transmission, number corresponding to data read request is judged
It is whether normal according to disk；When data disk exception corresponding to data read request, judge that inner nuclear layer multivolume file system is current
Whether state is data recovery state；When the current state of inner nuclear layer multivolume file system is data recovery state, number is judged
Whether it is data block to be restored according to data block corresponding to read request；When data block corresponding to data read request is data block to be restored
When, terminate data read request, and the configured information of failure is read to distributed memory system returned data, so that application layer is distributed
Formula storage system reads other copies of this document.
In addition, when the current state of inner nuclear layer multivolume file system is not data recovery state, show inner nuclear layer polydisc
File system does not have started carry out data recovery, now, terminates data write request, and return to application layer distributed memory system
The configured information of digital independent failure, so that application layer distributed memory system reads other copies of this document.When data are read
Data disk where data block corresponding to request is normal data disk, or, when data block corresponding to data read request not
For data block to be restored when, be normally carried out data read operation.
The embodiment of the present invention additionally provides a kind of data recovery system, referring to Fig. 4, showing the data recovery system
Structural representation, it can include：Inner nuclear layer multivolume file system 401 and application layer distributed memory system 402.
Wherein, inner nuclear layer multivolume file system 401 includes metadata disk and data disk, is stored with metadata disk
The incidence relation of data disk, file and data block.
Inner nuclear layer multivolume file system 401, for detecting the status information of disk in inner nuclear layer multivolume file system；Work as shape
Abnormal disk be present in state information instruction inner nuclear layer multivolume file system, and when abnormal disk is data disk, if inner nuclear layer
Normal data disk in multivolume file system also be present, and meet data recovery condition, then by data disk, file and
The data block that the incidence relation of data block determines to store in abnormal disk is and true by the incidence relation as data block to be restored
File belonging to fixed data block to be restored is as file to be restored；The identification information of file to be restored is sent to application layer and is distributed
Formula storage system 402.
Application layer distributed memory system 402, for determining the storage location information of file to be restored based on identification information,
And storage location information is sent to inner nuclear layer multivolume file system 401.
Inner nuclear layer multivolume file system 401, it is additionally operable to receive the storage location that application layer distributed memory system 402 is sent
Information, obtains the data block to be restored in file to be restored from the storage server of storage location information instruction, and by acquisition
Data block writes normal data disk.
Data recovery system provided by the invention, using inner nuclear layer multivolume file system as application layer distributed memory system
Rear end storage, when the data disk in inner nuclear layer multivolume file system causes the data block that is distributed on the data disk extremely
During loss, the number of incidence relation determination exception that can be based on data disk, file and the data block stored in metadata disk
According to the data block stored in disk data block i.e. to be restored, and file belonging to the data block is file to be restored, then
Mark based on the file to be restored determines storage location of the file to be restored in distributed memory system, and then is based on being somebody's turn to do
Storage location obtains the data block to be restored in file to be restored from application layer distributed memory system, and by the data of acquisition
Block writes normal data disk.As can be seen here, the embodiment of the present invention utilizes the positional information and related data of file to be restored
The information of block can accomplish the other data recovery of block level, can so greatly reduce the amount of recovery of data, and data recovery
In the processing of file system aspect, the burden of application layer distributed memory system can be mitigated, the influence to front-end business is reduced, carry
High restorability, shorten recovery time.
Above-described embodiment provide data recovery system in, inner nuclear layer multivolume file system 401, specifically for storage position
A storage server in multiple storage servers of confidence breath instruction sends data read request, reads in file to be restored and treats
Recover data block, when the data block to be restored in file to be restored reads failure, to multiple the depositing of storage location information instruction
Another storage server stored up in server sends data read request, reads the data block to be restored in file to be restored.
In the data recovery system that above-described embodiment provides, inner nuclear layer multivolume file system 401, it is additionally operable to metadata magnetic
In the incidence relation of disk storage, it is revised as writing the normal of data block to be restored with the data disk of data block associated to be restored
Data disk.
In the data recovery system that above-described embodiment provides, inner nuclear layer multivolume file system 401, it is additionally operable to work as abnormal disk
For metadata disk when, or, abnormal disk be not present in data disk and inner nuclear layer multivolume file system it is other normal
During data disk, the state of inner nuclear layer multivolume file system is arranged to inaccessible state.
In the data recovery system that above-described embodiment provides, inner nuclear layer multivolume file system 401, it is additionally operable to work as to receive and answers
During the data write request sent with layer distributed memory system, judge whether data disk is normal corresponding to data write request, when
Corresponding to data write request during data disk exception, judge whether the current state of inner nuclear layer multivolume file system is data recovery
State, when the current state of inner nuclear layer multivolume file system is data recovery state, judge data corresponding to data write request
Whether block is data block to be restored, and when data block corresponding to data write request is data block to be restored, data write request is hung
Rise, when data block to be restored is recovered to complete, wake up data write request to carry out data write operation.
In the data recovery system that above-described embodiment provides, inner nuclear layer multivolume file system, it is additionally operable to that application ought be received
During the data read request that layer distributed memory system is sent, judge whether data disk corresponding to data read request is normal；Work as number
According to during data disk exception, judging whether the current state of inner nuclear layer multivolume file system is data recovery shape corresponding to read request
State；When the current state of inner nuclear layer multivolume file system is data recovery state, data block corresponding to data read request is judged
Whether it is data block to be restored；When data block corresponding to data read request is data block to be restored, terminate data read request, and
The configured information of failure is read to application layer distributed memory system returned data.
Each embodiment is described by the way of progressive in this specification, what each embodiment stressed be and other
The difference of embodiment, between each embodiment identical similar portion mutually referring to.
In several embodiments provided herein, it should be understood that disclosed method, apparatus and equipment, can be with
Realize by another way.For example, device embodiment described above is only schematical, for example, the unit
Division, only a kind of division of logic function, can there is other dividing mode, such as multiple units or component when actually realizing
Another system can be combined or be desirably integrated into, or some features can be ignored, or do not perform.It is another, it is shown or
The mutual coupling discussed or direct-coupling or communication connection can be by some communication interfaces, between device or unit
Coupling or communication connection are connect, can be electrical, mechanical or other forms.
The unit illustrated as separating component can be or may not be physically separate, show as unit
The part shown can be or may not be physical location, you can with positioned at a place, or can also be distributed to multiple
On NE.Some or all of unit therein can be selected to realize the mesh of this embodiment scheme according to the actual needs
's.In addition, each functional unit in each embodiment of the present invention can be integrated in it is in a processing unit or each
Unit is individually physically present, can also two or more units it is integrated in a unit.
If the function is realized in the form of SFU software functional unit and is used as independent production marketing or in use, can be with
It is stored in a computer read/write memory medium.Based on such understanding, technical scheme is substantially in other words
The part to be contributed to prior art or the part of the technical scheme can be embodied in the form of software product, the meter
Calculation machine software product is stored in a storage medium, including some instructions are causing a computer equipment (can be
People's computer, server, or network equipment etc.) perform all or part of step of each embodiment methods described of the present invention.
And foregoing storage medium includes：USB flash disk, mobile hard disk, read-only storage (ROM, Read-Only Memory), arbitrary access are deposited
Reservoir (RAM, RandomAccess Memory), magnetic disc or CD etc. are various can be with the medium of store program codes.
The foregoing description of the disclosed embodiments, professional and technical personnel in the field are enable to realize or using the present invention.
A variety of modifications to these embodiments will be apparent for those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, it is of the invention
The embodiments shown herein is not intended to be limited to, and is to fit to and principles disclosed herein and features of novelty phase one
The most wide scope caused.
Claims (10)
- A kind of 1. data reconstruction method, it is characterised in that applied to inner nuclear layer multivolume file system, the inner nuclear layer multivolume file System can carry out data interaction with application layer distributed memory system, and the inner nuclear layer multivolume file system includes metadata disk And data disk, the incidence relation of data disk, file and data block is stored with the metadata disk；The data reconstruction method includes：Detect the status information of disk in the inner nuclear layer multivolume file system；When the status information indicates abnormal disk in the inner nuclear layer multivolume file system be present, and the abnormal disk is number During according to disk, if normal data disk also be present in the inner nuclear layer multivolume file system, and meet data recovery condition, The data block for then determining to store in the abnormal disk by the incidence relation is used as data block to be restored, and passes through the pass Connection relation determines file belonging to the data block to be restored as file to be restored；The identification information of the file to be restored is sent to the application layer distributed memory system, so that the application layer point Cloth storage system determines the storage location information of the file to be restored based on the identification information；The storage location information that the application layer distributed memory system is sent is received, is indicated from the storage location information Storage server obtain data block to be restored in the file to be restored, it is and the data block write-in of acquisition is described normal Data disk.
- 2. data reconstruction method according to claim 1, it is characterised in that the data reconstruction method also includes：By in the incidence relation of the metadata disk storage, it is revised as writing with the data disk of the data block associated to be restored Enter the normal data disk of the data block to be restored.
- 3. data reconstruction method according to claim 1, it is characterised in that when the storage location of the file to be restored is believed When breath instruction has multiple storage servers to be stored with the file to be restored, from the storage service of storage location information instruction Device obtains the data block to be restored in the file to be restored, including：A storage server in the multiple storage servers indicated to the storage location information sends data read request, reads Data block to be restored in the file to be restored；When the data block to be restored in the file to be restored reads failure, to multiple the depositing of storage location information instruction Another storage server stored up in server sends data read request, reads the data block to be restored in the file to be restored.
- 4. data reconstruction method according to claim 1, it is characterised in that the data reconstruction method also includes：When the abnormal disk is the metadata disk, or, the abnormal disk be the data disk and it is described in When other normal data disks being not present in stratum nucleare multivolume file system, the state of the inner nuclear layer multivolume file system is set It is set to inaccessible state.
- 5. the data reconstruction method according to any one in Claims 1 to 4, it is characterised in that methods described also includes：When receiving the data write request that the application layer distributed memory system is sent, judge that the data write request is corresponding Data disk it is whether normal；When data disk exception corresponding to the data write request, the current state of the inner nuclear layer multivolume file system is judged Whether it is data recovery state；When the current state of the inner nuclear layer multivolume file system is data recovery state, judge that the data write request is corresponding Data block whether be the data block to be restored；When data block is the data block to be restored corresponding to the data write request, the data write request is hung up；When the data block to be restored is recovered to complete, the data write request is waken up to carry out data write operation.
- 6. the data reconstruction method according to any one in Claims 1 to 4, it is characterised in that methods described also includes：When receiving the data read request that the application layer distributed memory system is sent, judge that the data read request is corresponding Data disk it is whether normal；When data disk exception corresponding to the data read request, the current state of the inner nuclear layer multivolume file system is judged Whether it is data recovery state；When the current state of the inner nuclear layer multivolume file system is data recovery state, judge that the data read request is corresponding Data block whether be data block to be restored；When data block corresponding to the data read request is the data block to be restored, terminate the data read request, and to The application layer distributed memory system returned data reads the configured information of failure.
- 7. a kind of data recovery system, it is characterised in that the system includes：Inner nuclear layer multivolume file system and application layer distribution Formula storage system；The inner nuclear layer multivolume file system includes metadata disk and data disk, and data are stored with the metadata disk The incidence relation of disk, file and data block；The inner nuclear layer multivolume file system, for detecting the status information of disk in the inner nuclear layer multivolume file system；When The status information indicates abnormal disk be present in the inner nuclear layer multivolume file system, and the abnormal disk is data disk When, if normal data disk also be present in the inner nuclear layer multivolume file system, and meet data recovery condition, then pass through The data block that the incidence relation determines to store in the abnormal disk passes through the incidence relation as data block to be restored Determine the file belonging to the data block to be restored as file to be restored；By the identification information of the file to be restored send to The application layer distributed memory system；The application layer distributed memory system, for determining the storage location of the file to be restored based on the identification information Information, and the storage location information is sent to the inner nuclear layer multivolume file system；The inner nuclear layer multivolume file system, it is additionally operable to receive the storage position that the application layer distributed memory system is sent Confidence is ceased, and the data block to be restored in the file to be restored is obtained from the storage server of storage location information instruction, And the data block of acquisition is write into the normal data disk.
- 8. data recovery system according to claim 7, it is characterised in that the inner nuclear layer multivolume file system, specifically Data read request is sent for the storage server in multiple storage servers for being indicated to the storage location information, is read Data block to be restored in the file to be restored, when the data block to be restored in the file to be restored reads failure, to Another storage server in multiple storage servers of storage location information instruction sends data read request, described in reading Data block to be restored in file to be restored.
- 9. the data recovery system according to claim 7 or 8, it is characterised in that the inner nuclear layer multivolume file system, also For when receiving the data write request that the application layer distributed memory system is sent, judging that the data write request is corresponding Data disk it is whether normal, when data disk exception corresponding to the data write request, judge inner nuclear layer polydisc text Whether the current state of part system is data recovery state, when the current state of the inner nuclear layer multivolume file system is that data are extensive During multiple state, judge whether data block corresponding to the data write request is the data block to be restored, being write when the data please When data block corresponding to asking is the data block to be restored, the data write request is hung up, when the data block to be restored When recovering to complete, the data write request is waken up to carry out data write operation.
- 10. the data recovery system according to claim 7 or 8, it is characterised in that the inner nuclear layer multivolume file system, It is additionally operable to, when receiving the data read request that the application layer distributed memory system is sent, judge the data read request pair Whether the data disk answered is normal；When data disk exception corresponding to the data read request, the inner nuclear layer polydisc is judged Whether the current state of file system is data recovery state；When the current state of the inner nuclear layer multivolume file system is data During recovery state, judge whether data block corresponding to the data read request is data block to be restored；When the data read request When corresponding data block is the data block to be restored, terminate the data read request, and to the application layer distributed storage System returned data reads the configured information of failure.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710765240.4A CN107506266B (en) | 2017-08-30 | 2017-08-30 | Data recovery method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710765240.4A CN107506266B (en) | 2017-08-30 | 2017-08-30 | Data recovery method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107506266A true CN107506266A (en) | 2017-12-22 |
| CN107506266B CN107506266B (en) | 2020-06-23 |
Family
ID=60693545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710765240.4A Active CN107506266B (en) | 2017-08-30 | 2017-08-30 | Data recovery method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107506266B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287164A (en) * | 2019-06-28 | 2019-09-27 | 联想(北京)有限公司 | A kind of data reconstruction method, device and computer equipment |
| WO2020000316A1 (en) * | 2018-06-28 | 2020-01-02 | 华为技术有限公司 | Fault tolerance processing method, device, and server |
| CN110737543A (en) * | 2019-09-10 | 2020-01-31 | 苏州浪潮智能科技有限公司 | method, device and storage medium for recovering distributed file system data |
| CN110781031A (en) * | 2019-09-25 | 2020-02-11 | 潍柴动力股份有限公司 | Controller data recovery method and device, controller and automobile |
| CN110781028A (en) * | 2018-07-30 | 2020-02-11 | 阿里巴巴集团控股有限公司 | Data backup method, data recovery method, data backup device, data recovery device and computing equipment |
| CN113051104A (en) * | 2021-03-11 | 2021-06-29 | 重庆紫光华山智安科技有限公司 | Method and related device for recovering data between disks based on erasure codes |
| CN113485872A (en) * | 2021-07-14 | 2021-10-08 | 北京金山云网络技术有限公司 | Fault processing method and device and distributed storage system |
| WO2023240944A1 (en) * | 2022-06-14 | 2023-12-21 | 锐凌无线有限责任公司 | Data recovery method and apparatus, electronic device, and storage medium |
| WO2024174682A1 (en) * | 2023-02-21 | 2024-08-29 | 中兴通讯股份有限公司 | Data processing method and apparatus, and electronic device and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499027A (en) * | 2009-03-06 | 2009-08-05 | 赵晓宇 | Intelligent memory system based on independent kernel and distributed architecture |
| US20130125190A1 (en) * | 2009-12-29 | 2013-05-16 | Cleversafe, Inc. | Digital content distribution utilizing dispersed storage |
| US20140282764A1 (en) * | 2009-10-29 | 2014-09-18 | Cleversafe, Inc. | Media distribution to a plurality of devices utilizing buffered dispersed storage |
-
2017
- 2017-08-30 CN CN201710765240.4A patent/CN107506266B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499027A (en) * | 2009-03-06 | 2009-08-05 | 赵晓宇 | Intelligent memory system based on independent kernel and distributed architecture |
| US20140282764A1 (en) * | 2009-10-29 | 2014-09-18 | Cleversafe, Inc. | Media distribution to a plurality of devices utilizing buffered dispersed storage |
| US20130125190A1 (en) * | 2009-12-29 | 2013-05-16 | Cleversafe, Inc. | Digital content distribution utilizing dispersed storage |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11231983B2 (en) | 2018-06-28 | 2022-01-25 | Huawei Technologies Co., Ltd. | Fault tolerance processing method, apparatus, and server |
| WO2020000316A1 (en) * | 2018-06-28 | 2020-01-02 | 华为技术有限公司 | Fault tolerance processing method, device, and server |
| CN113626235A (en) * | 2018-06-28 | 2021-11-09 | 华为技术有限公司 | Fault tolerance processing method and device and server |
| CN110781028B (en) * | 2018-07-30 | 2023-04-11 | 阿里巴巴集团控股有限公司 | Data backup method, data recovery method, data backup device, data recovery device and computing equipment |
| CN110781028A (en) * | 2018-07-30 | 2020-02-11 | 阿里巴巴集团控股有限公司 | Data backup method, data recovery method, data backup device, data recovery device and computing equipment |
| CN110287164A (en) * | 2019-06-28 | 2019-09-27 | 联想(北京)有限公司 | A kind of data reconstruction method, device and computer equipment |
| CN110737543A (en) * | 2019-09-10 | 2020-01-31 | 苏州浪潮智能科技有限公司 | method, device and storage medium for recovering distributed file system data |
| CN110737543B (en) * | 2019-09-10 | 2022-07-19 | 苏州浪潮智能科技有限公司 | Method, device and storage medium for recovering distributed file system data |
| CN110781031B (en) * | 2019-09-25 | 2022-04-26 | 潍柴动力股份有限公司 | Controller data recovery method and device, controller and automobile |
| CN110781031A (en) * | 2019-09-25 | 2020-02-11 | 潍柴动力股份有限公司 | Controller data recovery method and device, controller and automobile |
| CN113051104A (en) * | 2021-03-11 | 2021-06-29 | 重庆紫光华山智安科技有限公司 | Method and related device for recovering data between disks based on erasure codes |
| CN113051104B (en) * | 2021-03-11 | 2022-10-11 | 重庆紫光华山智安科技有限公司 | Method and related device for recovering data between disks based on erasure codes |
| CN113485872A (en) * | 2021-07-14 | 2021-10-08 | 北京金山云网络技术有限公司 | Fault processing method and device and distributed storage system |
| WO2023240944A1 (en) * | 2022-06-14 | 2023-12-21 | 锐凌无线有限责任公司 | Data recovery method and apparatus, electronic device, and storage medium |
| WO2024174682A1 (en) * | 2023-02-21 | 2024-08-29 | 中兴通讯股份有限公司 | Data processing method and apparatus, and electronic device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107506266B (en) | 2020-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107506266A (en) | A kind of data reconstruction method and system | |
| CN110807064B (en) | Data recovery device in RAC distributed database cluster system | |
| US9965364B2 (en) | Fault tolerant listener registration in the presence of node crashes in a data grid | |
| CN102567478B (en) | Online fault verification in a file system | |
| CN104685474B (en) | For the method for handling not repairable EMS memory error and non-transient processor readable medium | |
| CN113535474B (en) | Method, system, medium and terminal for automatically repairing heterogeneous cloud storage cluster fault | |
| CN110825562B (en) | Data backup method, device, system and storage medium | |
| US8554727B2 (en) | Method and system of tiered quiescing | |
| CN108604201B (en) | Snapshot rollback method, device, storage controller and system | |
| CN109753378A (en) | A kind of partition method of memory failure, device, system and readable storage medium storing program for executing | |
| CN106331166A (en) | Access method and device of storage resource | |
| CN106528338A (en) | Remote data replication method, storage equipment and storage system | |
| CN110333971A (en) | SSD bad block table backup method, device, computer equipment and storage medium | |
| CN102025758B (en) | Method, device and system for recovering data copy in distributed system | |
| CN102457400B (en) | Method for preventing split brain phenomenon from occurring on distributed replicated block device (DRBD) resource | |
| CN110046065A (en) | A kind of storage array method for reconstructing, device, equipment and storage medium | |
| CN105892954A (en) | Data storage method and device based on multiple copies | |
| CN104407806A (en) | Method and device for revising hard disk information of redundant array group of independent disk (RAID) | |
| US20080250421A1 (en) | Data Processing System And Method | |
| CN113051428A (en) | Method and device for storing and backing up front end of camera | |
| CN116401122A (en) | Method, system, device and medium for processing node faults of distributed storage system | |
| CN106776142B (en) | Data storage method and data storage device | |
| CN109325005A (en) | A kind of data processing method and electronic equipment | |
| US10089202B1 (en) | Providing data high availability to a set of host computers via automatic failover | |
| CN107025150A (en) | A kind of system and method for realizing the control of data backup real-time recovery |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |