WO2015101084A1

WO2015101084A1 - Method, server and system for data recovery

Info

Publication number: WO2015101084A1
Application number: PCT/CN2014/088962
Authority: WO
Inventors: 郝志敏; 李红
Original assignee: 华为技术有限公司
Priority date: 2013-12-31
Filing date: 2014-10-20
Publication date: 2015-07-09
Also published as: CN103761161B; CN103761161A

Abstract

The present invention relates to the field of information technologies. Disclosed are a method, server and system for data recovery. The method comprises: a master server receiving first recovery log files returned by slave servers in a storage system according to a log acquisition request sent by the master server, and performing an operation on to-be-recovered data according to latest operation information recorded in all the first recovery log files and in a second recovery log file generated by the master server, the first recovery log files and the second recovery log file recording location information and operation information of the to-be-recovered data identified by a to-be-recovered data identifier. In the present invention, the master server sends a log acquisition request to the slave servers in the storage system and then recovers to-be-recovered data according to all first recovery log files returned by the slave servers and a second recovery log file generated by the master server, so that data recovery can be realized without scanning all data or using a metadata server, and the data recovery efficiency is improved.

Description

Method, server and system for restoring data

The present application claims priority to Chinese Patent Application No. 20131075452, filed on Dec. 31, 2013, the entire disclosure of which is incorporated herein by reference. .

Technical field

The present invention relates to the field of information technology, and in particular, to a method, a server, and a system for recovering data.

Background technique

With the continuous development of information technology, the scale of storage systems is growing. In large-scale storage systems, the frequency of device failures also increases. In the event of a device failure, how to recover data to ensure data security has become a concern.

Currently, there are three ways to recover data. The first method is to scan the file system by the fsck command to find the data to be recovered and restore the data to be recovered. The second method is: reading the log information of the data to be restored in the system operation log through the metadata server, and controlling the data server to perform the operation on the data according to the read log information of the data to be restored, thereby restoring the data. The third method is: reading the log information of the data to be restored through the metadata server, and controlling the data server to perform the operation on the data according to the read log information to be restored, thereby restoring the data.

The prior art has at least the following problems:

The first way is to scan all the data, causing a waste of system resources. The second and third methods require the metadata server to recover data, while the metadata server recovers data at a slower rate, resulting in less efficient data recovery.

Summary of the invention

The embodiment of the invention provides a method, a server and a system for restoring data, which can solve the problems of the prior art. The technical solution is as follows:

In a first aspect, a method for recovering data is provided, the method being applied to a primary server of a storage system, the storage system including a primary server and a secondary server, both the primary server and the secondary server are configured to store the same data The primary server is the first server that successfully stores data, and the method includes:

When any one of the storage systems fails to store data, the primary server is Each of the storage systems sends a log obtaining request, where the log obtaining request carries the data identifier to be restored;

Receiving, by the respective slave server, the first recovery log file returned according to the log obtaining request, where each first recovery log file records location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier;

And operating the restored data according to the latest operation information in the operation information recorded by the first recovery log file and the second recovery log file, where the to-be-recovered data is searched according to the location information of the to-be-recovered data, The second recovery log file is generated by the primary server, and records location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier.

In conjunction with the first aspect, in a first possible implementation manner of the first aspect, before the sending a log obtaining request to the server in the storage system, the method further includes:

Receiving and storing data sent by the client, and returning, to the client, first write status information that includes the identification identifier of the data, where the first write status information is write success information or write failure information;

Receiving total write status information sent by the client, where the total write status information includes first write status information returned by the primary server to the client, and each slave server returns to the client Second write status information;

If the total write status information sent by the client includes the write failure information, determining that the data sent by the client is the data to be restored, and generating the second location information and the operation information for recording the data to be restored. Restore the log file.

With reference to the first aspect, in a second possible implementation manner of the first aspect, the performing the data to be restored according to the latest operation information in the operation information recorded by all the first recovery log file and the second recovery log file Operations, including:

Reading each of the first recovery log file and the second recovery log file in sequence;

Determining whether the index information of the currently read recovery log file is in a preset structure, the index information includes a recovery log file identifier, an offset information, and data length information, where the preset structure stores index information and an operation time;

If the index information of the currently read recovery log file is not in the preset structure, adding the index information and the operation time of the currently read recovery log file to the preset structure;

Continue to read the next recovery log file, process the next recovery log file read in the same way as the currently read recovery log file, until the last recovery log file is processed, and the file is saved. Prestored with or without repeated index information and operation time;

Obtaining a recovery log file corresponding to the index information and the operation time in the preset structure, and using the recovery log file corresponding to the index information and the operation time in the preset structure as the filtered recovery log file, according to the screening The operation information in the recovery log file is operated on the data to be restored.

With reference to the first aspect to the second possible implementation of the first aspect, in a third possible implementation manner of the first aspect, the determining whether the index information of the currently read recovery log file is in a preset structure After that, it also includes:

If the index information of the currently read recovery log file is in the preset structure, the operation time of the currently read recovery log file and the operation time of the index information corresponding to the currently read recovery log file in the preset structure Compare;

If the operation time of the currently read recovery log file is later than the operation time of the index information corresponding to the currently read recovery log file in the preset structure, the operation time of the currently read recovery log file is replaced with the preset structure and The operation time corresponding to the index information of the currently read recovery log file, and the step of continuing to read the next recovery log file.

In combination with the first aspect to the second or the third possible implementation of the first aspect, in a fourth possible implementation of the first aspect, obtaining the preset The index information in the structure and the recovery log file corresponding to the operation time, including:

And sequentially reading the first recovery log file and the second recovery log file corresponding to the to-be-recovered data identifier, and sequentially comparing the index information of the currently read recovery log file with the index information in the preset structure. Correct;

If the index information of the recovery log file that is currently read is the same as the index information of the currently read recovery log file, compare the operation time of the currently read recovery log file with the currently read recovery log in the preset structure. The operation time of the index information of the file;

If the operation time of the index information corresponding to the currently read recovery log file is the same as the operation time of the currently read recovery log file in the preset structure, the currently read recovery log file is added to the filtered Restore the log file and read the next recovery log file;

And so on, until the last recovery log file is compared, the recovery log file corresponding to each index information in the preset structure is obtained.

With reference to the first aspect to the second possible implementation of the first aspect, in a fifth possible implementation manner of the first aspect, the recovering the information corresponding to the index information and the operation time in the preset structure After the complex log file is used as the filtered recovery log file, it also includes:

The filtered recovery log file is sent to each slave server, and the data to be recovered is restored by each slave server according to the filtered recovery log file.

In a second aspect, a method for recovering data is provided, the method being applied to a primary server of a storage system, the storage system including the primary server and a secondary server, the primary server and the secondary server are both used for storage The same data, the primary server is the first server to successfully store data, and the method includes:

Receiving, by the server, a log obtaining request sent by the primary server, where the log obtaining request carries a data identifier to be restored;

Returning, to the primary server, a first recovery log file according to the log obtaining request, and causing the primary server to recover the to-be-recovered data according to the first recovery log file and the second recovery log file generated by the primary server;

Receiving the filtered recovery log file sent by the primary server, and operating the restored data according to the operation information recorded by the filtered recovery log file;

The first recovery log file and the second recovery log file respectively record location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier.

With reference to the second aspect, in a first possible implementation manner of the second aspect, before the returning the first recovery log file to the primary server according to the log obtaining request, the method further includes:

Receiving and storing data sent by the client, and returning, to the client, second write status information including an identification identifier of the data, where the second write status information includes write success information or write failure information;

Receiving total write status information sent by the client, where the total write status information includes the second write status information returned from the server to the client, and the return of the primary server to the client First write status information;

If the total write status information sent by the client includes the write failure information, determining that the data sent by the client is the data to be restored, and generating the first location information and the operation information for recording the data to be restored. Complex log file.

With reference to the second aspect, in a second possible implementation manner of the second aspect, after the returning the first recovery log file to the primary server according to the log obtaining request, the method further includes:

Receiving the filtered recovery log file sent by the primary server, and recovering the to-be-recovered data according to the filtered recovery log file.

In a third aspect, a server for recovering data is provided, the server being a primary server applied to a storage system, the storage system including the primary server and a secondary server, the primary server and the secondary server are both used The same data is stored, and the primary server is the first server that successfully stores data, and the server includes:

a first sending module, configured to send a log obtaining request to each of the slave servers in the storage system when the storage of the data in the storage system fails, where the log obtaining request carries the data identifier to be restored;

a first receiving module, configured to receive a first recovery log file returned by each slave server according to the log obtaining request, where each first recovery log file records location information of the to-be-recovered data identified by the to-be-recovered data identifier and Operational information;

a recovery module, configured to perform, according to the latest operation information in the operation information recorded by the first recovery log file and the second recovery log file, the data to be restored according to the location information of the data to be restored The second recovery log file is generated by the primary server, and the location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier are recorded.

In conjunction with the third aspect, in a first possible implementation manner of the third aspect, the server further includes:

a storage module, configured to receive and store data sent by the client;

a returning module, configured to return, to the client, first write state information that includes an identifier of the data, where the first write state information includes write success information or write failure information;

a second receiving module, configured to receive total write status information sent by the client, where the total write status information includes first write status information returned by the primary server to the client, and each slave server Second write status information returned to the client;

a generating module, configured to: when the total write status information sent by the client includes the write failure information, determine that the data sent by the client is the data to be restored, and generate location information that records the to-be-recovered data and The second recovery log file of the operation information.

With reference to the third aspect, in a second possible implementation manner of the third aspect, the recovery module includes:

a first reading unit, configured to sequentially read each of the first recovery log file and the second recovery log file;

a determining unit, configured to determine whether the index information of the currently read recovery log file is in a preset structure The index information includes a recovery log file identifier, an offset information, and data length information, where the preset structure is used to store index information and an operation time;

An adding unit, configured to add index information and an operation time of the currently read recovery log file to the preset structure when the index information of the currently read recovery log file is not in the preset structure;

a second reading unit, configured to continue to read the next recovery log file;

a processing unit, configured to process the read next recovery log file in a manner of processing the currently read recovery log file, until the last recovery log file is processed, to obtain a preset structure for storing the duplicate index information and the operation time;

An obtaining unit, configured to acquire a recovery log file corresponding to the index information and the operation time in the preset structure, and use the recovery log file corresponding to the index information and the operation time in the preset structure as the filtered recovery Log file

A recovery unit, configured to operate on the restored data according to the operation information in the filtered recovery log file.

With reference to the third aspect to the second possible implementation of the third aspect, in a third possible implementation manner of the third aspect, the recovery module further includes:

a comparison unit, configured to: when the index information of the currently read recovery log file is in the preset structure, the operation time of the currently read recovery log file and the correspondingly read recovery log file in the preset structure The operation time of the index information is compared;

a replacement unit, configured to replace the operation time of the currently read recovery log file when the operation time of the currently read recovery log file is later than the operation time of the index information corresponding to the currently read recovery log file in the preset structure The operation time in the preset structure corresponding to the index information of the currently read recovery log file;

The second reading unit is further configured to perform the step of continuing to read the next recovery log file.

With reference to the third aspect to any one of the second or the third possible implementation manners of the first aspect, in a fourth possible implementation manner of the third aspect, the acquiring unit includes :

a first reading subunit, configured to sequentially read the first recovery log file and a second recovery log file corresponding to the to-be-recovered data identifier;

The comparison subunit is configured to compare the index information of the currently read recovery log file with the index information in the preset structure;

a comparison subunit, configured to: when the index information of the recovery log file currently read is the same as the index information of the currently read recovery log file, compare the operation time of the currently read recovery log file with the preset structure The operation time of the index information of the currently read recovery log file;

Adding a subunit, configured to: when the operation time of the index information corresponding to the currently read recovery log file is the same as the operation time of the currently read recovery log file in the preset structure, the currently read recovery log The file is added to the filtered recovery log file;

The second read subunit is configured to read the next recovery log file; and so on, until the last recovery log file is compared, and the recovery log file corresponding to each index information in the preset structure is obtained.

In conjunction with the third aspect, in a fifth possible implementation manner of the third aspect, the server further includes:

The second sending module is configured to send the filtered recovery log file to each slave server, and the slave servers recover the data to be restored according to the filtered recovery log file.

A fourth aspect provides a server for recovering data, the server being a slave server applied to a storage system, the storage system including a master server and the slave server, the master server and the slave server are both used The same data is stored, and the primary server is the first server that successfully stores data, and the server includes:

a first receiving module, configured to receive a log obtaining request sent by the primary server, where the log obtaining request carries a data identifier to be restored;

a first returning module, configured to return a first recovery log file to the primary server according to the log obtaining request, so that the primary server generates a second recovery log file according to the first recovery log file and the primary server Recovering the data to be recovered;

a second receiving module, configured to receive the filtered recovery log file sent by the primary server;

a recovery module, configured to perform operation on the restored data according to the operation information recorded by the filtered recovery log file;

With reference to the fourth aspect, in a first possible implementation manner of the fourth aspect, the server further includes:

a storage module, configured to receive and store data sent by the client;

a second returning module, configured to return, to the client, a second write that includes the identification identifier of the data Entering status information, the second write status information includes write success information or write failure information;

a third receiving module, configured to receive total write status information sent by the client, where the total write status information includes the second write status information returned from the server to the client, and the primary server First write status information returned to the client;

a generating module, configured to: when the total write status information sent by the client includes the write failure information, determine that the data sent by the client is the data to be restored, and generate location information that records the to-be-recovered data and The first recovery log file for operational information.

In a fifth aspect, a system for recovering data is provided, the system comprising: a primary server and at least one secondary server;

Wherein, the primary server is the first type of server as described above, and the secondary server is the second type of server as described above.

The beneficial effects brought by the technical solutions provided by the embodiments of the present invention are:

After the primary server sends a log obtaining request to each slave server in the storage system, the first recovery log file returned by each slave server is received, and the first recovery log file and the second recovery log file generated by the master server are restored to be restored. Data, so you don't need to scan all the data, you can restore data without using a metadata server, and improve the efficiency of restoring data.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Other drawings may also be obtained from those of ordinary skill in the art in light of the inventive work.

1 is a flowchart of a method for restoring data according to Embodiment 1 of the present invention;

2 is a flowchart of another method for restoring data according to Embodiment 1 of the present invention;

3 is a flowchart of a method for restoring data according to Embodiment 2 of the present invention;

4 is a schematic diagram of an interaction process provided by Embodiment 2 of the present invention;

FIG. 5 is a schematic structural diagram of a server for restoring data according to Embodiment 3 of the present invention; FIG.

6 is a schematic structural diagram of a server for recovering data according to Embodiment 4 of the present invention;

FIG. 7 is a schematic structural diagram of a system for restoring data according to Embodiment 5 of the present invention.

detailed description

The embodiments of the present invention will be described in detail as an alternative embodiment with reference to the accompanying drawings.

Embodiment 1

An embodiment of the present invention provides a method for recovering data, where the method is applied to a primary server of a storage system, where the storage system includes a primary server and a secondary server, and the primary server and the secondary server are both used to store the same data, and the primary server is the first For example, the server of the embodiment of the present invention includes:

When the storage of the data in the storage system fails, the primary server sends a log obtaining request to each of the secondary servers in the storage system, and the log obtaining request carries the data identifier to be restored.

As an optional embodiment, before sending a log obtaining request to each slave server in the storage system, the method further includes:

Receiving and storing the data sent by the client, and returning, to the client, the first write status information that includes the identification identifier of the data, where the first write status information is a write success information or a write failure information;

Receiving total write status information sent by the client, where the total write status information includes first write status information returned by the primary server to the client and second write status information returned from the server to the client;

If the total write status information sent by the client includes the write failure information, it is determined that the data sent by the client is the data to be restored, and the second recovery log file that records the location information of the data to be restored and the operation information is generated.

Step 102: Receive a first recovery log file returned by each slave server according to the log obtaining request, where each first recovery log file records location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier;

103: Perform operation on the restored data according to the latest operation information in the operation information recorded by all the first recovery log files and the second recovery log file, and the data to be restored is searched according to the location information of the data to be restored, and the second recovery log file is obtained by The primary server generates and records location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier.

As an optional embodiment, the data is to be restored according to the latest operation information in the operation information recorded by all the first recovery log files and the second recovery log file, including:

Reading each first recovery log file and second recovery log file in turn;

Determining whether the index information of the currently read recovery log file is in a preset structure, and indexing the information packet The recovery log file identifier, the offset information, and the data length information are used, and the preset structure is used to store the index information and the operation time;

If the index information of the currently read recovery log file is not in the preset structure, the index information and the operation time of the currently read recovery log file are added to the preset structure;

Continue to read the next recovery log file, and process the next recovery log file read in the manner of processing the currently read recovery log file until the last recovery log file is processed, and whether the storage has duplicate index information and operation time is obtained. Preset structure;

Obtaining a recovery log file corresponding to the index information and the operation time in the preset structure, and using the recovery log file corresponding to the index information and the operation time in the preset structure as the filtered recovery log file, according to the filtered recovery log The operational information in the file operates on the recovered data.

As an optional embodiment, after determining whether the index information of the currently read recovery log file is in the preset structure, the method further includes:

If the index information of the currently read recovery log file is in the preset structure, compare the operation time of the currently read recovery log file with the operation time of the index information corresponding to the currently read recovery log file in the preset structure. ;

As an optional embodiment, the recovery log file corresponding to the index information and the operation time in the preset structure is obtained, including:

The first recovery log file and the second recovery log file corresponding to the to-be-recovered data identifier are sequentially read, and the index information of the currently-recovered recovery log file is sequentially compared with the index information in the preset structure.

If there is index information in the preset structure that is the same as the index information of the currently-recovered recovery log file, compare the operation time of the currently-recovered recovery log file with the index information of the currently-recovered recovery log file in the preset structure. Operating time

If the operation time of the index information corresponding to the currently read recovery log file is the same as the operation time of the currently read recovery log file in the preset structure, the currently read recovery log file is added to the filtered recovery log. File and read the next recovery log file;

And so on, until the last recovery log file is compared, get each cable in the default structure. The recovery log file corresponding to the information.

As an optional embodiment, after the recovery log file corresponding to the index information and the operation time in the preset structure is used as the filtered recovery log file, the method further includes:

The filtered recovery log file is sent to each slave server, and each slave server recovers the data to be restored according to the filtered recovery log file.

The method is applied to the primary server of the storage system. The storage system includes a primary server and a secondary server. The primary server and the secondary server are used to store the same data. The primary server is the first server that successfully stores data. For example, see FIG. 2, The method flow provided by the embodiment of the invention includes:

201: Each slave server receives a log obtaining request sent by the primary server, where the log obtaining request carries the data identifier to be restored;

202: Return, according to the log obtaining request, the first recovery log file to the primary server, so that the primary server recovers the data to be restored according to the first recovery log file and the second recovery log file generated by the primary server;

The first recovery log file and the second recovery log file record the location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier.

As an optional embodiment, before returning the first recovery log file to the primary server according to the log obtaining request, the method further includes:

Receiving total write status information sent by the client, where the total write status information includes second write status information returned from the server to the client and first write status information returned by the primary server to the client;

If the total write status information sent by the client includes the write failure information, it is determined that the data sent by the client is the data to be restored, and the first recovery log file that records the location information and the operation information of the data to be restored is generated.

203: Receive the filtered recovery log file sent by the primary server, and perform operations on the restored data according to the operation information in the filtered recovery log file.

The method provided by the embodiment of the present invention, after the primary server sends a log obtaining request to each slave server in the storage system, according to the first recovery log file returned by each slave server, and generated according to all the first recovery log files and the primary server. The second recovery log file recovers the data to be recovered, thereby eliminating the need to scan all the data and recovering the data without using a metadata server, thereby improving the efficiency of restoring data.

Embodiment 2

The embodiment of the present invention provides a method for restoring data. For ease of understanding, the data processing method provided by the embodiment of the present invention is explained in detail by taking the storage system as an example in combination with the content of the first embodiment. The storage system includes a primary server and at least two secondary servers. The primary server is the first server that successfully stores data, and the secondary server is a server other than the primary server in the storage system. Referring to FIG. 3, the method process provided by this embodiment includes:

301: The primary server and the secondary server respectively receive and store data sent by the client, and the primary server returns, to the client, first write status information including the identification identifier of the data, and returns a second identifier containing the identification identifier of the data from the server to the client. Write status information, the first write status information and the second write status information include write success information or write failure information;

In order to enable the primary server and the secondary server to receive the data sent by the client separately, the client obtains the data in advance. The manner in which the client obtains data is not specifically limited in this embodiment. During specific implementation, the user can write data to the client through the APP (Application), and then the client sends the data to each server. Of course, in addition to the above manner of receiving data, other methods may be used, which are not specifically limited in this embodiment.

As an alternative embodiment, the primary server and the secondary server store data through interaction with the client. The interaction process between the primary server and the secondary server and the client adopts a two-phase transaction submission method, which is a preparation phase and a submission phase. In the preparation phase, the primary server stores the data sent by the client, and returns the first write status information containing the identification identifier of the data to the client, stores the data sent by the client from the server, and returns the identification identifier containing the data to the client. The second write status information, that is, step 301 corresponds to the preparation phase.

For ease of understanding, the interaction process shown in FIG. 4 is taken as an example for description. There are three servers in the storage system, namely server 1, server 2 and server 3. The user writes the data 1 to the client through the APP, the client creates a transaction, and sends the data 1 written by the user to the server 1 to the server 3 respectively. Server 1 to server 3 respectively receive and store data 1 sent by the client, that is, server 1 to server 3 store the same data 1; server 1 and server 2 store data 1 successfully, server 3 fails to store data 1; server 1 and server 2 Returning the write status information including the identifier of the data 1 to the client, the write status information is the write success information; the server 3 returns the write status information including the identifier of the data 1 to the client, where the write status information is a write failure information. In addition, since the server 1 is the first server that successfully stores the data 1, the server 1 is used as the primary server, and the write status information returned by the server 1 to the client is the first write status information; The server other than the primary server, that is, the server 2 and the server 3 are used as the secondary server, and the write status information returned by the server 2 and the server 3 to the client is used as the second write status information.

302: The primary server and the secondary server respectively receive total write status information sent by the client, and the total write status information includes first write status information returned by the primary server to the client, and second return from each server to the client. Write status information;

Since the primary server returns the first write status information including the identification identifier of the data to the client, the second write status information including the identification identifier of the data is returned from the server to the client, and the client receives the information sent by each server. Writing the total write status information of the status information and the second write status information, and transmitting the total write status information including the write failure information to the server whose write status information is the write success information, thereby causing each server Ability to receive the total write status information sent by the client.

As an optional embodiment, in the commit phase of the interaction process between the server and the client, the primary server and the secondary server respectively receive the total write status information sent by the client, that is, step 302 corresponds to the submission phase.

For ease of understanding, the interaction process shown in FIG. 4 is still taken as an example for description. There are three servers in the storage system, namely server 1, server 2 and server 3. Since the write status information returned by the server 1 and the server 2 to the client is the write success information; the write status information returned by the server 3 to the client is the write failure information, then in the commit phase, the client goes to the server 1, the server 2 and the server 3 sends the total write status information, the total write status information includes the data 1 write failure information, and the server 1, the server 2, and the server 3 receive the total write status information sent by the client.

303: When the total write status information sent by the client includes the write failure information, the primary server and the secondary server respectively determine that the data sent by the client is data to be restored, and the primary server generates location information and operation information for recording the data to be restored. a second recovery log file, and a first recovery log file for recording location information of the data to be restored and operation information from the server;

When the total write status information sent by the client includes the write failure information, it indicates that at least one server fails to store data. In order to recover data and reduce data loss, the primary server and the secondary server respectively determine that the data sent by the client is the data to be restored. .

It should be noted that the location information of the data to be recovered recorded in the recovery log file may be offset information, data length information, etc., used to determine the location of the data to be restored; the operation information may be writing data information, deleting data information. Etc., used to describe the operation of the data, this embodiment does not restore the log The location information of the data to be recovered recorded in the file and the content of the operation information are specifically limited.

For ease of understanding, the interaction process shown in FIG. 4 is still taken as an example for description. There are three servers in the storage system, namely server 1, server 2 and server 3. Since the server 1, the server 2, and the server 3 receive the total write status information sent by the client including the write failure information, the server 1, the server 2, and the server 3 determine that the data 1 is the data to be restored, and generates the record write data. Recovery log file for location information and operational information. For example, if the server 1 is a master server and the server 2 and the server 3 are slave servers, the server 1 generates a second recovery log file that records location information and operation information of the data to be restored. The server 2 and the server 3 record the error log through the data brush, that is, generate the first recovery log file that records the location information of the data to be restored and the operation information.

The interaction process between the client and each server reflects the process of storing data and generating recovery log files for each server. For the specific process of restoring data, refer to the following steps.

304: When any one of the storage systems fails to store data, the primary server sends a log obtaining request to each of the secondary servers in the storage system, and the log obtaining request carries the data identifier to be restored.

The manner of triggering the primary server to send a log obtaining request to each of the secondary servers in the storage system is not specifically limited in this embodiment. During specific implementation, if any server in the storage system stores data fails, after the log file to be restored is generated, the server periodically checks the recovery log file and starts data recovery, thereby triggering the primary server to send to each slave server in the storage system. The log gets the request. Of course, in addition to the above methods, other methods can be employed.

For ease of understanding, the server 1 and the server 2 still store the data 1 successfully, and the server 3 stores the data 1 failure as an example. Among them, the server 1 is the main server. The server 1 to the server 3 periodically check the recorded recovery log file, and it is found that the data 1 needs to be restored by checking. A log acquisition request carrying the data identification of the data 1 is transmitted by the server 1 to the server 2 and the server 3.

305: Receive a log obtaining request sent by the primary server from the server, where the log obtaining request carries the data identifier to be restored;

The slave server can directly receive the log obtaining request sent by the master server to the slave server in the storage system. The slave server can directly receive the log obtaining request sent by the master server. The recovery log file is returned according to the to-be-recovered data identifier in the log obtaining request.

For ease of understanding, the server 1 and the server 2 still store the data 1 successfully, and the server 3 stores the data 1 failure as an example. Among them, the server 1 is the main server. Server 2 and server 3 Receiving a log obtaining request sent by the server 1, where the log obtaining request carries the data identifier of the data 1.

306: The slave server returns a first recovery log file to the primary server according to the log obtaining request, where the first recovery log file records location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier.

The log obtaining request carries the to-be-recovered data identifier, and the slave server can search for the recovery log file corresponding to the to-be-recovered data identifier in the recorded recovery log file according to the to-be-recovered data identifier, and return the found recovery log to the primary server. file. The manner in which the recovery log file is searched from the server is not specifically limited in this embodiment. During the specific implementation, the recorded recovery log file may be sequentially read, and the data identifier in the read recovery log file is compared with the to-be-recovered data identifier carried in the log obtaining request; if the recovered recovery log file is read The data identifier is the same as the to-be-recovered data identifier carried in the log obtaining request. The recovered log file is used as the first recovered log file and returned to the primary server. Of course, in addition to the above, other methods can be used.

For ease of understanding, the server 1 and the server 2 still store the data 1 successfully, and the server 3 stores the data 1 failure as an example. Among them, the server 1 is the main server. The server 2 and the server 3 sequentially read the recovery log file. If the data identifier in the recovered recovery log file 1 is the same as the identifier of the data 1, the recovery log file 1 is used as the first recovery log file found, and Server 1 returns recovery log file 1.

307: The primary server receives the first recovery log file returned by each slave server according to the log obtaining request, and each recovery log file records location information and operation information of the data to be restored.

Since the first recovery log file is returned from the server according to the log acquisition request to the primary server, the primary server can directly receive the first recovery log file returned by each secondary server according to the log acquisition request.

For ease of understanding, the server 1 and the server 2 still store the data 1 successfully, and the server 3 stores the data 1 failure as an example. Among them, the server 1 is the main server. The server 1 receives the first recovery log file corresponding to the data 1 identifier returned by the server 2 and the server 3 according to the log acquisition request, and each of the first recovery log files records the location information and the operation information of the data 1.

308: The primary server operates, according to the latest operation information in the operation information recorded by the first recovery log file and the second recovery log file, the second recovery log file is generated by the primary server, and the data identifier to be restored is recorded. Location information and operation information of the identified data to be recovered;

Specifically, since each of the first recovery log file and the second recovery log file records the location information and the operation information of the data to be restored, the server may find the to-be-recovered according to the location information of the data to be restored in each recovery log file. The data is processed according to the latest operation information in the operation information recorded in each recovery log file, thereby realizing data recovery for the data to be restored.

However, since all the first recovery log files and the second recovery log files acquired by the primary server may have the same operation for the same to-be-recovered data, it is not necessary to perform repeated data recovery according to the repeated recovery log files. Operation, thereby saving time for restoring data. The method provided in this embodiment includes, but is not limited to, operating on the data to be restored according to the latest operation information in the operation information recorded by all the first recovery log files and the second recovery log file. :

(a1) sequentially reading each of the first recovery log file and the second recovery log file;

(a2) determining whether the index information of the currently read recovery log file is in a preset structure, the index information includes a recovery log file identifier, an offset information, and a data length information, and the preset structure is configured to store the index information and the operation time; If the index information of the currently read recovery log file is not in the preset structure, execute (a3); if the index information of the currently read recovery log file is in the preset structure, execute (a4);

(a3) adding the index information and operation time of the currently read recovery log file to the preset structure, and executing (a6);

(a4) comparing the operation time of the currently read recovery log file with the operation time of the index information corresponding to the currently read recovery log file in the preset structure; if the currently read recovery log file is operated later than Setting the operation time of the index information corresponding to the currently read recovery log file in the structure is performed (a5); if the currently read recovery log file is operated earlier than the preset structure and the currently read recovery log file The operation time corresponding to the index information is executed (a6);

(a5) Replace the operation time of the currently read recovery log file with the operation time corresponding to the index information of the currently read recovery log file in the preset structure, and execute (a6).

(a6) continue to read the next recovery log file, process the next recovery log file read in the same manner as the currently read recovery log file, until the last recovery log file is processed, and execute (a7);

(a7) obtaining a recovery log file corresponding to the index information and the operation time in the preset structure, and using the recovery log file corresponding to the index information and the operation time in the preset structure as the filtered recovery log file, according to the screening Operation information in the recovery log file to operate on the recovered data Work.

It should be noted that, before restoring the data to be restored according to all the first recovery log files and the second recovery log file, the preset structure is an empty structure. In the process of restoring the data to be restored, the index information and the operation time of the recovery log file that meet the above comparison and comparison requirements are added to the preset structure, so that the preset structure includes the index information and operation of the filtered recovery log file. The time, that is, the obtained preset structure is a preset structure in which the information without the repeated index and the operation time is stored, that is, the index information stored in the preset structure and the operation time do not overlap. Therefore, the recovery log file determined according to the preset structure will be a recovery log file with no repetitive operations. By filtering the recovery log file by using the preset structure, the efficiency of the recovery log file filtering is improved, and the workload of restoring the data to be restored according to the filtered recovery log file in the subsequent steps is reduced, and the speed of recovering the data is accelerated. The specific form of the preset structure is not specifically limited in this embodiment. In a specific implementation, the preset structure includes, but is not limited to, a hash table, a red black tree, and the like.

For the sake of understanding, the server 1 is the main server, the first recovery log file 21 and the first recovery log file 22 returned by the server 2, and the first recovery log file 31 returned by the server 3 are taken as an example for description. The first recovery log file 21 records the write data 1, the index information is a1+a2+a3, the operation time is December 12, 2013, 13:22; the first recovery log file 22 records the deletion data 1, the index information For a1+a2+a3, the operation time is December 12, 2013, 13:49; the first recovery log file 31 records the write data 1, the index information is a1+a2+a3, and the operation time is December 2013. 12, 13:22. Where a1 is the recovery log file identifier, a2 is the offset information, and a3 is the data length information.

In addition, the server 1 generates a second recovery log file 11 and a second recovery log file 12; the second recovery log file 11 records the written data 1, the index information is a1+a2+a3, and the operation time is December 12, 2013. , 13:22; the second recovery log file 12 records the deletion data 1, the index information is a1+a2+a3, and the operation time is December 12, 2013, 13:49.

The server 1 reads the second recovery log file 11. Since the recovery log file filtering has not been performed at this time, the preset structure is an empty structure, and the index information a1+a2+a3 of the second recovery log file 11 and the operation time 2013 are directly directly. On December 12th, 13:22 is added to the preset structure; continue to read the next second recovery log file 12, and judge that the second recovery log file 12 index information a1+a2+a3 is in the preset structure, Operation time of the second recovery log file 12 on December 12, 2013, 13:49 and the operation information of the index information a1+a2+a3 of the second recovery log file 11 in the preset structure, December 12, 2013, 13: 22 to compare; by comparison, the second recovery log file 12 operation time 2013 On December 12, 2013, 13:49 nights in the default structure, the second recovery log file 11 index information a1+a2+a3 operation time December 12, 2013, 13:22, the second recovery log file Operation time of 12, December 12, 2013, 13:49 replaces the operation time corresponding to the index information a1+a2+a3 of the second recovery log file 11 in the preset structure on December 12, 2013, 13:22, The replaced preset structure includes the index information a1+a2+a3 and the operation time corresponding to the index information a1+a2+a3 on December 12, 2013, 13:49.

Continue to read the next first recovery log file 21, determine that the index information a1+a2+a3 of the first recovery log file 21 is in the preset structure, and then operate the first recovery log file 21 on December 12, 2013. 13:22 is compared with the operation time of the index information a1+a2+a3 in the preset structure on December 12, 2013, 13:49; by comparison, the operation time of the first recovery log file 21 is December 12, 2013. On the day of 13:22, the operation time corresponding to the index information a1+a2+a3 in the preset structure is December 12, 2013, 13:49, indicating that the first recovery log file 21 belongs to the period of server aging, which occurs. Overlapping recovery log files, that is, multiple recovery log files operate with the same data. The content of the recovery log file operation with a late operation time will overwrite the contents of the recovery log file operation operation time earlier. In this case, it is necessary to perform the operation time later. The log file can be recovered, and it is not necessary to perform an operation log recovery file early. At this time, the operation time corresponding to the index information and the index information in the preset structure is unchanged, that is, the operation information corresponding to the index information a1+a2+a3 and the index information a1+a2+a3 is December 12, 2013. 13:49. For example, in the actual application scenario, the client writes a piece of data A to generate a recovery log file; after a period of time, the client deletes a piece of data A previously written, and generates a recovery log file. If the two recovery log files are executed while the data is being restored, an operation of writing a piece of data A and deleting a piece of data A will occur, and the operation of writing a piece of data A is useless. Therefore, you only need to delete a piece of data A.

In order to filter the recovery log file, continue to read the next first recovery log file 22, and determine that the index information a1+a2+a3 of the first recovery log file 22 is in the preset structure, then the operation time of the first recovery log file 22 is On December 12, 2013, 13:49 is compared with the operation time of the index information a1+a2+a3 in the preset structure on December 12, 2013, 13:49; by comparison, the first recovery log file 22 is Operation time December 12, 2013, 13:49 The operation time corresponding to the index information a1+a2+a3 in the preset structure is the same on December 12, 2013, 13:49, and continues to read the next first. Restore the log file. At this time, the operation time corresponding to the index information and the index information in the preset structure is unchanged, that is, the operation information corresponding to the index information a1+a2+a3 and the index information a1+a2+a3 is December 12, 2013. 13:49.

The first recovery log file 31 is read, and the index information a1+a2+a3 of the first recovery log file 31 is determined to be in the preset structure, and the operation time of the first recovery log file 31 is December 12, 2013, 13: 22 is compared with the operation time of the index information a1+a2+a3 in the preset structure on December 12, 2013, 13:49; by comparison, the operation time of the first recovery log file 31 is December 12, 2013, 13:22 The operation time corresponding to the index information a1+a2+a3 in the preset structure is December 12, 2013, 13:49, and the next first recovery log file is continuously read. So far, the received first recovery log files 21, 22, 31 and the second recovery log files 11, 12 generated by the server 1 corresponding to the data 1 identification have been processed. At this time, the preset structure includes the index information a1+a2+a3 and the operation time corresponding to the index information a1+a2+a3 on December 12, 2013, 13:49.

As an optional embodiment, the recovery log file corresponding to the index information and the operation time in the preset structure is obtained in (a7), including but not limited to:

(a71) sequentially reading the first recovery log file and the second recovery log file corresponding to the to-be-recovered data identifier, and comparing the index information of the currently read recovery log file with the index information in the preset structure. If the index information in the preset structure is the same as the index information of the currently read recovery log file, execute (a72); if there is no index in the preset structure that is the same as the index information of the currently read recovery log file; Information is then executed (a74);

(a72) comparing the operation time of the currently read recovery log file with the operation time of the index information corresponding to the currently read recovery log file in the preset structure; if the operation of the currently-recovered recovery log file exists in the preset structure The operation time of the same time corresponding to the index information of the currently read recovery log file is executed (a73); if the preset structure does not have the same operation time as the currently read recovery log file, the current read recovery is the same. The operation time of the index information of the log file is executed (a74);

(a73) adding the currently read recovery log file to the filtered recovery log file and executing (a74);

(a74) Read the next recovery log file; and so on, until the last recovery log file is compared, and the recovery log file corresponding to each index information in the preset structure is obtained.

For ease of understanding, the operation time corresponding to the index information a1+a2+a3 and the index information a1+a2+a3 in the preset structure is December 12, 2013, 13:49 as an example. The second recovery log file 11 is read, and the index information a1+a2+a3 of the second recovery log file 11 is compared with the index information a1+a2+a3 in the preset structure, and it is known that the preset structure exists and the second Restoring the same index information of the index information a1+a2+a3 of the log file 11; continuing to compare the operations of the second recovery log file 11 At the time of December 12, 2013, 13:22 and the operation time of the index information a1+a2+a3 in the preset structure, December 12, 2013, 13:49, by comparison, it is known that the preset structure does not exist and The operation time of the recovery log file 11 is the same operation time on December 12, 2013, 13:22, and the next recovery log file is continuously read.

The second recovery log file 12 is read, and the index information a1+a2+a3 of the second recovery log file 12 is compared with the index information a1+a2+a3 in the preset structure, and it is known that the preset structure exists and the second Restoring the same index information of the index information a1+a2+a3 of the log file 12; continuing to compare the operation time of the second recovery log file 12 on December 12, 2013, 13:49 with the index information a1+a2+ in the preset structure The operation time of a3 is December 12, 2013, 13:49. By comparison, it is known that there is the same operation time in the preset structure as the operation time of the recovery log file 12, December 12, 2013, 13:49, the current reading will be The second recovery log file 12 is added to the filtered recovery log file and continues to read the next recovery log file. By analogy, the index information and the operation time of the first recovery log files 21, 22, and 31 are compared with the index information and the operation time in the preset structure, and the first recovery log file is not obtained. Therefore, only The second recovery log file 12 is used as the filtered recovery log file.

Since the recovery log file is filtered and the filtered recovery log file records the latest operation information of the data to be restored, the data to be restored may be operated according to the operation information recorded in the filtered recovery log file. The manner of restoring data is not specifically limited in this embodiment. During specific implementation, the data may be re-operated by the primary server according to the operation information of the data to be recovered recorded in the recovery log file. Of course, in addition to the above methods, other methods can be employed. In addition, a plurality of log information of the data to be restored may be recorded in a recovery log file. For this case, each log information of the data to be restored may be sequentially read by the primary server, and according to each log information of the data to be restored. The operational information re-operates on the data.

For ease of understanding, the filtered log file is a recovery log file 12 as an example. The operation of deleting the data 1 is recorded in the recovery log file 12, and the server 1 can perform the operation of deleting the data 1 to complete the data recovery.

309: The primary server sends the filtered recovery log file to each slave server, and each slave server operates the data to be restored according to the operation information in the filtered recovery log file.

The method provided in this embodiment further includes, but is not limited to, the primary server sending the filtered recovery log file to each slave server, in order to improve the speed of restoring data in the storage system. In this way, each slave server is based on the filtered recovery log. The file is restored in parallel with the recovered data, so that the data can be recovered without the metadata server, which not only reduces the load of the metadata server, but also improves the speed of data recovery.

For ease of understanding, the filtered recovery log file is the second recovery log file 12 as an example. The server 1 transmits the second recovery log file 12 to the server 2 and the server 3, and the server 2 and the server 3 restore the data 1 based on the second recovery log file 12.

As an optional embodiment, the method provided in this embodiment further includes, but is not limited to, the primary server deleting all the first recovery log files and the second recovery log files, and notifying each of the secondary servers to delete all the first recovery log files and the first Second recovery log file;

Since the primary server has completed data recovery, there is no need to record the recovery log file for the recovered data, and in order to save the storage resources in the primary server, the primary server deletes all the first recovery log files and the second recovery log files.

Similarly, in order to save storage resources from the server, the primary server will notify each slave server to delete all the first recovery log files and the second recovery log files for recovering the data to be restored after successfully recovering the data. Regarding the manner of notifying the slave server, the embodiment is not specifically limited. In a specific implementation, the slave server may be notified by sending a delete message to the slave server. Of course, in addition to the above methods, other methods can be employed.

310: Receive, from the server, the filtered recovery log file sent by the primary server, and operate the restored data according to the operation information in the filtered recovery log file.

After the filtered recovery log file records the location information and the operation information of the data to be restored, after receiving the filtered recovery log file sent by the primary server, the slave server may perform the recovery data according to the filtered recovery log file. restore. For the manner of restoring data, refer to the manner in which the primary server recovers the data to be restored according to the filtered recovery log file in the foregoing step 308, and details are not described herein again.

As an optional embodiment, the method provided in this embodiment further includes, but is not limited to, receiving, from the server, a notification sent by the primary server to delete all the first recovery log files and the second recovery log file, and deleting all the first recovery according to the notification. The log file and the second recovery log file.

Since the primary server notifies each of the secondary servers to delete all the first recovery log files and the second recovery log files, the secondary server may receive a notification sent by the primary server to delete all the first recovery log files and the second recovery log files, thereby triggering the notification according to the notification. After successfully recovering the data, delete all the first recovery log files and the second recovery log files.

The method provided in this embodiment sends the date to each slave server in the storage system through the primary server. Acquiring the request, receiving the first recovery log file returned by each slave server, and restoring the data to be recovered to recover the data to be recovered according to all the first recovery log files and the second recovery log file generated by the main server, thereby eliminating the need to scan all data, Recover data without using a metadata server, improving the efficiency of restoring data.

Embodiment 3

Referring to FIG. 5, an embodiment of the present invention provides a server for recovering data, which is a primary server applied to a storage system, where the storage system includes a primary server and a secondary server, and both the primary server and the secondary server are used to store the same data. The server is the first server to successfully store data, and the server includes:

The first sending module 401 is configured to send a log obtaining request to each slave server in the storage system when the storage server fails to store data in the storage system, where the log obtaining request carries the data identifier to be restored;

The first receiving module 402 is configured to receive a first recovery log file returned by each slave server according to the log obtaining request, where each first recovery log file records location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier;

The recovery module 403 is configured to perform operation on the restored data according to the latest operation information in the operation information recorded by all the first recovery log files and the second recovery log file, and the data to be restored is searched according to the location information of the data to be restored, and the second The recovery log file is generated by the primary server, and records the location information and operation information of the data to be recovered identified by the data identifier to be restored.

Optionally, the server further includes:

a storage module, configured to receive and store data sent by the client;

a returning module, configured to return, to the client, first write state information that includes an identification identifier of the data, where the first write state information includes a write success information or a write failure information;

a second receiving module, configured to receive total write status information sent by the client, where the total write status information includes first write status information returned by the primary server to the client, and second write returned from the server to the client. Status information;

a generating module, configured to: when the total write status information sent by the client includes the write failure information, determine that the data sent by the client is the data to be restored, and generate a second recovery of the location information and the operation information for recording the data to be restored. Log file.

Optionally, the recovery module 403 includes:

The determining unit is configured to determine whether the index information of the currently read recovery log file is in a preset structure, the index information includes a recovery log file identifier, an offset information, and a data length information, and the preset structure stores the index information and the operation time. ;

Adding a unit, when the index information of the currently read recovery log file is not in the preset structure, adding the index information and the operation time of the currently read recovery log file to the preset structure;

The obtaining unit is configured to obtain a recovery log file corresponding to the index information and the operation time in the preset structure, and use the recovery log file corresponding to the index information and the operation time in the preset structure as the filtered recovery log file;

Optionally, the recovery module 403 further includes:

a comparison unit, configured to: when the index information of the currently read recovery log file is in the preset structure, the operation time of the currently read recovery log file and the index information of the currently read recovery log file in the preset structure Comparison of operating time;

Optionally, the obtaining unit includes:

a first reading subunit, configured to sequentially read the first recovery log file and the second recovery log file corresponding to the to-be-recovered data identifier;

Comparison subunit for indexing information in the default structure that exists with the currently read recovery log file When the same index information is used, the operation time of the currently read recovery log file and the operation time of the index information corresponding to the currently read recovery log file in the preset structure are compared;

Adding a subunit, when the operation time corresponding to the index information of the currently read recovery log file is the same as the operation time of the currently read recovery log file in the preset structure, adding the currently read recovery log file To the filtered recovery log file;

The second read subunit is configured to read the next recovery log file; and so on until the last recovery log file is compared, and the recovery log file corresponding to each index information in the preset structure is obtained.

Optionally, the server further includes:

The second sending module is configured to send the filtered recovery log file to each slave server, and each slave server recovers the data to be restored according to the filtered recovery log file.

Optionally, the server further includes:

Deleting a module for deleting all first recovery log files and second recovery log files;

The notification module is configured to notify each of the slave servers to delete all the first recovery log files and the second recovery log files.

The server provided by the embodiment of the present invention receives the first recovery log file returned by each slave server by sending a log acquisition request to each slave server in the storage system, and generates a second recovery according to all the first recovery log files and the primary server. The log file recovers the data to be recovered, so that it is not necessary to scan all the data, and the metadata server can be used to recover the data, which improves the efficiency of restoring data.

Embodiment 4

Referring to FIG. 6, an embodiment of the present invention provides a server for recovering data, which is a slave server applied to a storage system, where the storage system includes a primary server and a secondary server, and both the primary server and the secondary server are used to store the same data. The server is the first server to successfully store data, and the server includes:

The first receiving module 501 is configured to receive a log obtaining request sent by the primary server, where the log obtaining request carries the data identifier to be restored;

The first returning module 502 is configured to return a first recovery log file to the primary server according to the log obtaining request, so that the primary server recovers the to-be-recovered data according to the first recovery log file and the second recovery log file generated by the primary server;

The second receiving module 503 is configured to receive the filtered recovery log file sent by the primary server.

The recovery module 504 is configured to perform operation on the restored data according to the operation information recorded by the filtered recovery log file;

Optionally, the server further includes:

a storage module, configured to receive and store data sent by the client;

a second returning module, configured to return, to the client, second writing state information that includes an identification identifier of the data, where the second writing state information includes a writing success information or a writing failure information;

a third receiving module, configured to receive total write status information sent by the client, where the total write status information includes second write status information returned from the server to the client, and a first write status returned by the primary server to the client information;

a generating module, configured to: when the total write status information sent by the client includes the write failure information, determine that the data sent by the client is the data to be restored, and generate the first recovery of the location information and the operation information of the data to be restored. Log file.

Optionally, the server further includes:

a fourth receiving module, configured to receive a notification sent by the primary server to delete all the first recovery log files and the second recovery log files;

The deleting module is configured to delete all the first recovery log files and the second recovery log files according to the notification.

The server provided in this embodiment receives the log obtaining request sent by the primary server, and returns a first recovery log file to the primary server according to the log obtaining request, so that the primary server generates the second recovery log according to the first recovery log file and the primary server. The file recovers the data to be recovered, so that it is not necessary to scan all the data, and the metadata server can be used to recover the data, which improves the efficiency of restoring data.

Embodiment 5

Referring to FIG. 7, an embodiment of the present invention provides a system for recovering data, where the system includes a main server 701 and at least one slave server 702. The master server 701 is a server for recovering data provided by the foregoing third embodiment. The server 702 is the server for restoring data provided in the above-mentioned fourth embodiment. For details, refer to the contents of the third and fourth embodiments, and details are not described herein again.

The system provided by this embodiment sends the date to each slave server in the storage system through the primary server. Acquire the request, receive the first recovery log file returned from each server, and recover the data to be restored according to all the first recovery log files and the second recovery log file generated by the primary server, thereby eliminating the need to scan all data and use metadata The server can recover data and improve the efficiency of data recovery.

Embodiment 6

The embodiment of the present invention provides a server for recovering data, which is a primary server applied to a storage system, where the storage system includes a primary server and a secondary server, and the primary server and the secondary server are both used to store the same data, and the primary server is the first server. a server that successfully stores data, the server including a transmitter, a receiver, and a processor;

The transmitter is configured to send a log obtaining request to each slave server in the storage system when the server fails to store data in the storage system, and the log obtaining request carries the data identifier to be restored.

a receiver, configured to receive a first recovery log file returned by each slave server according to the log obtaining request, where each first recovery log file records location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier;

a processor, configured to perform, according to the latest operation information in the operation information recorded by the first recovery log file and the second recovery log file, the data to be restored is searched according to the location information of the data to be restored, and the second recovery is performed. The log file is generated by the primary server, and records the location information and operation information of the data to be recovered identified by the data identifier to be restored.

As an optional embodiment, the processor is further configured to receive and store data sent by the client;

The transmitter is further configured to return, to the client, first write status information that includes an identification identifier of the data, where the first write status information includes write success information or write failure information;

The receiver is further configured to receive total write status information sent by the client, where the total write status information includes first write status information returned by the primary server to the client, and second write each returned from the server to the client. status information;

The processor is further configured to: when the total write status information sent by the client includes the write failure information, determine the data as the data to be restored, and generate a second recovery log file that records the location information and the operation information of the data to be restored.

As an optional embodiment, the processor is configured to sequentially read each of the first recovery log file and the second recovery log file, and determine whether the index information of the currently read recovery log file is in a preset structure. The index information includes a recovery log file identifier, an offset information, and a data length information, and the preset structure is used to store the index information and the operation time; when the index information of the currently read recovery log file is not in the preset structure, Adding the index information and operation time of the currently read recovery log file to the preset structure; continuing to read the next recovery log file; processing the next recovery log file read in the same manner as processing the currently read recovery log file Until the last recovery log file is processed, obtaining a preset structure for storing the duplicate index information and the operation time; acquiring the recovery log file corresponding to the index information and the operation time in the preset structure, and the preset structure is The index information and the recovery log file corresponding to the operation time are used as the filtered recovery log file, and the data to be restored is operated according to the operation information in the filtered recovery log file.

As an optional embodiment, the processor is further configured to: when the index information of the currently read recovery log file is in the preset structure, compare the operation time of the currently read recovery log file with the current current in the preset structure. The operation time of the index information of the read recovery log file is compared; when the operation time of the currently read recovery log file is later than the operation time of the index information corresponding to the currently read recovery log file in the preset structure, the current The operation time of the read recovery log file replaces the operation time in the preset structure corresponding to the index information of the currently read recovery log file; and the step of continuing to read the next recovery log file is performed.

As an optional embodiment, the processor is configured to sequentially read the first recovery log file and the second recovery log file corresponding to the to-be-recovered data identifier; and sequentially, the index information of the currently read recovery log file is The index information in the structure is compared; when there is the same index information in the preset structure as the index information of the currently read recovery log file, the operation time of the currently read recovery log file is compared with the preset structure. The operation time of the index information of the currently read recovery log file; when there is an operation time corresponding to the operation time of the currently read recovery log file in the preset structure corresponding to the index information of the currently read recovery log file, The currently read recovery log file is added to the filtered recovery log file; the next recovery log file is read; and so on, until the last recovery log file is compared, and each index information in the preset structure is obtained. Recovery log file.

As an optional embodiment, the transmitter is further configured to send the filtered recovery log file to each slave server, and each slave server recovers the data to be restored according to the filtered recovery log file.

As an optional embodiment, the processor is further configured to delete all the first recovery log files and the second recovery log files; notify each of the slave servers to delete all the first recovery log files and the second recovery log. file.

The server provided by the embodiment of the present invention receives the first recovery log file returned by each slave server in the storage system according to the log obtaining request sent by the primary server, and according to all the first recovery log files and the second recovery log generated by the primary server. The file recovers the data to be recovered, so that it is not necessary to scan all the data, and the metadata server can be used to recover the data, which improves the efficiency of restoring data.

Example 7

The embodiment of the invention provides a server for recovering data, the server is a slave server applied to the storage system, the storage system includes a master server and a slave server, and the master server and the slave server are both used to store the same data, and the master server is the first server. a server that successfully stores data, the server including a transmitter, a receiver, and a processor;

The receiver is configured to receive a log obtaining request sent by the primary server, where the log obtaining request carries the data identifier to be restored;

a transmitter, configured to return a first recovery log file to the primary server according to the log obtaining request, so that the primary server recovers the to-be-recovered data according to the first recovery log file and the second recovery log file generated by the primary server;

The receiver is further configured to receive the filtered recovery log file sent by the primary server;

a processor, configured to perform operation on the restored data according to the operation information recorded by the filtered recovery log file;

Each of the first recovery log file and the second recovery log file records location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier.

The transmitter is further configured to return, to the client, second write status information that includes an identification identifier of the data, where the second write status information includes write success information or write failure information;

The receiver is further configured to receive total write status information sent by the client, where the total write status information includes second write status information returned from the server to the client, and first write status information returned by the primary server to the client. ;

The processor is further configured to: when the total write status information sent by the client includes the write failure information, determine that the data sent by the client is the data to be restored, and generate the first location information and operation information for recording the data to be restored. Restore the log file.

As an optional embodiment, the receiver is further configured to receive all the first deletions sent by the primary server. Notification of recovery log file and second recovery log file;

The processor is further configured to delete all the first recovery log files and the second recovery log files according to the notification.

The server provided by the embodiment of the present invention receives the log obtaining request sent by the primary server, and returns a first recovery log file to the primary server according to the log obtaining request, so that the primary server generates the second recovery according to the first recovery log file and the primary server. The log file recovers the data to be recovered, so that it is not necessary to scan all the data, and the metadata server can be used to recover the data, which improves the efficiency of restoring data.

It should be noted that, when recovering data, the server for restoring data provided by the foregoing embodiment is only illustrated by dividing the foregoing functional modules. In actual applications, the function allocation may be completed by different functional modules as needed. The internal structure of the server is divided into different functional modules to complete all or part of the functions described above. In addition, the server for restoring data and the method for restoring data provided by the foregoing embodiments are in the same concept, and the specific implementation process is described in detail in the method embodiment, and details are not described herein again.

The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

A person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium. The storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.

The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalents, improvements, etc., which are within the spirit and scope of the present invention, should be included in the protection of the present invention. Within the scope.

Claims

A method for restoring data, the method being applied to a primary server of a storage system, the storage system comprising a primary server and a secondary server, the primary server and the secondary server are both configured to store the same data, The primary server is the first server that successfully stores data, and the method includes:

When the storage server fails to store the data in the storage system, the primary server sends a log acquisition request to each slave server in the storage system, where the log acquisition request carries the data identifier to be restored;

Receiving, by the respective slave server, the first recovery log file returned according to the log obtaining request, where each first recovery log file records location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier;

And operating the restored data according to the latest operation information in the operation information recorded by the first recovery log file and the second recovery log file, where the to-be-recovered data is searched according to the location information of the to-be-recovered data, The second recovery log file is generated by the primary server, and records location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier.
The method according to claim 1, wherein before the sending of the log obtaining request to each slave server in the storage system, the method further comprises:

Receiving and storing data sent by the client, and returning, to the client, first write status information that includes the identification identifier of the data, where the first write status information is write success information or write failure information;

Receiving total write status information sent by the client, where the total write status information includes first write status information returned by the primary server to the client, and each slave server returns to the client Second write status information;

If the total write status information sent by the client includes the write failure information, determining that the data sent by the client is the data to be restored, and generating the second location information and the operation information for recording the data to be restored. Restore the log file.
The method of claim 1 wherein said first according to all of said first The latest operation information in the operation information recorded in the recovery log file and the second recovery log file is to be operated on the restored data, including:

Reading each of the first recovery log file and the second recovery log file in sequence;

Determining whether the index information of the currently read recovery log file is in a preset structure, the index information includes a recovery log file identifier, an offset information, and data length information, where the preset structure is used to store index information and operation time. ;

If the index information of the currently read recovery log file is not in the preset structure, adding the index information and the operation time of the currently read recovery log file to the preset structure;

Continue to read the next recovery log file, and process the next recovery log file read in the manner of processing the currently read recovery log file until the last recovery log file is processed, and whether the storage has duplicate index information and operation time is obtained. Preset structure;

Obtaining a recovery log file corresponding to the index information and the operation time in the preset structure, and using the recovery log file corresponding to the index information and the operation time in the preset structure as the filtered recovery log file, according to the screening The operation information in the recovery log file is operated on the data to be restored.
The method according to claim 3, wherein the determining whether the index information of the currently read recovery log file is in the preset structure further comprises:

If the index information of the currently read recovery log file is in the preset structure, the operation time of the currently read recovery log file and the operation time of the index information corresponding to the currently read recovery log file in the preset structure Compare;

If the operation time of the currently read recovery log file is later than the operation time of the index information corresponding to the currently read recovery log file in the preset structure, the operation time of the currently read recovery log file is replaced with the preset structure and The operation time corresponding to the index information of the currently read recovery log file, and the step of continuing to read the next recovery log file.
The method according to claim 3 or 4, wherein the obtaining a recovery log file corresponding to the index information and the operation time in the preset structure comprises:

And sequentially reading the first recovery log file and the second recovery log file corresponding to the to-be-recovered data identifier, and sequentially comparing the index information of the currently read recovery log file with the index information in the preset structure. Correct;

If the index information of the recovery log file that is currently read is the same as the index information of the currently read recovery log file, compare the operation time of the currently read recovery log file with the currently read recovery log in the preset structure. The operation time of the index information of the file;

If the operation time of the index information corresponding to the currently read recovery log file is the same as the operation time of the currently read recovery log file in the preset structure, the currently read recovery log file is added to the filtered Restore the log file and read the next recovery log file;

And so on, until the last recovery log file is compared, the recovery log file corresponding to each index information in the preset structure is obtained.
A method for restoring data, the method being applied to a primary server of a storage system, the storage system including the primary server and a secondary server, the primary server and the secondary server are both configured to store the same data The primary server is the first server that successfully stores data, and the method includes:

Receiving, by the server, a log obtaining request sent by the primary server, where the log obtaining request carries a data identifier to be restored;

Returning, to the primary server, a first recovery log file according to the log obtaining request, and causing the primary server to recover the to-be-recovered data according to the first recovery log file and the second recovery log file generated by the primary server;

Receiving the filtered recovery log file sent by the primary server, and operating the restored data according to the operation information recorded by the filtered recovery log file;

The first recovery log file and the second recovery log file respectively record location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier.
The method according to claim 6, wherein before the returning the first recovery log file to the primary server according to the log obtaining request, the method further includes:

Receiving and storing data sent by the client, and returning, to the client, second write status information including an identification identifier of the data, where the second write status information includes write success information or write failure information;

Receiving total write status information sent by the client, where the total write status information includes the second write status information returned from the server to the client, and the return of the primary server to the client First write status information;

If the total write status information sent by the client includes the write failure information, determining that the data sent by the client is the data to be restored, and generating the first location information and the operation information for recording the data to be restored. Restore the log file.
A server for recovering data, characterized in that the server is a primary server applied to a storage system, the storage system includes the primary server and a secondary server, and the primary server and the secondary server are both used to store the same Data, the primary server is the first server to successfully store data, and the server includes:

a first sending module, configured to send a log obtaining request to each of the slave servers in the storage system when the storage of the data in the storage system fails, where the log obtaining request carries the data identifier to be restored;

a first receiving module, configured to receive a first recovery log file returned by each slave server according to the log obtaining request, where each first recovery log file records location information of the to-be-recovered data identified by the to-be-recovered data identifier and Operational information;

a recovery module, configured to perform, according to the latest operation information in the operation information recorded by the first recovery log file and the second recovery log file, the data to be restored according to the location information of the data to be restored The second recovery log file is generated by the primary server, and the location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier are recorded.
The server according to claim 8, wherein the server further comprises:

a storage module, configured to receive and store data sent by the client;

a returning module, configured to return, to the client, first write state information that includes an identifier of the data, where the first write state information includes write success information or write failure information;

a second receiving module, configured to receive total write status information sent by the client, where the total write status information includes first write status information returned by the primary server to the client, and each slave server Second write status information returned to the client;

a generating module, configured to: when the total write status information sent by the client includes the write failure information, determine that the data sent by the client is the data to be restored, and generate location information that records the to-be-recovered data and The second recovery log file of the operation information.
The server according to claim 8, wherein the recovery module comprises:

a first reading unit, configured to sequentially read each of the first recovery log file and the second recovery log file;

a determining unit, configured to determine whether the index information of the currently read recovery log file is in a preset structure, where the index information includes a recovery log file identifier, offset information, and data length information, where the preset structure is used for storing Index information and operation time;

An adding unit, configured to add index information and an operation time of the currently read recovery log file to the preset structure when the index information of the currently read recovery log file is not in the preset structure;

a second reading unit, configured to continue to read the next recovery log file;

a processing unit, configured to process the read next recovery log file in a manner of processing the currently read recovery log file, until the last recovery log file is processed, to obtain a preset structure for storing the duplicate index information and the operation time;

An obtaining unit, configured to acquire a recovery log file corresponding to the index information and the operation time in the preset structure, and use the recovery log file corresponding to the index information and the operation time in the preset structure as the filtered recovery Log file

A recovery unit, configured to operate on the restored data according to the operation information in the filtered recovery log file.
The server according to claim 10, wherein the recovery module further comprises:

a comparison unit, configured to: when the index information of the currently read recovery log file is in the preset structure, the operation time of the currently read recovery log file and the correspondingly read recovery log file in the preset structure The operation time of the index information is compared;

a replacement unit, configured to replace the operation time of the currently read recovery log file when the operation time of the currently read recovery log file is later than the operation time of the index information corresponding to the currently read recovery log file in the preset structure The operation time in the preset structure corresponding to the index information of the currently read recovery log file;

The second reading unit is further configured to perform the step of continuing to read the next recovery log file.
The server according to claim 10 or 11, wherein the obtaining unit comprises:

a first reading subunit, configured to sequentially read the first recovery log file and a second recovery log file corresponding to the to-be-recovered data identifier;

The comparison subunit is configured to compare the index information of the currently read recovery log file with the index information in the preset structure;

a comparison subunit, configured to: when the index information of the recovery log file currently read is the same as the index information of the currently read recovery log file, compare the operation time of the currently read recovery log file with the preset structure The operation time of the index information of the currently read recovery log file;

Adding a subunit, configured to: when the operation time of the index information corresponding to the currently read recovery log file is the same as the operation time of the currently read recovery log file in the preset structure, the currently read recovery log The file is added to the filtered recovery log file;

The second read subunit is configured to read the next recovery log file; and so on, until the last recovery log file is compared, and the recovery log file corresponding to each index information in the preset structure is obtained.
A server for recovering data, characterized in that the server is a slave server applied to a storage system, the storage system includes a master server and the slave server, and the master server and the slave server are both used to store the same Data, the primary server is the first server to successfully store data, and the server includes:

a first receiving module, configured to receive a log obtaining request sent by the primary server, where the log obtaining request carries a data identifier to be restored;

a first returning module, configured to return a first recovery log file to the primary server according to the log obtaining request, so that the primary server generates a second recovery log file according to the first recovery log file and the primary server Recovering the data to be recovered;

a second receiving module, configured to receive the filtered recovery log file sent by the primary server;

a recovery module, configured to perform operation on the restored data according to the operation information recorded by the filtered recovery log file;

The first recovery log file and the second recovery log file respectively record location information and operation information of the to-be-recovered data identified by the to-be-recovered data identifier.
The server according to claim 13, wherein the server further comprises:

a storage module, configured to receive and store data sent by the client;

a second returning module, configured to return, to the client, second write state information that includes an identifier of the data, where the second write state information includes write success information or write failure information;

a third receiving module, configured to receive total write status information sent by the client, where the total write status information includes the second write status information returned from the server to the client, and the primary server First write status information returned to the client;

a generating module, configured to: when the total write status information sent by the client includes the write failure information, determine that the data sent by the client is the data to be restored, and generate location information that records the to-be-recovered data and The first recovery log file for operational information.
A system for recovering data, characterized in that the system comprises: a primary server and at least one secondary server;

The server according to any one of claims 8 to 12, wherein the slave server is a server according to any one of claims 13 to 14.