CN116521440A - Metadata recovery method, related device, equipment, system and storage medium - Google Patents

Abstract

The application discloses a metadata recovery method, and related devices, equipment, systems and storage media. The metadata recovery method comprises the following steps: responding to complete recovery of the metadata management system fault, and acquiring log data from a second metadata device serving as a slave device in the metadata management system by a first metadata device serving as a master device in the metadata management system; combining the log data in the first metadata device and the log data acquired from the second metadata device based on the first log identifier and the second log identifier in the log data; and synchronizing the combined log data to the second metadata equipment. According to the scheme, the metadata can be continuously and completely recovered after the system fault is completely recovered, particularly after the multipoint fault is completely recovered.

Description

Metadata recovery method, related device, equipment, system and storage medium

Technical Field

The present disclosure relates to the field of data processing technologies, and in particular, to a metadata recovery method, and related devices, apparatuses, systems, and storage media.

Background

In the age of information explosion, the data which can be obtained by people are exponentially increased, the requirement on computer storage is also higher and higher, and the problems of data storage and management are effectively solved by the appearance of storage systems such as a distributed file system and the like.

Taking a distributed file system as an example, a storage system generally has a main architecture such as a metadata management service, a data storage service, and a client access module. The metadata management service is responsible for managing file metadata information, the data storage service is responsible for storing file data, and the client access module provides a user reading and writing file interface. In addition, metadata management services typically employ a master-slave mode. However, in this active-standby mode, the prior art cannot ensure continuous, complete recovery of metadata after complete recovery of the system failure. Particularly, when the system suffers from multi-point faults, as the main and the standby have the operation time of writing data, after the multi-point faults are completely recovered, the data written by the main and the standby respectively need to be combined, but at the moment, the combination can cause the overlapping of the logs of part of the data, and the overlapped part can be directly covered to cause the data loss. In view of this, how to continuously and completely recover metadata after a complete recovery of a system failure, particularly after a complete recovery of a multipoint failure, is a problem to be solved.

Disclosure of Invention

The technical problem to be solved mainly in the application is to provide a metadata recovery method, a related device, equipment, a system and a storage medium, which can continuously and completely recover metadata after complete recovery of system faults, especially after complete recovery of multipoint faults.

In order to solve the technical problem, a first aspect of the present application provides a metadata recovery method, where the metadata recovery method includes responding to a complete recovery of a metadata management system failure, where a first metadata device currently serving as a master device in the metadata management system obtains log data from a second metadata device currently serving as a slave device in the metadata management system; combining the log data in the first metadata device and the log data acquired from the second metadata device based on the first log identifier and the second log identifier in the log data; synchronizing the combined log data to a second metadata device; the log data is recorded with file operation information, each piece of log data is attached with a first log identifier and a second log identifier, the log data respectively written in the same normal operation period of the same equipment have the same first log identifier, and the second log identifier is unique and is increased along with the increment writing of the log data.

In order to solve the above technical problem, a second aspect of the present application provides an electronic device, including a communication circuit, a memory, and a processor, where the communication circuit and the memory are respectively coupled to the processor, and the processor is configured to execute program instructions stored in the memory, so as to implement the metadata recovery method in the first aspect.

In order to solve the technical problem, a third aspect of the present application provides a metadata management system, which includes a first metadata device and a second metadata device that are communicatively connected to a storage system, and are configured to write log data recorded with file operation information when the first metadata device and the second metadata device are used as a master device, where the first metadata device and the second metadata device adopt a master-slave hot-standby architecture, and the first metadata device and the second metadata device are both electronic devices in the second aspect.

In order to solve the above technical problem, a fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which when executed by a processor, implement the metadata recovery method in the above first aspect.

According to the scheme, after the system fails and recovers, the first metadata device serving as the master device in the metadata management system acquires log data from the second metadata device serving as the slave device, merges the log data in the two metadata devices into the first metadata device according to the first log identification and the second log identification, and synchronizes the merged data into the second metadata device; on one hand, the first log mark is kept unchanged only in the same normal operation period of the same equipment, so that log data recorded by the main equipment in the operation period before each fault can be distinguished through the first log mark even if a multipoint fault occurs, and the fault tolerance of the system is improved; on the other hand, the first log identifier and the second log identifier are utilized to merge log data, so that continuity and integrity of the log data written by the system are facilitated. Thus, metadata can be continuously and completely restored after the system failure is completely restored, particularly after the multipoint failure is completely restored.

Drawings

FIG. 1 is a flow chart of an embodiment of a metadata recovery method of the present application;

FIG. 2 is a schematic diagram of an embodiment of a multipoint failure of the present application;

FIG. 3 is a process diagram of one embodiment of a metadata recovery method of the present application;

FIG. 4 is a flow chart of another embodiment of a metadata recovery method of the present application;

FIG. 5 is a schematic diagram of a metadata retrieval device according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a framework of an embodiment of the electronic device of the present application;

FIG. 7 is a schematic diagram of a framework of one embodiment of a metadata management system of the present application;

FIG. 8 is a schematic diagram of a framework of one embodiment of a computer readable storage medium of the present application.

Detailed Description

The following describes the embodiments of the present application in detail with reference to the drawings.

In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, interfaces, techniques, etc., in order to provide a thorough understanding of the present application.

The terms "first metadata device" and "current master device" are often used interchangeably herein, and the terms "second metadata device" and "current slave device" are often used interchangeably herein. "multiple" herein means two or more than two. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

Referring to fig. 1, fig. 1 is a flowchart of an embodiment of a metadata recovery method of the present application. Specifically, the method may include the steps of:

step S11: in response to a complete recovery of the metadata management system failure, a first metadata device in the metadata management system currently acting as a master device obtains log data from a second metadata device in the metadata management system currently acting as a slave device.

It should be noted that, as described above, the metadata management system may include a first metadata device and a second metadata device, where both devices adopt a master-slave hot-standby architecture, and the master-slave hot-standby architecture refers to that servers that are mutually backed up are used to jointly execute the same service, where one host is a working machine, and the other host is a backup host. Under the normal condition of the system, the working machine provides service for the application system, the backup machine monitors the operation condition of the working machine, and when the working machine is abnormal and cannot support the operation of the application system, the backup machine actively takes over the operation of the working machine, continuously supports the key application service and ensures the uninterrupted operation of the system.

In the embodiment of the disclosure, the first metadata device is used as a working machine, and the second metadata device is used as a backup host. On this basis, the complete recovery of the metadata management system failure means that both the first metadata device and the second metadata device work normally. That is, the first metadata device can normally write log data and synchronize the log data to the second metadata device in real time, and on the other hand, the second metadata device can also normally receive the log data synchronized by the first metadata device.

In addition, the metadata recovery method can be suitable for single-point faults and multi-point faults. The single-point fault refers to that after the current main equipment fails, the current slave equipment becomes the current main equipment to operate, and at the moment, the fault is recovered, and the current main equipment can normally synchronize log data to the equipment recovered from the fault; the multipoint fault refers to that after the current master device fails, the current slave device becomes the current master device to operate, and before the data is not synchronized, that is, when the previous master device has not recovered from the fault, the slave device becomes the current master device, and the master and slave devices are all in fault. After the fault is recovered, the current master device directly synchronizes the data to the current slave device, and the data written by the non-current master device during the fault is covered by the data of the current master device, so that the written data is lost, the continuity of the data is damaged, and the reliability of the metadata management system is reduced. In expansion, under the condition that both the master and the slave devices fail, one of the master and the slave devices is recovered, the other is not recovered, synchronization cannot be completed, and if the recovered device fails again, after the failure is recovered, the data of the current master device also covers the data written by the non-current master device during the failure period, so that the multi-point failure comprises but is not limited to continuous failure of two points.

In one implementation scenario, please refer to fig. 2, fig. 2 is a schematic diagram of a system with two-point failure. M1 is used as the current master equipment to normally operate, recorded log data is synchronized to the current slave equipment M2 in real time, at a certain moment, M1 fails, M2 is used as the current master equipment to normally operate, at the moment, M2 cannot synchronize the log data to M1, M2 also fails before M1 is recovered, and after both M1 and M2 fail, in the example, M1 is recovered to fail first to operate as the current master equipment, and M2 is recovered to fail later. So far, M1 is used as the current main equipment to normally operate, and M2 normally receives the synchronous log data.

In the embodiment of the disclosure, the log data includes basic information such as a first log identifier, a second log identifier, a first file identifier, a second file identifier, file operation information, and the like.

In one implementation scenario, the log data format is:

{ Timestamp-LogId } { put } { File name- > [ Timestamp-FileId|State|Length|creation time|modification time|location information ] }

The Timestamp in the Timestamp-LogId is the first File identifier, the Timestamp in the Timestamp-File is the second File identifier, and other information is File operation information. Taking the following log data as an example:

{1667848896-10001001}{put}{2022-03-11/10:00:00.dav->[1667848896-30000001|0|1667843244|1667843244|(3,5,2,8,3)]}

wherein 1667848896 of 1667848896-10001001 represents a first log identity of a current run period and 10001001 represents a second log identity; 1667848896 of 1667848896-30000001 denotes a first file identification, 30000001 denotes a second file identification.

The first metadata device obtains log data from the second metadata device in a manner including, but not limited to, overall acquisition, comparative acquisition, and the like. The whole acquisition means that all log data in the second metadata equipment are acquired; the comparison and acquisition means that log data in the second metadata device are compared, and only log data which is different from any one of the first log identifier and the second log identifier in the first metadata device is acquired.

Step S12: and merging the log data in the first metadata device and the log data acquired from the second metadata device based on the first log identifier and the second log identifier in the log data.

In the embodiment of the disclosure, the first log identifier is a field that increases linearly, and log data written in the same normal operation period of the same device respectively has the same first log identifier.

In an implementation scenario, the first log identifier is a timestamp field, the master device and the slave device are time-synchronized, the timestamp field is determined by the time when the first metadata device starts to operate normally, and the conversion relationship between the timestamp field and a certain time can refer to the generation technology of the timestamp field, which is not described herein. Log data written respectively in the same normal operation period of the same device has the same time stamp field determined when the period starts to operate.

In another implementation scenario, different from the foregoing manner, the determination of the first log identifier performs the increment of the target number of times based on the latest first log identifier of the current device, to obtain the first log identifier as the first metadata device writing log data; wherein the target number of times matches the type of failure of the metadata management system. For example, a two-point failure occurs consecutively, with the first log identity incremented twice.

In the embodiment of the disclosure, the second log identifier is unique and increases progressively with the incremental writing of the log data, and the second log identifier is overlapped among different devices, so that the fault tolerance of the system is improved due to the arrangement of the first log identifier.

It should be further noted that, merging the log data in the first metadata device and the log data obtained from the second metadata device, where the merging mode includes detecting whether the log data in the first metadata device and the log data obtained from the second metadata device have the same first log identifier and the same second log identifier, and merging the log data in the first metadata device and the log data obtained from the second metadata device by adopting a merging policy matched with the detection result. In addition, the log data may be ordered before or after detection.

In one implementation scenario, it is detected that log data in a first metadata device and log data obtained from a second metadata device have different first log identifications; firstly, sorting log data in first metadata equipment and log data acquired from second metadata equipment for the first time according to the sequence from the first log mark to the large, and sorting the log data after the first sorting is performed for the second time according to the sequence from the second log mark to the large, so that the sorting and merging are carried out to obtain finally arranged log data; or the sorting order is exchanged, firstly sorting the log data in the first metadata equipment and the log data obtained from the second metadata equipment according to the order from the second log identification to the large, and then sorting the log data after the first sorting is executed according to the order from the first log identification to the large.

In one implementation scenario, it is detected that the log data in the first metadata device and the log data obtained from the second metadata device have the same first log identifier and the same second log identifier, and the log data of the current master device is used as the combined log data.

In one embodiment, referring to fig. 3 in combination, the first log identifier and the second log identifier shown in fig. 3 are in the form of: first log identity-second log identity; if the current master M1 starts writing T0-10001001 at the time T0, synchronizing the log data to M2; the time T1 is the time when the current master M1 breaks down, the current slave M2 becomes the current master to write in T1-00000001-T1-00002001 during the period of T1-T2, T0-10001001, T1-00000001-T1-00002001 exist in M2, the current master M2 changed from the current slave at the time T2 breaks down before the log data is synchronized, M1 breaks down is recovered, the device M1 writes in T2-00000001-T2-00040001 during the period of T2-T3, and T0-10001001 and T2-00000001-T2-00040001 exist in M1; the time T3 is the time when the device M2 recovers from the fault, and at this time, the M1 acquires the log data from the M2 and merges the log data. Specifically, it is detected that T0-10001001 of M1 and T0-10001001 acquired from M2 have the same first log identity and the same second log identity, and T0-10001001 in M1 is directly used as the combined log data; T2-00000001-T2-00040001 in M1 is detected to be different from T1-00000001-T1-00002001 obtained from M2, the corresponding merging strategy is to firstly sort according to the sequence of T1 and T2, and under the same first log mark T1, log data subjected to primary sorting is secondarily sorted according to the sequence of second log mark 00000001 ～ 00002001 from small to large under the same first log mark T2 and according to the sequence of second log mark 00000001 ～ 00040001 from small to large, and the log data subjected to primary sorting is merged into M1. The combined M1 has log data arranged in the order of T0-10001001, T1-00000001-T1-00002001, and T2-00000001-T2-00040001.

Step S13: and synchronizing the combined log data to the second metadata equipment.

And after the log data is merged, synchronizing the merged log data to the current slave device.

According to the scheme, after the system fails and recovers, the first metadata device serving as the master device in the metadata management system acquires log data from the second metadata device serving as the slave device, merges the log data in the two metadata devices into the first metadata device according to the first log identification and the second log identification, and synchronizes the merged data into the second metadata device; on one hand, the first log mark is kept unchanged only in the same normal operation period of the same equipment, so that log data recorded by the main equipment in the operation period before each fault can be distinguished through the first log mark even if a multipoint fault occurs, and the fault tolerance of the system is improved; on the other hand, the first log identifier and the second log identifier are utilized to merge log data, so that continuity and integrity of the log data written by the system are facilitated. Thus, metadata can be continuously and completely restored after the system failure is completely restored, particularly after the multipoint failure is completely restored.

Referring to fig. 4, fig. 4 is a flowchart illustrating another embodiment of a metadata recovery method according to the present application. It should be noted that, the log data further includes a first file identifier and a second file identifier; the file operation information further includes a first type of operation including a create file operation and a second type of operation including, but not limited to, an modify file operation, a delete file operation, and the like. The first file identifier is a field with linear increment, log data written in the same normal operation period of the same equipment respectively have the same first file identifier, and the specific increment form of the first file identifier is similar to that of the first log identifier and is not repeated here; the second file identity is unique and is incremented with incremental execution of the first type of operation; specifically, embodiments of the present disclosure may include the steps of:

step S41: in response to the file operation request, it is found whether log data relating to a file corresponding to the file operation request exists.

And the first metadata equipment responds to the file operation request, and searches whether log data related to a file corresponding to the file operation request exists by taking the first file identifier and the second file identifier as references.

Step S42: and updating the log data of the first metadata device based on the search result.

In one implementation scenario, the search result does not have log data related to the file corresponding to the file operation request, at this time, it may be determined that the file operation request belongs to a first type of operation, the latest second file identifier is incremented to obtain a new second file identifier, and the new log data is recorded based on the first log identifier, the second log identifier, the first file identifier, and the new second file identifier. Specifically, taking creating a file as an example, searching that no corresponding log data exists, increasing the latest second file identifier, and recording the latest log data:

{1667848896-10001001}{put}{2022-03-11/10:00:00.dav->[1667848896-30000001|0|1667843244|1667843244|(3,5,2,8,3)]}

at this point, the second log identity is incremented once to 10001001 and the second file identity is incremented once to 30000001.

In one implementation scenario, there is a first file identifier and a second file identifier related to a file corresponding to a file operation request, at which time it may be determined that the file operation request belongs to a second type of operation, and log data related to the file corresponding to the file operation request is taken as a target log, and the target log is updated based on the file operation request. Specifically, taking the deletion operation of a certain file as an example, the target log is found:

{1667848896-10001003}{put}{2022-03-13/11:00:00.dav->[1667848896-30020030|0|1667843244|1667843244|(1,2,4,7,5)]}

the first file identifier and the second file identifier are respectively: 1667848896 and 30020030, delete the file, record log data after the delete operation is performed:

{1667848896-10001004}{del}{2022-04-10/13:00:00.dav}

the second log identity of the log recorded at this point is incremented from 10001003 once to 10001004 and the first file identity and the second file identity no longer exist.

According to the scheme, outside master-slave synchronization, namely when the first metadata equipment normally writes log data, the first file identifier and the second file identifier are set to distinguish file operation information recorded in different running time periods of different equipment, so that the accuracy of the metadata management service in response to the file operation request is improved.

Referring to fig. 5, fig. 5 is a schematic diagram illustrating an embodiment of a metadata recovery apparatus 50 according to the present application. The metadata recovery apparatus 50 includes an obtaining module 51, a merging module 52, and a synchronization module 53, where the obtaining module 51 is configured to respond to a complete recovery of a metadata management system failure, and a first metadata device currently serving as a master device in the metadata management system obtains log data from a second metadata device currently serving as a slave device in the metadata management system; the merging module 52 is configured to merge the log data in the first metadata device and the log data acquired from the second metadata device based on the first log identifier and the second log identifier in the log data; the synchronization module 53 is configured to synchronize the combined log data to the second metadata device; the log data is recorded with file operation information, each piece of log data is attached with a first log identifier and a second log identifier, the log data respectively written in the same normal operation period of the same equipment have the same first log identifier, and the second log identifier is unique and is increased along with the increment writing of the log data.

According to the scheme, after the system fails and recovers, the first metadata device serving as the master device in the metadata management system acquires log data from the second metadata device serving as the slave device, merges the log data in the two metadata devices into the first metadata device according to the first log identification and the second log identification, and synchronizes the merged data into the second metadata device; on one hand, the first log mark is kept unchanged only in the same normal operation period of the same equipment, so that log data recorded by the main equipment in the operation period before each fault can be distinguished through the first log mark even if a multipoint fault occurs, and the fault tolerance of the system is improved; on the other hand, the first log identifier and the second log identifier are utilized to merge log data, so that continuity and integrity of the log data written by the system are facilitated. Thus, metadata can be continuously and completely restored after the system failure is completely restored, particularly after the multipoint failure is completely restored.

In some disclosed embodiments, the merge module 52 includes an identification detection sub-module for detecting whether the log data in the first metadata device and the log data obtained from the second metadata device have the same first log identification and the same second log identification, and the merge module 52 further includes a log merge sub-module for merging the log data in the first metadata device and the log data obtained from the second metadata device using a merge policy that matches the detection result.

Therefore, by taking the log data in the first metadata device as the log data after being combined, the step of combining the same log data is avoided, and the combining efficiency is improved.

In some disclosed embodiments, the log merging sub-module includes an identity response unit configured to respond to log data in the first metadata device and log data obtained from the second metadata device having the same first log identity and the same second log identity, and to take the log data in the first metadata device as the merged log data.

In some disclosed embodiments, the log merging sub-module further includes a sorting unit configured to sort the log data in the first metadata device and the log data acquired from the second metadata device based on the first log identifier and the second log identifier in the log data before or after detecting whether the log data in the first metadata device and the log data acquired from the second metadata device have the same first log identifier and the same second log identifier.

In some disclosed embodiments, the sorting unit sorts the log data in the first metadata device and the log data obtained from the second metadata device once in order of the first log identification from small to large; and (3) performing secondary sorting on the log data after the primary sorting is performed according to the order of the second log marks from small to large, wherein the order can be exchanged.

Therefore, the sorting unit sorts the log data before or after detecting whether the log data in the first metadata device and the log data acquired from the second metadata device have the same first log identification and the same second log identification, which is beneficial to maintaining the log data order and improving the stability.

In some disclosed embodiments, the obtaining module 51 includes a first identifier obtaining sub-module, configured to determine, when the first metadata device and the second metadata device are time-synchronized, a first log or a file identifier of the first metadata device written in log data based on a time when the first metadata device starts to operate normally.

In some disclosed embodiments, the first identifier obtaining sub-module is further configured to increment a first log or a file identifier, and obtain a first log or a file identifier of the first metadata device with the latest log data; performing target times increment based on the latest first log or file identifier to obtain the first log or file identifier serving as the first metadata equipment written log data; wherein the target number of times matches the type of failure of the metadata management system.

In some disclosed embodiments, the obtaining module 51 further includes a second identifier obtaining sub-module for obtaining a second file identifier that is unique and incremented with the incremental execution of the first type of operation, wherein the file operation information includes a file operation type, the file operation type being any one of a first type of operation that causes a change in the second file identifier, and a second type of operation that does not change in the second file identifier.

In some disclosed embodiments, the second identifier obtaining sub-module includes an identifier searching unit, configured to search whether log data related to a file corresponding to the file operation request exists in response to the file operation request; the second identifier obtaining sub-module further comprises a log updating unit, which is used for updating log data of the first metadata device based on the searching result.

In some disclosed embodiments, the identifier lookup unit is configured to determine that the file operation request belongs to a first type of operation in response to the lookup result including no log data related to the file corresponding to the file operation request, and increment a latest second file identifier to obtain a new second file identifier, and the log update unit records the new log data based on the first log identifier, the second log identifier, the first file identifier, and the new second file identifier.

In some disclosed embodiments, the identification searching unit is further configured to determine that the file operation request belongs to the second type of operation in response to the searching result including the presence of the log data related to the file corresponding to the file operation request, and take the log data related to the file corresponding to the file operation request as the target log, and the log updating unit updates the target log based on the file operation request.

According to the scheme, after the system fails and recovers, the first metadata device serving as the master device in the metadata management system acquires log data from the second metadata device serving as the slave device, merges the log data in the two metadata devices into the first metadata device according to the first log identification and the second log identification, and synchronizes the merged data into the second metadata device; on one hand, the first log mark is kept unchanged only in the same normal operation period of the same equipment, so that log data recorded by the main equipment in the operation period before each fault can be distinguished through the first log mark even if a multipoint fault occurs, and the fault tolerance of the system is improved; on the other hand, the first log identifier and the second log identifier are utilized to merge log data, so that continuity and integrity of the log data written by the system are facilitated. Thus, metadata can be continuously and completely restored after the system failure is completely restored, particularly after the multipoint failure is completely restored.

Referring to fig. 6, fig. 6 is a schematic diagram of a frame of an embodiment of an electronic device 60 of the present application. The electronic device 60 comprises a communication circuit 61, a memory 62 and a processor 63, the communication circuit 61, the memory 62 being coupled to the processor 63, respectively, the processor 63 being adapted to execute program instructions stored in the memory 62 for implementing the steps of any of the metadata recovery method embodiments described above. In particular, electronic device 60 may include, but is not limited to: the microcomputer and the server are not limited herein. In addition, the communication circuit 61 may be used to exchange information (e.g., log data). Reference may be made in particular to steps in an embodiment of a metadata recovery method.

In particular, the processor 63 is configured to control itself as well as the communication circuit 61, the memory 62 to implement the steps of any of the metadata recovery method embodiments described above. The processor 63 may also be referred to as a CPU (Central Processing Unit ). Processor 63 may be an integrated circuit chip with signal processing capabilities. Processor 63 may also be a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a Field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. In addition, the processor 63 may be commonly implemented by an integrated circuit chip.

According to the scheme, after the system fails and recovers, the first metadata device serving as the master device in the metadata management system acquires log data from the second metadata device serving as the slave device, merges the log data in the two metadata devices into the first metadata device according to the first log identification and the second log identification, and synchronizes the merged data into the second metadata device; on one hand, the first log mark is kept unchanged only in the same normal operation period of the same equipment, so that log data recorded by the main equipment in the operation period before each fault can be distinguished through the first log mark even if a multipoint fault occurs, and the fault tolerance of the system is improved; on the other hand, the first log identifier and the second log identifier are utilized to merge log data, so that continuity and integrity of the log data written by the system are facilitated. Thus, metadata can be continuously and completely restored after the system failure is completely restored, particularly after the multipoint failure is completely restored.

Referring to fig. 7, fig. 7 is a schematic diagram illustrating an embodiment of a metadata management system 70 according to the present application. The metadata management system 70 includes a first metadata device 71 and a second metadata device 72 that are communicatively connected to the storage system, and are used for writing log data recorded with file operation information when they are used as a master device, where the first metadata device and the second metadata device adopt a master-slave hot-standby architecture, and the first metadata device 71 and the second metadata device 72 are both electronic devices in the above-described electronic device embodiments.

According to the scheme, after the system fails and recovers, the first metadata device serving as the master device in the metadata management system acquires log data from the second metadata device serving as the slave device, merges the log data in the two metadata devices into the first metadata device according to the first log identification and the second log identification, and synchronizes the merged data into the second metadata device; on one hand, the first log mark is kept unchanged only in the same normal operation period of the same equipment, so that log data recorded by the main equipment in the operation period before each fault can be distinguished through the first log mark even if a multipoint fault occurs, and the fault tolerance of the system is improved; on the other hand, the first log identifier and the second log identifier are utilized to merge log data, so that continuity and integrity of the log data written by the system are facilitated. Thus, metadata can be continuously and completely restored after the system failure is completely restored, particularly after the multipoint failure is completely restored.

Referring to FIG. 8, FIG. 8 is a schematic diagram illustrating an embodiment of a computer readable storage medium 80 of the present application. The computer-readable storage medium 80 stores program instructions 81 that can be executed by a processor, the program instructions 81 being for implementing the steps of the metadata recovery method embodiments described above.

According to the scheme, after the system fails and recovers, the first metadata device serving as the master device in the metadata management system acquires log data from the second metadata device serving as the slave device, merges the log data in the two metadata devices into the first metadata device according to the first log identification and the second log identification, and synchronizes the merged data into the second metadata device; on one hand, the first log mark is kept unchanged only in the same normal operation period of the same equipment, so that log data recorded by the main equipment in the operation period before each fault can be distinguished through the first log mark even if a multipoint fault occurs, and the fault tolerance of the system is improved; on the other hand, the first log identifier and the second log identifier are utilized to merge log data, so that continuity and integrity of the log data written by the system are facilitated. Thus, metadata can be continuously and completely restored after the system failure is completely restored, particularly after the multipoint failure is completely restored.

In some embodiments, functions or modules included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.

The foregoing description of various embodiments is intended to highlight differences between the various embodiments, which may be the same or similar to each other by reference, and is not repeated herein for the sake of brevity.

In the several embodiments provided in the present application, it should be understood that the disclosed methods and apparatus may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of modules or units is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical, or other forms.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all or part of the technical solution contributing to the prior art or in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Claims (12)

1. A metadata retrieval method, comprising:

responding to complete recovery of a metadata management system fault, wherein a first metadata device currently serving as a master device in the metadata management system acquires log data from a second metadata device currently serving as a slave device in the metadata management system;

combining the log data in the first metadata device and the log data acquired from the second metadata device based on the first log identifier and the second log identifier in the log data;

synchronizing the combined log data to the second metadata device;

the log data are recorded with file operation information, each piece of log data is attached with the first log identifier and the second log identifier, the log data respectively written in the same normal operation period of the same equipment have the same first log identifier, and the second log identifier is unique and is increased along with the increment writing of the log data.

2. The method of claim 1, wherein the merging the log data in the first metadata device and the log data obtained from the second metadata device based on the first log identification and the second log identification in the log data comprises:

detecting whether the log data in the first metadata device and the log data acquired from the second metadata device have the same first log identifier and the same second log identifier;

and merging the log data in the first metadata equipment and the log data acquired from the second metadata equipment by adopting a merging strategy matched with the detection result.

3. The method according to claim 2, wherein the merging the log data in the first metadata device and the log data acquired from the second metadata device using a merging policy matching the detection result includes:

and responding to the fact that the log data in the first metadata device and the log data acquired from the second metadata device have the same first log identification and the same second log identification, and taking the log data in the first metadata device as combined log data.

4. The method of claim 2, wherein the second log identification is incremented sequentially, the method further comprising, before or after said detecting whether the log data in the first metadata device and the log data obtained from the second metadata device have the same first log identification and the same second log identification:

and sequencing the log data in the first metadata device and the log data acquired from the second metadata device based on the first log identifier and the second log identifier in the log data.

5. The method of claim 4, wherein the sorting the log data in the first metadata device and the log data obtained from the second metadata device based on the first log identification and the second log identification in the log data comprises:

sequencing the log data in the first metadata equipment and the log data acquired from the second metadata equipment once according to the sequence from the first log identification to the second log identification;

and performing secondary sorting on the log data after the primary sorting is performed according to the order of the second log marks from small to large.

6. The method according to claim 1, wherein the log data is further accompanied by a first file identification and a second file identification, the file operation information including a file operation type, the file operation type being any one of a first type of operation causing a change in the second file identification, a second type of operation in which the second file identification does not change; the file operation information written in the same normal operation period of the same equipment has the same first file identification, and the second file identification is unique and is increased along with the incremental execution of the first type of operation.

7. The method of claim 6, wherein the method further comprises:

responding to a file operation request, and searching whether log data related to a file corresponding to the file operation request exists or not;

and updating the log data of the first metadata device based on the search result.

8. The method of claim 7, wherein updating the log data of the first metadata device based on the search result comprises at least one of:

responding to the searching result to comprise that log data related to a file corresponding to the file operation request does not exist, determining that the file operation request belongs to the first type of operation, increasing the latest second file identifier to obtain a new second file identifier, and recording new log data based on the first log identifier, the second log identifier, the first file identifier and the new second file identifier;

and responding to the search result to comprise log data related to the file corresponding to the file operation request, determining that the file operation request belongs to the second type of operation, taking the log data related to the file corresponding to the file operation request as a target log, and updating the target log based on the file operation request.

9. The method according to any of claims 1 to 8, wherein the failure type of the metadata management system comprises at least a multipoint failure.

10. An electronic device comprising communication circuitry, a memory, and a processor, the communication circuitry, the memory being respectively coupled to the processor, the processor being configured to execute program instructions stored in the memory to implement the metadata retrieval method of any one of claims 1 to 9.

11. A metadata management system, comprising a first metadata device and a second metadata device communicatively connected to a storage system, for writing log data recorded with file operation information when the first metadata device and the second metadata device are used as a master device, the first metadata device and the second metadata device adopt a master-slave hot-standby architecture, and the first metadata device and the second metadata device are both the electronic device of claim 10.

12. A computer readable storage medium having stored thereon program instructions, which when executed by a processor implement the metadata recovery method of any of claims 1 to 9.