WO2021093735A1 - Data synchronization method, apparatus and device for distributed storage system, and storage medium - Google Patents

Abstract

A data synchronization method, apparatus and device for a distributed storage system, and a storage system. The distributed storage system comprises a master storage server and a slave storage server. The method is implemented by the master storage server, and comprises: determining whether the slave storage server successfully writes first log information, wherein the first log information comprises the latest log entry in a local log file of the master storage server; after it is determined that the first log information has been successfully written, determining whether data is written; and if it is determined that no data is written, sending second log information to the slave storage server to inform the slave storage server of storing third log information, wherein there is no log entry in the second log information, and the third log information includes a current set number of persistent latest log entries of other slave storage servers. By means of the solution, the problem of a log entry in a storage engine of the slave storage server being updated late and failing to be kept synchronous with a log entry in a storage engine of the master storage server can be solved.

Description

Data synchronization method, device, equipment and storage medium of distributed storage system

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on November 15, 2019 with the application number 201911121853.X and the invention titled "Data synchronization method, device, equipment and storage medium of a distributed storage system". The entire content is incorporated into this application by reference.

Technical field

This application relates to the field of communication technology, and more specifically, to a data synchronization method of a distributed storage system, a data synchronization device of a distributed storage system, a storage server of a distributed storage system, and a computer storage medium.

Background technique

Currently, in a distributed storage system, the primary storage server can synchronize log entries in the local log file to the secondary storage server. At the same time, the primary storage server will also notify the secondary storage server of the latest log entry that has been persisted by most storage servers, so that the secondary storage server can read the latest log entry and all previous log entries. And store the read content in the storage engine corresponding to the slave storage server.

This method of data synchronization has the following drawbacks: After the primary storage server synchronizes the last log entry in the local log file to the secondary storage server, there may be a situation where no new data is written for a certain period of time, and the primary storage The server only interacts with the secondary storage server when it needs to synchronize log entries. Then, the primary storage server will no longer interact with the secondary storage server within a certain period of time. Therefore, the primary storage server cannot promptly notify the secondary storage server of the latest log entry information that has been persisted by most storage servers, resulting in the lag in updating the log entries in the storage engine of the secondary storage server, and cannot communicate with the storage engine of the primary storage server. The log entries in are kept in sync.

Summary of the invention

One purpose of this application is to provide a data synchronization method, device, device, and storage medium for a distributed storage system to solve the problem of the lag in updating log entries in the storage engine of the storage server in the related art, which cannot be compared with the storage of the main storage server. The log entries in the engine remain synchronized.

According to the first aspect of the present application, there is provided a data synchronization method for a distributed storage system. The distributed storage system includes a primary storage server and a secondary storage server. The method is implemented by the primary storage server and includes: determining Whether the slave storage server successfully writes the first log information, where the first log information includes the latest log entry in the local log file of the master storage server; when it is determined that the slave storage server has successfully written the first log entry After a log information, it is determined whether there is data written; if it is determined that no data is written, the second log information is sent to the slave storage server to notify the slave storage server to store the third log information, and the second log information There is no log entry; wherein, the third log information includes a set number of other latest log entries persisted from the storage server.

Optionally, the determining whether data is written after it is determined that the slave storage server has successfully written the first log information includes: determining that the slave server has successfully written the first log information When the timer reaches the preset time, monitor whether there are log entries added to the local log file; if it is determined that there are log entries added to the local log file, it is determined that data is written; if it is determined If no log entry is added to the local log file, it is determined that no data is written.

Optionally, the method further includes: at any time before the timing time reaches a preset time, if it is monitored that a log entry is added to the local log file, stopping timing; Four log information, wherein the fourth log information includes at least log entries added to the local log file.

Optionally, the method further includes: when it is determined that the slave storage server has not successfully written the first log information, receiving a first serial number sent by the slave storage server, where the first serial number is the slave The number of the most recent log entry in the storage server.

Optionally, the method further includes: sending fifth log information to the slave storage server according to the first serial number, wherein the fifth log information includes the first serial number to the second serial number The log entries corresponding to all consecutive serial numbers in between, where the second serial number is the number corresponding to the log entry with the latest writing time in the local log file.

Optionally, wherein the determining whether the slave storage server successfully writes the first log information includes: receiving reply information from the slave storage server; when the reply information is a notification of successful writing, determining The writing of the first log information from the storage server is successful; when the reply information is a notification of writing failure, it is determined that the writing of the first log information from the storage server has failed.

According to a second aspect of the present application, there is provided a data synchronization device of a distributed storage system, including: a first determining module configured to determine whether the slave storage server successfully writes first log information, the first log The information includes the latest log entry in the local log file of the primary storage server; the second determining module is configured to determine whether there is data to be written after it is determined that the secondary storage server has successfully written the first log information; The sending module is configured to send second log information to the slave storage server if it is determined that no data is written to notify the slave storage server to store the third log information, and the second log information does not have a log entry; wherein, The third log information includes a set number of other latest log entries persisted from the storage server.

Optionally, the second determining module is specifically configured to: when it is determined that the slave server has successfully written the first log information, start timing; during the period when the timing time reaches a preset time, monitor the local Whether there are log entries added to the log file; if it is determined that there are log entries added to the local log file, it is determined that data is written; if it is determined that there are no log entries added to the local log file, it is determined that no data is written.

According to a third aspect of the present application, there is provided a storage server of a distributed storage system, including: a processor and a memory; the memory is used to store executable instructions, and the instructions are used to control the processor to execute The method of any one of the first aspect.

According to a fourth aspect of the present application, a computer storage medium is provided, the storage medium stores computer instructions, and when the computer instructions in the storage medium are executed by a processor, a Methods.

In this embodiment, it is first determined whether the first log information is successfully written from the storage server. Since the first log information includes the latest log entry in the local log file of the primary storage server, when it is determined that the secondary storage server has successfully written the first log information, it indicates that the log entry in the local log file of the primary storage server is the same as Synchronize log entries from the local log file of the storage server. At this time, determine whether there is data written in the local log file of the primary storage server; if it is determined that there is no data written, send the second log information to the secondary storage server to notify the secondary storage server to store the third log information, and the second log information There is no log entry; because the third log information contains the latest log entries persisted by other secondary storage servers that are currently set, although the log entries in the local log file of the primary storage server are different from those of the secondary storage server. The log entries in the log file are synchronized, but the primary storage server can still notify the secondary storage server of the latest persistent log entries that have been set by the number of other secondary storage servers in a timely manner. Further, the slave storage server reads and stores the current set number of other latest log entries persisted from the storage server and previous log entries from the local log file to the storage engine. This avoids the lag in updating the log entries in the storage engine of the slave storage server. At this time, the log entries in the storage engine of the slave storage server are synchronized with the log entries in the storage engine of the primary storage server.

Description of the drawings

In order to more clearly describe the technical solutions of the embodiments of the present application and related technologies, the following briefly introduces the drawings that need to be used in the embodiments and related technologies. Obviously, the drawings in the following description are only of the present application. For some embodiments, those of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

FIG. 1 is a block diagram of the hardware configuration of a storage server in a distributed storage system provided by an embodiment of the present application;

2 is a flowchart of a data synchronization method of a distributed storage system provided by an embodiment of the present application;

FIG. 3 is a schematic structural diagram of a data synchronization device of a distributed storage system provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a storage server of a distributed storage system provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions, and advantages of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this application.

<Hardware Configuration>

FIG. 1 is a block diagram of the hardware configuration of a storage server in a distributed storage system provided by an embodiment of the present application. Wherein, the storage server may be a primary storage server in a distributed storage system, and may also be a secondary storage server in a distributed storage system.

The storage server 1000 may be a virtual machine or a physical machine. The storage server 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a microphone 1800, and so on. Exemplarily, the processor 1100 may be a central processing unit CPU, a microprocessor MCU, or the like. Exemplarily, the memory 1200 may include ROM (Read Only Memory), RAM (Random Access Memory), non-volatile memory such as a hard disk, and the like. Exemplarily, the interface device 1300 may include a USB (Universal Serial Bus, universal serial bus) interface, a headphone interface, and the like. Exemplarily, the communication device 1400 can perform wired or wireless communication. Exemplarily, the display device 1500 may include a liquid crystal display screen, a touch display screen, and the like. Exemplarily, the input device 1600 may include a touch screen, a keyboard, and the like. The user can input/output voice information through the speaker 1700 and the microphone 1800.

Although multiple devices are shown for the storage server 1000 in FIG. 1, the present application may only involve some of the devices. For example, the storage server 1000 only involves the memory 1200 and the processor 1100.

In the above description, technicians can design instructions according to the solutions disclosed in this application. How the instruction controls the processor to operate is well known in the art, so it will not be described in detail here.

<System Example>

An embodiment of the present application provides a data synchronization method for a distributed storage system, where the distributed storage system includes a master storage server and a slave storage server, and the number of slave storage servers may be at least one. The description of the primary storage server and secondary storage server in the distributed storage system is as follows:

In a distributed storage system, users upload table data to the main storage server through the client. The main storage server generates a corresponding log entry based on each table data uploaded by the user, and writes the log entry locally to the main storage server In the log file (also known as the primary storage server to persist log entries). In other words, the local log file of the primary storage server consists of at least one log entry. It should be emphasized that the above table data is only an example of the data to be stored that can be uploaded by the user. The user can also upload other types of data to the main storage server through the client. The specific storage type can be compared with the database in the storage system. The storage type corresponds to the storage type, which is not limited in the embodiment of the present application. In order to describe clearly, the following still uses tabular data as an example to introduce the solution.

It should be noted that the main storage server generates a corresponding log entry according to each table data uploaded by the user. Among them, a log entry corresponding to each table data includes: the uploaded table data, the number of the generated log entry, and the tenure number of the main storage server. In addition, the number of log entries generated by the primary storage server is incremental, for example, strictly continuous increments, that is, the log entry corresponding to the largest number in the primary storage server is the latest log entry generated in the primary storage server. Among them, the uploaded table data is the data to be stored, the number of the generated log entry is the number of the log entry given when the log entry is generated, and the term number of the primary storage server can represent the primary storage server as the primary server. Effective time limit.

After the primary storage service writes the log entries into the local log file, the primary storage server obtains a batch of log entries that need to be synchronized to the secondary storage server from the local log file in a descending and continuous manner, and obtains The log entries are forwarded to the slave storage server.

When the secondary storage server receives the log entry sent by the primary storage server, it performs matching according to the current latest log entry (that is, the log entry corresponding to the largest number) in the local log file of the secondary storage server. When the secondary storage server determines that the match is successful, the secondary storage server writes the log entries forwarded by the primary storage server to the local log file (also known as the secondary storage server persisting the log entries), and sends the write to the primary storage server Successful notification. Correspondingly, when the secondary storage server determines that the match fails, the secondary storage server does not write the log entries forwarded by the storage server into the local log file, and sends a notification of the write failure to the primary storage server. Exemplarily, matching from the storage server according to the current latest log entry in the local log file may include: identifying the smallest number of the received log entry, whether it is the number of the current latest log entry in the local log file The next number, if it is, it is judged as a match, otherwise, it is judged as a mismatch; or, to identify whether the number of the received log entry is continuously increasing, and it contains the lower part of the number of the current latest log entry in the local log file. A number, if it is, it is judged as a match, otherwise, it is judged as a non-match.

After the primary storage server receives the notification that the write is successful, when there are log entries in the local log file of the primary storage server that have not been forwarded to the secondary storage server, the next batch obtained from the local log file needs to be forwarded to the secondary storage server And forward the obtained log entries to the secondary storage server; at the same time, inform the secondary storage server of the latest log entries that have been persisted by most storage servers. Exemplarily, most storage servers may include a primary storage server and other secondary storage servers other than the primary storage server, and the other secondary storage servers are storage servers other than the above-mentioned secondary storage server and primary storage server.

<Method Example>

The embodiment of the present application provides a data synchronization method of a distributed storage system, where the distributed storage system includes a primary storage server and a secondary storage server. As shown in Figure 2, the method is implemented by the main storage server in the distributed storage system, that is, the method is applied to the main storage server in the distributed storage system. The method includes the following S2100-S2300:

S2100. Determine whether the slave storage server successfully writes the first log information, where the first log information includes the latest log entry in the log of the master storage server.

In this embodiment, the first log information is the latest batch of log entries forwarded by the primary storage server to the secondary storage server, that is, during this synchronization process, the primary storage server forwarded to the secondary storage server and contains the latest log entry. The last batch of log entries for entries. Based on this, the first log information includes the latest log entry in the local log file of the primary storage server. In addition, the first log entry may include one log entry, or may include multiple log entries. It can be understood that the latest log entry in the local log file of the primary storage server is the log entry corresponding to the largest number in the local log file of the primary storage server.

In an embodiment, the above S2100 may be determined according to the reply information sent from the storage server. Based on this, the above S2100 can be implemented through the following S2110-S2112:

S2110. Receive a reply message from a storage server.

In this embodiment, the reply information from the secondary storage server is generated by the secondary storage server based on the first log information received. When the secondary storage server successfully writes the first log information into the local log file, the secondary storage server sends a notification of writing success to the primary storage server.

Correspondingly, when the secondary storage server does not successfully write the first log information into the local log file, the secondary storage server sends a notification of writing failure to the primary storage server.

Based on the above content, the reply message from the storage server may be a notification of successful writing or a notification of writing failure.

S2111. When the reply message is a notification of successful writing, it is determined that the first log information is successfully written from the storage server.

In this embodiment, when the reply message is a notification that the writing is successful, it means that the secondary storage server has written the first log information into the local log file, and it can be determined that the secondary storage server has successfully written the first log information.

S2112. When the reply message is a notification of writing failure, it is determined that writing of the first log information from the storage server has failed.

In this embodiment, when the reply message is a notification of writing failure, it means that the slave storage server has not successfully written the first log information to the local log file, and it can be determined that the writing of the first log information from the storage server has failed.

S2200: After it is determined that the first log information has been successfully written from the storage server, determine whether there is data to be written.

In this embodiment, when the slave storage server has successfully written the first log information, it means that the master storage server has sent all log entries in the local log file to the slave storage server, and the slave storage server has sent the master All log entries in the local log file of the storage server are written to its own local log file. At this time, the log entries in the local log file of the primary storage server are synchronized with the log entries in the local log file of the secondary storage server.

In this embodiment, after the primary storage server determines that the secondary storage server has successfully written the first log information, it determines whether there is data to be written, so as to determine whether the log entries in the primary storage server’s local log file are different from the secondary storage server’s log entries. Whether the log entries in the local log file change from the synchronized state to the unsynchronized state. In other words, determining whether data is written may refer to whether a new log entry is added to the local log file of the primary storage server relative to the time point when the secondary storage server has successfully written the first log information. In addition, when determining whether there is data to be written, it can be specifically determined whether there is data to be written within a predetermined period of time. The predetermined period of time can be set according to the actual situation, for example, the distributed storage system can tolerate the log entry update delay in the storage engine. Time to set.

S2300. If it is determined that no data is written, send the second log information to the slave storage server to notify the slave storage server to store the third log information to the storage engine of the slave storage server.

Among them, the second log information does not have log entries. And the third log information contains the current set number of other latest log entries persisted from the storage server. It should be noted that although there are no log entries in the second log information, in order to ensure that the secondary storage server can store the third log information in the storage engine, the second log information may carry the currently set number of other secondary storages. The number of the latest log entry persisted by the server.

In this embodiment, the set number of other storage servers and main storage servers are most of the storage servers in the distributed storage system. In this embodiment, if it is determined that no data is written, it is determined that the log entries in the local log file of the primary storage server are still synchronized with the log entries in the local log file of the secondary storage server. At this time, the primary storage server sends a second log information without a log entry to the secondary storage server to notify the secondary storage server to store the third log information to the storage engine of the secondary storage server.

Since the third log information contains the current set number of other latest log entries persisted from the storage server, when the third log information is obtained from the storage server, the storage server can know that the currently set number of log entries Other latest log entries persisted from the storage server. At this time, the log entry contained in the third log information in the local log file and the previous log entry are read from the storage server, and stored in its own storage engine. It should be noted that the log entry contained in the third log information can be read from the storage server, and all or part of the content or part of the previous log entry can be read from the storage server, and the read content can be stored in the storage engine. When the log entry includes the uploaded form data, the number of the generated log entry, and the term number of the main storage server, part of the content may be the uploaded form data, or the content containing the uploaded form data and the number of the generated log entry.

In addition, each storage server uses its own storage engine to create, query, update, and delete data operations. Illustratively, the storage engine in MySQL may include but is not limited to: MyISAM storage engine, innoDB storage engine, MEMORY storage engine, ARCHIVE storage engine.

In this embodiment, it is first determined whether the first log information is successfully written from the storage server. Since the first log information includes the latest log entry in the local log file of the primary storage server, when it is determined that the secondary storage server has successfully written the first log information, it indicates that the log entry in the local log file of the primary storage server is the same as Synchronize log entries from the local log file of the storage server. At this time, determine whether there is data written in the local log file of the primary storage server; if it is determined that there is no data written, send the second log information to the secondary storage server to notify the secondary storage server to store the third log information, and the second log information There is no log entry; because the third log information contains the latest log entries persisted by other secondary storage servers that are currently set, although the log entries in the local log file of the primary storage server are different from those of the secondary storage server. The log entries in the log file are synchronized, but the primary storage server can still notify the secondary storage server of the latest persistent log entries that have been set by the number of other secondary storage servers in a timely manner. Further, the slave storage server reads and stores the current set number of other latest log entries persisted from the storage server and previous log entries from the local log file to the storage engine. This avoids the lag in updating log entries from the storage engine of the storage server. At this time, the log entries in the storage engine of the slave storage server are synchronized with the log entries in the storage engine of the master storage server.

In an embodiment, the above S2200 can be implemented by the following S2210-S2113:

S2210: Start timing when it is determined that the slave server has successfully written the first log information.

In an embodiment, the above-mentioned S2210 may be specifically implemented as follows: upon receiving a notification of a successful write from the slave storage server, the master storage server starts timing.

S2211, during the period when the timing time reaches the preset time, monitor whether there are log entries added to the local log file.

In an embodiment, the above-mentioned preset time may be 5 minutes, or other. This embodiment is not limited. The so-called period during which the timing time reaches the preset time is the duration from the start of the timing to the preset time.

In one embodiment, during the period when the timing time reaches the preset time, the main storage server can monitor whether the storage amount of the local log file (relative to the storage amount of the local log file corresponding to the time when the timing is started) increases to monitor Whether there are log entries added to the local log file.

In another embodiment, when the client uploads the table data to the main storage server, the main storage server generates corresponding log entries for the uploaded table data and writes them to the local log file. Therefore, during the period when the timing time reaches the preset time, the main storage server can monitor whether the table data uploaded by the client is received to monitor whether there are log entries in the local log file.

S2212. If it is determined that a log entry is added to the local log file, it is determined that data is written.

In one embodiment, when the storage capacity of the local log file (relative to the storage amount of the local log file corresponding to the start timing) becomes larger, it can be determined that there are log entries added to the local log file, and further it can be determined that data is written .

In another embodiment, when it is monitored that the form data uploaded by the client is received, it can be determined that there are log entries added to the local log file, and it is further determined that data is written.

S2213. If it is determined that no log entry is added to the local log file, it is determined that no data is written.

In one embodiment, when the storage amount of the local log file (relative to the storage amount of the local log file corresponding to the start timing) does not change, it can be determined that no log entries are added to the local log file, and it is further determined that no data is written. Into.

In another embodiment, when the monitoring itself does not receive the form data uploaded by the client, it can be determined that no log entries have been added to the local log file, and it can be further determined that no data is written.

On the basis of the foregoing S2210-S2113, the data synchronization method of the distributed storage system provided in this embodiment further includes the following S2114 and S2115:

S2114: At any time before the timing time reaches the preset time, if a log entry is detected in the local log file, stop timing.

In this embodiment, at any time before the timing time reaches the preset time, if it is detected that a log entry is added to the local log file, the log entry in the local log file of the primary storage server is different from the local log entry of the secondary storage server. The log entries in the log file change from a synchronized state to an unsynchronized state.

S2115. Send fourth log information to the slave storage server, where the fourth log information includes at least log entries added to the local log file.

In this embodiment, the log entries in the local log file of the primary storage server and the log entries in the local log file of the secondary storage server change from a synchronized state to a non-synchronized state. At this time, the primary storage server needs to increase the number of log entries in the local log file of the secondary storage server. Log entries are sent to the slave storage server, that is, additional log entries need to be sent to the slave storage server. At this time, the primary storage server uses part of the newly added log entries or all the added log entries as the fourth log information to send to the secondary storage server. It should be noted that at any time before the timing time reaches the preset time, if a log entry is detected in the local log file, it indicates that the primary storage server has synchronized the last log entry in the local log file to the secondary storage server. , When new data is written, then, while the fourth log information is sent to the slave storage server, the slave storage server can be notified of the latest log entry information that has been persisted by most storage servers, so that the slave The storage server reads the latest log entry and all previous log entries, and stores the read content in the storage engine corresponding to the slave storage server.

In one embodiment, corresponding to the foregoing S2300, the data synchronization method of the distributed storage system provided in this embodiment further includes the following S2400:

S2400. When it is determined that the first log information has not been successfully written from the storage server, receive the first serial number sent from the storage server, where the first serial number is the number of the latest log entry from the storage server.

In this embodiment, after receiving the first log information from the storage server, the first log information is matched according to the first serial number in the own local log file. If the matching fails, the secondary storage server will not write the first log information to the local log file, that is, the secondary storage server does not successfully write the first log information to the local log file. At this time, a notification of write failure is sent to the primary storage server. When the primary storage server receives the reply message of the notification of the writing failure from the secondary storage server, it is determined that the secondary storage server has not successfully written the first log information. For the specific implementation of matching the first log information according to the first serial number in its own local log file, please refer to the above about matching from the storage server according to the current latest log entry in the local log file of the slave storage server. The way to achieve it.

In one embodiment, when the primary storage server determines that the secondary storage server has not successfully written the first log information, the primary storage server may instruct the secondary storage server to report the number of the new log entry in its local log file, that is, the first serial number . It can be understood that the number of the latest log entry from the storage server is the largest number of the latest log entry from the storage server.

In another embodiment, the secondary storage server may actively report the number of the latest log entry in its local log file, that is, the first sequence number, when sending a notification of write failure to the primary storage server.

In this embodiment, when the primary storage server determines that the secondary storage server has not successfully written the first log information, when it receives the first serial number sent from the storage server, it can learn the writing in the local log file from the storage server. Information about the log entry.

On the basis of the foregoing S2400, the data synchronization method of the distributed storage system provided in this embodiment further includes the following S2500:

S2500. Send fifth log information to the slave storage server according to the first serial number, where the fifth log information includes log entries corresponding to all consecutive serial numbers between the first serial number and the second serial number, where the second The serial number is the number corresponding to the log entry with the latest write time in the local log file.

In this embodiment, since the second serial number is the number corresponding to the log entry with the latest writing time in the local file, that is, the master server forwards the latest log entry in the local log file to the slave server. Therefore, if the master server After determining that the secondary server has successfully written the fifth log information, you can continue to determine whether there is data written. If it is determined that no data has been written, send the second log information to the secondary storage server to notify the secondary storage server to store the third log Information steps.

In this embodiment, the fifth log information includes all log entries corresponding to consecutive serial numbers between the first serial number and the second serial number, including log entries corresponding to the second serial number.

In this embodiment, when it is determined that the slave storage server has not successfully written the first log information, the master storage server sends the fifth log information to the slave storage server. Based on this, when the first log information is received from the storage server and the log entries in the fifth log information are written to the local log file, the local log file of the primary storage server and the local log file of the secondary storage server Log entries are synchronized. In this way, compared to the traditional one, when it is determined that the slave storage server has not successfully written the first log information, the master storage server only sends the first log information and the log entry preceding the log entry with the smallest number in the first log information to From the storage server, the local log file of the primary storage server can be quickly synchronized with the log entries in the local log file of the secondary storage server.

<Device Example>

As shown in FIG. 3, this embodiment also provides a data synchronization device 30 of a distributed storage system. The device 30 includes a first determining module 31, a second determining module 32, and a sending module 33. Wherein: the first determination module 31 is configured to determine whether the slave storage server successfully writes the first log information, the first log information includes the latest log entry in the local log file of the master storage server; the second determination The module 32 is used for determining whether there is data writing after it is determined that the secondary storage server has successfully written the first log information; the sending module 33 is used for sending data to the secondary storage server if it is determined that there is no data writing. Sending the second log information to notify the secondary storage server to store the third log information, the second log information does not have log entries; wherein the third log information includes the currently set number of other secondary storage servers The latest persistent log entry.

In one embodiment, the second determining module 32 is specifically configured to: start timing when it is determined that the slave server has successfully written the first log information; and monitor when the timing time reaches a preset time Whether there are log entries added to the local log file; if it is determined that there are log entries added to the local log file, it is determined that data is written; if it is determined that there are no log entries added to the local log file, it is determined that there is no data Write.

In an embodiment, the data synchronization device 30 of the distributed storage system provided in this embodiment further includes a second sending module. Wherein, the second sending module is configured to: at any time before the timing time reaches the preset time, if it is detected that a log entry is added to the local log file, stop timing; Log information, where the fourth log information includes at least log entries added to the local log file.

In one embodiment, the second determining module 32 is further configured to: when it is determined that the slave storage server has not successfully written the first log information, receive the first sequence number sent by the slave storage server, and the first sequence The number is the number of the latest log entry from the storage server.

In an embodiment, the data synchronization device 30 of the distributed storage system provided in this embodiment further includes a third sending module. Wherein, the third sending module is configured to: send fifth log information to the slave storage server according to the first serial number, wherein the fifth log information includes the first serial number to the second serial number. The log entries corresponding to all consecutive serial numbers in between, where the second serial number is the number corresponding to the log entry with the latest writing time in the local log file.

In one embodiment, the first determining module 31 is specifically configured to: receive reply information from the secondary storage server; when the reply information is a notification of successful writing, determine that the secondary storage server successfully writes the first log information ; When the reply message is a notification of writing failure, it is determined that the writing of the first log information from the storage server fails.

<Equipment Example>

As shown in FIG. 4, an embodiment of the present application also provides a storage server 40 of a distributed storage system. The storage server 40 includes the data synchronization device 30 of the distributed storage system in the foregoing device embodiment. Alternatively, the storage server of the distributed storage system includes a storage 41 and a processor 42. The memory is used to store executable instructions, and the instructions are used to control the processor to execute the method according to any one of the above method embodiments.

<Computer storage media>

The present application also provides a computer storage medium that stores computer instructions, and when the computer instructions in the storage medium are executed by a processor, the method according to any one of the above method embodiments is implemented.

In yet another embodiment of the present application, there is also provided a computer program product that may include a computer-readable storage medium, which carries computer-readable program instructions for enabling a processor to implement various aspects of the present application.

In another embodiment of the present application, there is also provided a computer program product containing instructions, which when run on a computer, causes the computer to execute the steps of any data synchronization method in the foregoing embodiments.

The computer-readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a printer with instructions stored thereon The protruding structure in the hole card or the groove, and any suitable combination of the above. The computer-readable storage medium used here is not interpreted as the instantaneous signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses through fiber optic cables), or through wires Transmission of electrical signals.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing/processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network, and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device .

The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, status setting data, or in one or more programming languages. Source code or object code written in any combination, the programming language includes object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as "C" language or similar programming languages. Computer-readable program instructions can be executed entirely on the user's computer, partly on the user's computer, executed as a stand-alone software package, partly on the user's computer and partly executed on a remote computer, or entirely on the remote computer or server carried out. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to access the Internet). connection). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can be customized by using the status information of the computer-readable program instructions. The computer-readable program instructions are executed to realize various aspects of the present application.

Here, various aspects of the present application are described with reference to flowcharts and/or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each block of the flowcharts and/or block diagrams, and combinations of blocks in the flowcharts and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine that makes these instructions when executed by the processor of the computer or other programmable data processing device , A device that implements the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams is produced. It is also possible to store these computer-readable program instructions in a computer-readable storage medium. These instructions make computers, programmable data processing apparatuses, and/or other devices work in a specific manner. Thus, the computer-readable medium storing the instructions includes An article of manufacture, which includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

It is also possible to load computer-readable program instructions on a computer, other programmable data processing device, or other equipment, so that a series of operation steps are executed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , So that the instructions executed on the computer, other programmable data processing apparatus, or other equipment realize the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions, and operations of the system, method, and computer program product according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of an instruction, and the module, program segment, or part of an instruction contains one or more components for realizing the specified logical function. Executable instructions. In some alternative implementations, the functions marked in the block may also occur in a different order than the order marked in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions Or it can be realized by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that realization by hardware, realization by software, and realization by a combination of software and hardware are all equivalent.

The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Without departing from the scope and spirit of the described embodiments, many modifications and changes are obvious to those of ordinary skill in the art. The choice of terms used herein is intended to best explain the principles, practical applications, or technical improvements of the technologies in the market, or to enable other ordinary skilled in the art to understand the embodiments disclosed herein. The scope of the application is defined by the appended claims.

Industrial applicability

In the solution provided by this application, it is first determined whether the first log information is successfully written from the storage server. Since the first log information includes the latest log entry in the local log file of the primary storage server, when it is determined that the secondary storage server has successfully written the first log information, it indicates that the log entry in the local log file of the primary storage server is the same as Synchronize log entries from the local log file of the storage server. At this time, determine whether there is data written in the local log file of the primary storage server; if it is determined that there is no data written, send the second log information to the secondary storage server to notify the secondary storage server to store the third log information, and the second log information There is no log entry; because the third log information contains the latest log entries persisted by other secondary storage servers that are currently set, although the log entries in the local log file of the primary storage server are different from those of the secondary storage server. The log entries in the log file are synchronized, but the primary storage server can still notify the secondary storage server of the latest persistent log entries that have been set by the number of other secondary storage servers in a timely manner. Further, the slave storage server reads and stores a set number of other latest log entries persisted from the storage server and previous log entries from the local log file to the storage engine. This avoids the lag in updating the log entries in the storage engine of the slave storage server. At this time, the log entries in the storage engine of the slave storage server are synchronized with the log entries in the storage engine of the primary storage server.

Claims (11)

1. A data synchronization method for a distributed storage system. The distributed storage system includes a primary storage server and a secondary storage server. The method is implemented by the primary storage server and includes:

   Determining whether the slave storage server successfully writes first log information, where the first log information includes the latest log entry in the local log file of the master storage server;

   After it is determined that the slave storage server has successfully written the first log information, determining whether there is data to be written;

   If it is determined that no data is written, sending second log information to the slave storage server to notify the slave storage server to store the third log information, and the second log information does not have a log entry;

   Wherein, the third log information includes a set number of other latest log entries persisted from the storage server.
2. The method according to claim 1, wherein the determining whether data is written after it is determined that the slave storage server has successfully written the first log information comprises:

   Start timing when it is determined that the slave server has successfully written the first log information;

   During the period when the timing time reaches the preset time, monitoring whether there are log entries added to the local log file;

   If it is determined that there are log entries added to the local log file, it is determined that data is written;

   If it is determined that no log entry is added to the local log file, it is determined that no data is written.
3. The method according to claim 2, wherein the method further comprises:

   At any time before the timing time reaches the preset time, if it is detected that a log entry is added to the local log file, stop timing;

   Sending fourth log information to the slave storage server, where the fourth log information includes at least log entries added to the local log file.
4. The method according to any one of claims 1-3, wherein the method further comprises:

   When it is determined that the slave storage server has not successfully written the first log information, receiving the first serial number sent by the slave storage server, where the first serial number is the number of the latest log entry in the slave storage server.
5. The method according to claim 4, wherein the method further comprises:

   According to the first serial number, send fifth log information to the slave storage server, where the fifth log information includes logs corresponding to all consecutive serial numbers between the first serial number and the second serial number entry;

   Wherein, the second serial number is the number corresponding to the log entry with the latest writing time in the local log file.
6. The method according to any one of claims 1 to 3, wherein the determining whether the slave storage server successfully writes the first log information comprises:

   Receiving reply information from the slave storage server;

   When the reply message is a notification of successful writing, it is determined that the slave storage server successfully writes the first log information;

   When the reply information is a notification of writing failure, it is determined that the writing of the first log information from the storage server has failed.
7. A data synchronization device of a distributed storage system includes:

   A first determining module, configured to determine whether the slave storage server successfully writes first log information, where the first log information includes the latest log entry in the local log file of the master storage server;

   The second determining module is configured to determine whether data is written after it is determined that the slave storage server has successfully written the first log information;

   A sending module, configured to send second log information to the slave storage server if it is determined that no data is written, so as to notify the slave storage server to store third log information, and the second log information does not have a log entry;

   Wherein, the third log information includes a set number of other latest log entries persisted from the storage server.
8. The device according to claim 7, wherein the second determining module is configured to:

   Start timing when it is determined that the slave server has successfully written the first log information;

   During the period when the timing time reaches the preset time, monitoring whether there are log entries added to the local log file;

   If it is determined that there are log entries added to the local log file, it is determined that data is written;

   If it is determined that no log entry is added to the local log file, it is determined that no data is written.
9. A storage server of a distributed storage system, including: a processor and a memory;

   The memory is used to store executable instructions, and the instructions are used to control the processor to execute the method according to any one of claims 1-6.
10. A computer storage medium, the storage medium stores computer instructions, and when the computer instructions in the storage medium are executed by a processor, the method according to any one of claims 1-6 is implemented.
11. A computer program product, when it runs on a computer, causes the computer to execute the method according to any one of claims 1-6.

Images