WO2021212493A1

WO2021212493A1 - Data synchronization method and device, data storage system and computer-readable medium

Info

Publication number: WO2021212493A1
Application number: PCT/CN2020/086773
Authority: WO
Inventors: 汤见乐
Original assignee: 深圳市欢太科技有限公司; Oppo广东移动通信有限公司
Priority date: 2020-04-24
Filing date: 2020-04-24
Publication date: 2021-10-28
Also published as: CN115004662A

Abstract

A data synchronization method and device, a data storage system and a computer-readable medium, which relates to the technical field of computers, the method comprising: acquiring target data requested to be written into the data storage system (S201); writing the target data to a master room (S202); and after successfully writing the target data to the master room, synchronizing the target data to a slave room (S203). The method first writes data to the master room, and on the basis of the data in the master room, synchronizes, to the slave room, the data written to the master room, which may better maintain data consistency.

Description

Data synchronization method, device, data storage system and computer readable medium

Technical field

This application relates to the field of computer technology, and more specifically, to a data synchronization method, device, data storage system, and computer readable medium.

Background technique

With the increasing popularity of network services, the amount of user visits is increasing. If all business data is stored on a server, the server will often fail due to excessive visits, and is limited by the storage space of the server In an application scenario with a large amount of business data, one server often cannot carry all the business data. Therefore, multiple servers are required to provide services for users together. To this end, multiple servers that are commonly used to provide a service can be set up in the same computer room, and then multiple computer rooms form a system, and the system provides data storage functions for the business layer.

However, when data is stored in the business layer, each computer room in the system needs to be written once. Then, if the data in some computer rooms is written incorrectly, the data in the entire system will be inconsistent.

Summary of the invention

This application proposes a data synchronization method, device, data storage system, and computer readable medium to improve the above-mentioned defects.

In the first aspect, an embodiment of the present application provides a data synchronization method, which is applied to a host room of a data storage system, and the data storage system further includes a slave computer room corresponding to the host room. The method includes: obtaining target data requested to be written into the data storage system; writing the target data into the host room; after successfully writing the target data into the host room, writing the target data Synchronize to the slave room.

In a second aspect, an embodiment of the present application also provides a data synchronization device, which is applied to a host room of a data storage system, and the data storage system further includes a slave computer room corresponding to the host room. The device includes: an acquisition unit, a writing unit, and a synchronization unit. The obtaining unit is configured to obtain the target data requested to be written into the data storage system. The writing unit is used to write the target data into the host room. The synchronization unit is configured to synchronize the target data to the slave computer room after successfully writing the target data into the host room.

In a third aspect, an embodiment of the present application also provides a data storage system, including: a host room and a slave computer room corresponding to the host room; the host is used to execute the above method.

In a fourth aspect, the embodiments of the present application also provide a computer-readable medium, the readable storage medium storing a program code executable by a processor, and when the program code is executed by the processor, the processor Perform the above method.

The data synchronization method, device, data storage system and computer readable medium provided in this application will be used when the data storage system is requested to write target data, that is, whether it is requested to write data in the host room of the system or from the computer room. The target data is written into the host room first, and then, after the target data is successfully written into the host room, the target data is synchronized to the slave computer room. Therefore, not only can it avoid that when the target data is written into multiple computer rooms at the same time, if the data in a certain computer room is written incorrectly, it will cause the data in each computer room to be inconsistent. The data is subject to the synchronization of the data written in the host room to the slave room, which can better maintain data consistency.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

Figure 1 shows a schematic diagram of a data storage system provided by an embodiment of the present application;

FIG. 2 shows a method flowchart of a data synchronization method provided by an embodiment of the present application;

FIG. 3 shows a method flowchart of a data synchronization method provided by another embodiment of the present application;

FIG. 4 shows a flowchart of the method S320 in FIG. 3 according to an embodiment of the present application;

FIG. 5 shows a method flowchart of a data synchronization method provided by another embodiment of the present application;

FIG. 6 shows a flowchart of the method S540 in FIG. 5 provided by an embodiment of the present application;

FIG. 7 shows a schematic diagram of a client in a data storage system provided by an embodiment of the present application;

FIG. 8 shows a block diagram of a data synchronization device provided by another embodiment of the present application;

FIG. 9 shows a block diagram of a data synchronization device provided by another embodiment of the present application;

FIG. 10 shows a block diagram of modules of a data synchronization device provided by still another embodiment of the present application;

FIG. 11 shows a schematic diagram of an electronic device provided by an embodiment of the present application;

Fig. 12 is a storage unit for storing or carrying program code for implementing the data synchronization method according to the embodiment of the present application according to an embodiment of the present application.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application.

With the increasing popularity of network services, the amount of user visits is increasing. If all business data is stored on a server, the server will often fail due to excessive visits, and is limited by the storage space of the server In an application scenario with a large amount of business data, one server often cannot carry all the business data. Therefore, multiple servers are required to provide services for users together. To this end, multiple servers that are commonly used to provide a service can be set up in the same computer room, and the computer room can provide the service to users.

Among the above-mentioned data management services, in-memory databases are widely used. In-memory databases refer to databases that put data in memory and operate directly, and are generally deployed in a cluster. Such clusters are called in-memory database clusters. An in-memory database cluster includes an in-memory database master and an in-memory database slave, and data needs to be synchronized between the two. Further, synchronization is divided into full synchronization and incremental synchronization. Full synchronization means that the in-memory database slave synchronizes all the data of the in-memory database host at one time. Incremental synchronization means that the in-memory database slave only synchronizes it with the in-memory database host. Synchronize the different parts of the data.

Among them, Redis is a high-performance open source database that can be based on memory. It is written in ANSIC language and provides programming interfaces in multiple languages. It is often used to implement functions such as software or application data storage and fast access; in a production environment, redis applications It is very extensive. For example, in order to ensure the stability of the service while expanding the business, the redis cluster that supports distributed high availability and horizontal expansion is often used for data storage and access. In order to improve the stability of the system, the redis system usually adopts the master-slave structure. In the initialization phase of the redis slave node and when the master node is writing new data, it is necessary to copy the data on the redis master node for backup. This event is called redis master-slave data synchronization.

As shown in FIG. 1, FIG. 1 shows a schematic diagram of a data storage system. The data storage system includes multiple computer rooms. As shown in FIG. 1, the multiple computer rooms include a computer room 1, a computer room 2 and a computer room 3. Each computer room includes a host computer and at least one slave computer corresponding to the host computer. As shown in FIG. 1, the computer room 1 includes a host 10 and a slave 11 and a slave 12 corresponding to the host. The host 10 and the slave 11 and the slave 12 are in a master-slave relationship. Among them, the host is also called the master node (master), which provides external services. The slave is also called the slave. The slave replicates the data of the Master node in real time to ensure that it is consistent with the Master node. The slave machine cannot be performed by other devices to perform data writing operations. The data in the slave machine is synchronized by the master to the slave machine through data synchronization.

In addition, as shown in FIG. 1, the computer room 2 includes a host 20 and slaves 21 and 22 corresponding to the host, and the computer room 3 includes a host 30 and slaves 31 and 32 corresponding to the host.

It should be noted that the data storage system formed by each computer room can provide data storage services for the business layer, that is, the data generated by the business layer can be stored in the data storage system, and data can also be read from the data storage system.

In some embodiments, the business layer can write data into the data storage system. The way of writing can be to write the data into any computer room in the data storage system. Specifically, it can be to write data into the data storage system. In the host computer of any computer room, in order to ensure the consistency of the data, the written data needs to be written in each computer room.

However, the inventor found in his research that the current implementation of cross-computer room multi-active solutions based on Redis adopts multi-write solutions, that is, when the business layer is storing data, it needs to perform a write operation on the storage clusters of each computer room. Data storage is considered successful only after all storage clusters in the computer room have been successfully written. However, when data is stored in the business layer, each computer room in the system needs to be written once. Then, if the data in some computer rooms is written incorrectly, the data in the entire system will be inconsistent.

Therefore, in order to overcome the above shortcomings, in the embodiment of the present application, multiple computer rooms in the data storage system are divided into a host room and a slave computer room, that is, among multiple computer rooms in the data storage system, one computer room is designated as the host room, The other computer rooms are the slave computer rooms corresponding to the host room.

The embodiment of the present application provides a data synchronization method, which is applied to the above-mentioned data storage system. As an implementation manner, each computer room in the data storage system is provided with a client corresponding to the data synchronization method. Write to a certain computer room, and execute the embodiment of the method described in this application. As an implementation manner, the client may be installed in a host in each computer room, or may be installed in a terminal independent of the host in each computer room. In the embodiment of the present application, the client is installed in the host.

As shown in FIG. 2, FIG. 2 shows a data synchronization method. The execution subject of the method may be the above-mentioned host room. Specifically, the execution subject may be a client in the host room. Specifically, the method includes: S201 to S203.

S201: Obtain target data requested to be written to the data storage system.

As an implementation manner, the data storage system may obtain a data write request, and the data write request is used to write the target data into the data storage system. Specifically, the business layer may send a data write request to the data storage system.

In some embodiments, the data writing request may include a label designating the computer room requested to be written, and the label is used as the identity information of the computer room. Then in this embodiment, the business layer requests to write the target data into the computer room corresponding to the designated label. In other embodiments, the data write request may not include a label designating the computer room where the write is requested.

Regardless of whether it is specified to be written to a certain computer room or not specified to be written to a certain computer room, as long as the data corresponding to the data write request requesting to be written to the data storage system, it can be used as the target data requested to be written to the data storage system.

Wherein, the host room can monitor and write data in each computer room (including the host room and the slave room corresponding to the host room), so that the host room can obtain the target data requested to be written into the data storage system.

As an implementation manner, both the master room and the slave computer room have message queues, the master room subscribes to the message queues of all the slave computer rooms, and each of the slave computer rooms subscribes to the message queue of the master room. Communication between each computer room and between application programs such as the business layer and each computer room is through a message queue. Specifically, the message queue (MQ) is an application-to-application communication method. Application programs communicate by writing and retrieving application-specific data (messages) in and out of the queue.

For example, when the business layer requests to write target data to a certain computer room, it will send a data write request to the message queue of the computer room. If the computer room is a slave computer room, because the master room subscribes to the message queue of the slave computer room, the host The room can monitor the operation of the slave computer room on the message queue, so that when a data writing request is written into the message queue, it can be monitored by the host room, and then the target data requested to be written into the data storage system can be obtained.

S202: Write the target data into the host room.

After obtaining the target data requested to be written into the data storage system, the target data is written into the host room.

As an implementation manner, the host room includes a host computer and a slave device corresponding to the host computer. The implementation manner of writing the target data into the host computer room may be to write the target data into the host computer, and then by The master synchronizes the target data to the slave corresponding to the master.

Assume that the aforementioned computer room 1 is a host room and the target data is data data. The host 10 will synchronize the data to the slave 11 and the slave 12 corresponding to the host.

As an implementation manner, each computer room is provided with a log pipelining function, which is used to record the operation of the host writing data. In the Redis cluster cross-computer room multiple activity solution based on log pipelining, all Redis data writes The input will be saved in the log stream of the local disk. The external scanner scans the log stream and synchronizes the data modification to the Redis clusters of other computer rooms. Each Redis cluster can be understood as the aforementioned computer room. The log stream can record each write operation of each computer room, so as to realize data synchronization.

Specifically, taking the above storage system as the Redis system as an example, modify the source code implementation of the Redis component, add the binlog (ie log pipeline) function, record each write operation in the log file, and ensure the order of log records and the actual reception The sequence of the write operations is the same. For example, to write multiple data to the host, the multiple data are data1, data2, data3, data4, data5, and data6 respectively, that is, the writing order of the multiple data is data1, data2, data3, data4, data5, and data6 , The multiple data are recorded in the log file, and the order of the log recording is also data1, data2, data3, data4, data5, and data6. Among them, data1, data2, data3, data4, data5, and data6 can be the multiple The identification of the data can also record the writing time of each written data in the log record.

In addition, both the master and the slave can have log files, that is, both the master and the slave can execute the log pipelining function, which is used to record the write operations of the master and the slave. Among them, the write operation for the slave can be understood as a master pair The data synchronization operation of the slave.

Therefore, through the above-mentioned recording of the written data, the target data written by the host can be obtained, where the target data can be a collection of multiple data continuously written, or multiple data continuously written One of the data in.

S203: After successfully writing the target data into the host room, synchronize the target data to the slave computer room.

Specifically, it is determined whether the target data is successfully written into the host room, and if the target data is successfully written into the host room, the target data is synchronized to the slave computer room.

Wherein, it is determined whether the target data is successfully written into the host room in the implementation manner, and after the target data is written into the host, it is determined whether the target data is successfully synchronized to the slave machine corresponding to the host. Specifically, how to determine whether the target data is successfully synchronized to the slave machine corresponding to the master can refer to the subsequent embodiments.

After the target data is written in the host room, the target data is stored in the storage space corresponding to the host room, and then the data synchronization operation is performed. Specifically, the above-mentioned client may send the target data written in the host room to the host The slave room corresponding to the room allows the data stored in the slave room to be updated. In other words, the target data is synchronized from the storage space corresponding to the computer room.

If the target data is successfully synchronized to the slave corresponding to the master, it can be determined that the target data written to the master has been successfully synchronized to the slave corresponding to the master, and then the target data can be synchronized to the slave at this time. From the computer room.

Taking the above-mentioned data storage system as an example, the host 10 in the computer room 1 writes target data, and the host 10 synchronizes the target data to the slave 11 and the slave 12. After determining that the slave 11 and the slave 12 successfully synchronize the target data, The target data is sent to the computer room 2 and the computer room 3, and the computer room 2 and the computer room 3 execute the synchronization operation of the target data after obtaining the target data. Specifically, the host 10 sends target data to the host 20 in the computer room 2 and the host 30 in the computer room 3. The host 20 writes the target data, and synchronizes the target data to the slave 21 and the slave 22 corresponding to the host 20. The host 30 is written with target data, and the target data is synchronized to the slave 31 and the slave 32 corresponding to the host 20.

Therefore, when the data storage system is requested to write target data, that is, whether it is requested to write data in the host room of the system or from the computer room, the target data will be written to the host room first, and then, after the target is successfully written After the data is written into the host room, the target data is synchronized to the slave computer room. Therefore, not only can it avoid that when the target data is written into multiple computer rooms at the same time, if the data in a certain computer room is written incorrectly, it will cause the data in each computer room to be inconsistent. The data is subject to the synchronization of the data written in the host room to the slave room, which can better maintain data consistency.

Please refer to FIG. 3, which shows a data synchronization method. The execution subject of the method may be the above-mentioned host room. Specifically, the execution subject may be a client in the host room. Specifically, the method includes: S310 to S340.

S310: Monitoring the write operation of the host room and the slave computer room.

As an implementation manner, the client in the host in the host room can detect write operations for all computer rooms. For example, through the above message queue, the host room subscribes to the message queues of all slave rooms, and then writes in any slave room When data, the write operation from the computer room can be monitored, and when the host room itself is writing data, the write operation will also be written to the message queue of the host room, thus subscribing to the message of the host room The corresponding slave computer room can also know that data is written in the host room, and any computer room in the data storage system can be regarded as a write operation to the data storage system.

Therefore, by monitoring the message queues of each computer room, it is possible to detect the current write operation for the data storage system when data is written in any computer room.

S320: When the write operation is detected, obtain the target data according to the write operation.

Among them, the write operation can specify the data requested to be written. For example, the write operation can be a data packet that includes the data requested to be written. The host room can analyze the data when the write operation is monitored. The write operation obtains the data requested to be written.

As an implementation manner, when there is only one write operation, the data requested to be written may be used as the target data.

As another implementation manner, if there are multiple write operations, the data requested to be written corresponding to each write operation is determined. If the data requested to be written for all write operations are the same, you can choose one to write. The data requested to be written corresponding to the input operation is used as the target data.

As yet another implementation manner, if there are multiple write operations, then it is determined that the data modification request of each write operation is the same, and a write operation with the same data modification request is found. Among them, the data modification request can be that the data requested to be written is the same and the data object requested to be modified is the same. The data object requested to be modified can be a certain content to be modified stored in the host room or the slave computer room, or it can be Newly added data. Specifically, the write operation corresponds to the content to be modified and the data requested to be written, and the write operation is used to modify the content to be modified into the data requested to be written.

If the write operation is an update request, the content to be modified may be certain designated data stored in the computer room, and when the write operation is performed, the designated data is modified to the data requested to be written. If the write operation is newly added data, the content to be modified is the storage space in the computer room, and when the write operation is performed, the data requested to be written is written in the storage space.

Therefore, when there are multiple write operations, the content to be modified corresponding to each write operation is determined, and multiple write operations are distinguished according to the content to be modified to obtain write operations corresponding to different content to be modified, that is, Write operations for different modification purposes. In addition, if the content to be modified is specified data, each data in each computer room has a corresponding identifier, and the identifier is unified in each computer room. For example, if the data stored in the host room includes data1, The data stored in the computer room is also included in data1, and data1 in the host room and data1 in the slave computer room should maintain data consistency.

Therefore, if the content to be modified is designated data, the identifier of the designated data is obtained, and a write operation with the same identifier of the designated data is regarded as a write operation with the same modification purpose. If the content to be modified is a storage space, each write operation in which the content to be modified is a storage space can be used as a modification purpose write operation, or if the write operation is a newly added data, it will be The write operation configures a data identifier, and the data identifier is a newly added identifier. When data is written into the computer room, the data identifier also corresponds to the computer room.

For example, the write operations are operation 1, operation 2, operation 3, operation 4, operation 5, operation 6, and operation 7. The content to be modified corresponding to operation 1, operation 2, and operation 3 is data1, and operation 4 and operation 5 are used For adding a piece of data, the identifier of the added data is data7, operation 6 is used to add a piece of data, the identifier of the added data is data8, operation 7 is used to add a piece of data, and the identifier of the added data is data9.

Then the above-mentioned 7 write operations can be divided into four modification goals, namely modification goal 1, modification goal 2, modification goal 3, and modification goal 4. The write operations corresponding to modification goal 1 are operation 1, operation 2, and operation. 3. The write operations corresponding to modification purpose 2 are operations 4 and 5, the write operation corresponding to modification purpose 3 is operation 6, and the write operation corresponding to modification purpose 4 is operation 7.

Therefore, for different modification purposes, the write operation corresponding to each modification purpose is determined. Specifically, for the implementation of determining the write operation corresponding to each modification purpose, reference may be made to steps S321 to S323 in FIG. 4. For each modification purpose, S321 to S323 can be executed to determine the write operation corresponding to each modification purpose.

S321: If there are multiple write operations, obtain a timestamp corresponding to each write operation.

Wherein, each of the write operations corresponds to designated data, and is used to modify the content to be modified into designated data. The designated data corresponding to each write operation can be the same or different.

As an implementation manner, each write operation corresponds to a time stamp, and the time stamp is used to record the time corresponding to the write operation. Specifically, when a write request requesting to write specified data is input to the data storage system at the business layer, the data storage system configures a timestamp for the write request and treats the write request as a write operation, thereby , Can configure a time stamp for the write operation.

S322: Determine the target write operation from among the multiple write operations according to the timestamp.

Wherein, the timestamp can be used as the time when the specified data is requested to be written corresponding to the write operation, that is, it can be used as the write time of a different write operation. As an implementation manner, the target write operation may be determined according to the order of the time stamp of each write operation. For example, select the write operation corresponding to the earliest or latest time stamp as the target write operation.

Specifically, the timestamps corresponding to multiple write operations may be the same or may not be all the same, so for all the same and not all the same, according to the timestamp, the target write operation is determined from the multiple write operations. The implementation can be different.

Specifically, it is determined whether the time stamps corresponding to each of the write operations are all the same, and if the time stamps are not all the same, the latest time stamp among all the time stamps is determined as the target time stamp.

Among them, the time stamps are not all the same, and it may be that all time stamps have the same time stamp or different time stamps, that is, not all time stamps are the same. Then, all the time stamps can be sorted in chronological order to obtain a time series, and the last time stamp in the time series can be used as the target time stamp.

Then, the write operation corresponding to the target timestamp is used as a write operation to be selected. Among them, the latest time stamp may include multiple time stamps, that is, multiple latest time stamps can be obtained, and the write operation corresponding to each latest time stamp can be used as a write operation to be selected.

As another implementation manner, all the write operations may be sorted according to the time sequence of the timestamp to obtain the candidate sequence. Wherein, the sequence to be selected includes multiple write operation identifiers, and the write operation identifiers may be used to record the identity information of each write operation. And in the candidate sequence, the identifier of each write operation corresponds to a serial number, and the earlier the serial number the write operation corresponds to the earlier the timestamp, that is, the write operations are sorted according to the chronological order of the timestamps, Get the candidate sequence. Then, the first write operation in the sequence to be selected is regarded as the write operation to be selected. Among them, if there are multiple write operations with the same time stamp, the sequence numbers corresponding to the write operations of the same time stamp are the same. Therefore, the obtained candidate write operations can be multiple, where each The timestamps corresponding to the write operations to be selected are the same, and among the timestamps corresponding to all write operations, the timestamp corresponding to each write operation to be selected is the latest timestamp.

Then, the target write operation is determined from the to-be-selected write operation. If the number of the write operation to be selected is 1, then the write operation to be selected is taken as the target write operation. If the number of the write operations to be selected is at least two, an operation object corresponding to each write operation to be selected is determined, and the operation object includes a host room or a slave computer room. Among them, the operation object can be the computer room designated to be written when the business layer requests to store data. Specifically, the data write request includes the identifier of the computer room, and when the host room and the slave computer room are preset, the computer room can be The identification of each computer room corresponds to a tag, which includes the host room tag and the slave computer room tag. Therefore, through the identification of the computer room included in the data write request, it can be determined whether the tag of the computer room corresponding to the identification of the computer room is the host room tag or From the computer room tag, it can then be determined whether the computer room requested to write data is the host room or the slave computer room, that is, the operation object corresponding to each write operation can be determined.

The write operation to be selected in the host room as the operation object is taken as the target write operation.

For example, take the above-mentioned multiple write operations including operation 1, operation 2, operation 3, operation 4, operation 5, operation 6 and operation 7 as an example, and operation 1, operation 2, operation 3, operation 4, operation 5. The time stamps corresponding to operation 6 and operation 7 are time t1, time t2, time t3, time t4, time t5, time t6, and time t7. In chronological order, the time sequence of each time stamp is time t1, time t2, (time t3 and time t4), (time t5, time t6, and time t7) respectively, that is, time t3 and time t4 are the same, time t5, Time t6 and time t7 are the same time, it can be understood that operation 1 is obtained first, operation 2 is obtained later, operation 3 and operation 4 are obtained at the same time, and operation 5, operation 6 and operation are obtained at the same time. 7.

Then, it can be determined that the write operations to be selected are operation 5, operation 6, and operation 7, and then, a target write operation is determined from operation 5, operation 6 and operation 7.

As an implementation manner, the operation object corresponding to operation 5, operation 6 and operation 7 may be determined. For example, the operation objects corresponding to operation 5, operation 6 and operation 7 are master room, slave room 1 and slave room 2 respectively. . Then, the write operation to be selected in the host room as the operation object is operation 5, so that it can be determined that the target write operation is operation 5.

In addition, if there is only one write operation to be selected, for example, operation 1, then operation 1 can be directly used as the target write operation.

As another implementation manner, if the time stamps corresponding to each of the write operations are all the same when the write operation is detected, it is not necessary to perform the write operation of finding the latest time stamp, and directly obtain each time stamp. For the operation object corresponding to the write operation, the operation object can refer to the foregoing description, which will not be repeated here.

Specifically, if the timestamps of each write operation are all the same, the operation object corresponding to each of the write operations to be selected is determined, and the operation object includes the host room or the slave computer room; and the operation object is the host room's The write operation to be selected is regarded as the target write operation.

S323: Use the designated data corresponding to the target write operation as target data.

Exemplarily, suppose that the currently acquired write operation includes operation 1, operation 2, operation 3, operation 4, operation 5, operation 6, operation 7, operation 8, operation 9 and operation 10, in which operation 1, The content to be modified corresponding to operation 2, operation 3, operation 4, operation 5, operation 6 and operation 7 is data1, and the content to be modified corresponding to operation 8, operation 9 and operation 10 is data7, and data1 exists in all computer rooms, The purpose of operation 1, operation 2, operation 3, operation 4, operation 5, operation 6 and operation 7 is to modify the data of data1, assuming that operation 1, operation 2, operation 3, operation 4, operation 5, operation 6 and operation 7 correspond The specified data are data 1, data 2, data 3, data 4, data 5, data 6, and data 7. Assume that operation 1, operation 2, operation 3, operation 4, operation 5, operation 6 and operation 7 correspond to the operation The objects are slave room 1, slave room 2, master room, slave room 3, master room, slave room 1 and slave room 2. For example, the purpose of operation 1 is to modify data1 stored in computer room 1 to data 1.

Then, if the timestamps corresponding to operation 5, operation 6 and operation 7 are the latest, it can be determined that the target write operation is operation 5 in the host room, and the determined target data is data 5. Finally, the modification of data1 is performed this time The method is determined to modify the data data1 in the master room to data 5, and synchronize the data 5 to other slave computer rooms, so that all data1 in the slave computer rooms are modified to data 5.

Then, after completing the operation of modifying data1 of each computer room to data 5, and then modify data7 of all computer rooms according to operation 8, operation 9 and operation 10, the method of determining the target data can refer to the foregoing, and the modification method is also to modify the host computer first data7, and then synchronize sequentially.

S330: Write the target data into the host room.

S340: After successfully writing the target data into the host room, synchronize the target data to the slave computer room.

Please refer to FIG. 5. FIG. 5 shows a data synchronization method. The execution subject of the method may be the aforementioned host room. Specifically, the execution subject may be a client in the host room. Specifically, the method includes: S510 to S550.

S510: Obtain the target data requested to be written to the data storage system.

S520: Write the target data into the host room.

S530: After successfully writing the target data into the host room, synchronize the target data to the designated computer room.

Wherein, the designated machine room is one of the plurality of slave machine rooms.

As an implementation manner, each slave room can be set to a synchronization state, the synchronization state includes unsynchronized and synchronized, then each time a slave room is selected from the unsynchronized slave room as the designated room, the target data Synchronize to the specified computer room. When the target data is successfully synchronized to the specified computer room, the synchronization status corresponding to the specified computer room will be changed, for example, change the synchronization status from unsynchronized to synchronized.

As an implementation manner, a synchronization sequence may be preset, and the host room synchronizes the target data to all the slave computer rooms according to the synchronization sequence, and then selects a slave computer room as the designated computer room in sequence according to the synchronization sequence.

S540: Determine whether the data is successfully synchronized to the designated computer room.

As an implementation manner, it can be determined whether the target data is successfully synchronized to the slave computer room corresponding to the master room according to the information replies from the computer room, or it can be based on the above-mentioned data writing record of the master room and the slave computer room, that is, The log files of the host room and the slave computer room are used to determine whether the target data is successfully synchronized to the slave computer room corresponding to the host room.

Specifically, an implementation manner for determining whether the data is successfully synchronized to the designated computer room may be to detect whether the synchronization success information fed back by the designated computer room is received; if it is received, it is determined that the data is successfully synchronized to the designated computer room. ; If it is not received, it is determined that the data has not been successfully synchronized to the designated computer room.

The successful synchronization information may be sent from the computer room to the host room after the target data is received and successfully stored. After receiving the information, the host room can determine that the target data is successfully synchronized to the corresponding slave in the host room. engine room.

As an implementation manner, it may be determined whether the synchronization success information fed back by the designated computer room is received within a predetermined period of time. Wherein, the time starting point of the predetermined time period may be the time point at which the operation of synchronizing the target data to the slave computer room corresponding to the host room is performed, and the predetermined time length of the predetermined time period may be predetermined, for example, It is set based on experience. Specifically, it may be to count the length of time for synchronizing data from the computer room, and determine the predetermined time length of the predetermined time period according to the time length.

As an implementation manner, the time length for synchronizing multiple data from the computer room may be acquired, and the predetermined time length of the predetermined time period is determined according to the multiple time lengths. Specifically, the average value of the multiple time lengths is calculated, and the average value is used as the predetermined time length of the predetermined time period.

As another implementation manner, the rate of synchronizing data from the computer room may be obtained by statistics, and the predetermined time length of the predetermined period of time may be obtained according to the data size of the target data and the rate of the synchronized data obtained by the statistics. For example, the ratio of the data size of the acquired target data to the rate of the synchronized data obtained by statistics is used as the predetermined time length of the predetermined time period.

As an implementation manner, referring to FIG. 6, the implementation manner of determining whether the target data is successfully synchronized to the designated computer room may include: S541 to S543.

S541: Acquire first record information corresponding to the host room, where the first record information is used to record the operation performed by the host room to write target data.

As an implementation manner, the first recording information includes the identification and operation time of the data written each time in the host room, where the operation time is used to record the writing time point of the written data, wherein the writing The identification of the data may be an identification assigned by the host room for each written data, where the identification may be an identification determined according to the content of the data, for example, it may be an identification of the data generated based on the data summary of the data As an implementation manner, the data summary of the target data may be used as the identifier of the target data. Among them, the data summary can be used to describe the content of the target data. Then the first record information includes the writing time corresponding to the target identifier of the target data.

S542: Acquire second record information corresponding to the designated computer room, where the second record information is used to record the data synchronization operation of the target data performed by the designated computer room.

In the same way, the second record information includes the identifier and operation time of the data synchronized each time from the computer room, where the operation time may be the time when the data is synchronized from the computer room, and the implementation of the data identifier can refer to the foregoing The embodiments are not repeated here. Each slave computer room corresponds to the second record information. Therefore, each second record information of the slave computer room can be the second record information of the designated computer room.

S543: Determine whether the data is successfully synchronized to the designated computer room according to the first record information and the second record information.

Wherein, the first record information includes the identifier of the data being written in the host room and the writing time point corresponding to the identifier, and the second record information includes the identifier of the data synchronized from the computer room and the synchronization time corresponding to the identifier. Determine whether the target data is successfully synchronized to the designated computer room according to the identifier of the data written in the host room and the write time point corresponding to the identifier, and the identifier of the data synchronized to the designated computer room and the synchronization time corresponding to the identifier.

As an implementation manner, it is determined whether the first operation time point in the first record information is earlier than the second operation time point in the second record information; if it is earlier, it is determined that the data is successfully synchronized to the Designated computer room; if it is not earlier, it is determined that the data has not been successfully synchronized to the designated computer room. Among them, the first operation time point is the writing time point corresponding to the target identifier of the target data written in the host room recorded in the first record information, and the second operation time point is the designated computer room data recorded in the second record information. Synchronize the synchronization time corresponding to the target identifier of the synchronization target data, determine whether the synchronization time is later than the writing time point, if yes, determine that the target data is successfully synchronized to the designated computer room, otherwise, determine that the target data is not successfully synchronized Go to the designated computer room.

Specifically, it may be judged whether the synchronization time corresponding to the identifier of the target data exists in the second record information, and if it exists, the operation of judging whether the synchronization time is later than the writing time point is performed. Wherein, the implementation of determining whether the synchronization time corresponding to the identifier of the target data exists in the second record information may be to determine whether the synchronization time corresponding to the identifier of the target data exists in the second record information within a predetermined period of time, where the predetermined time For the implementation of the segments, reference may be made to the foregoing embodiments.

As another implementation manner, it is considered that the method of recording the write operation of the host room and the data synchronization operation of the slave computer room can be the above-mentioned log file, that is, the write operation of the host and the data synchronization of the slave are recorded in a log stream. Operation, you can set a serial number for both the master room and the slave room, and each write operation of the master and every data synchronization operation of the slave can be recorded through the serial number.

Specifically, the first record information is a first serial number, and the first serial number is used to perform an update operation according to a first update rule when the host room is successfully written into the target data. Specifically, one can be predefined The first serial number, the initial value of the first serial number is the first initial value, then every time data is written in the host room, the value of the first serial number is updated according to the first update rule, where the The first update rule can be to increase or decrease a certain value, that is, each time a write operation is performed, a certain value is increased or decreased on the basis of the first serial number saved last time.

For example, the initial value of the first serial number is the first initial value, and the first update rule is to increase the first value. When the host room writes data for the first time, the value of the first serial number becomes the first The sum of the initial value and the first value. After the data is written again, the value of the first serial number becomes the sum of the first initial value and two first values, that is, on the basis of the first serial number saved last time Increased the first value.

In the embodiment of the present application, assuming that the first sequence number is seq1 and the initial value is 0, then after the data is written for the first time, the value of the first sequence number seq1 becomes 1, and after the data is written for the second time, the first The value of the serial number seq1 becomes 2, and so on, if the data to be written includes multiple data, the multiple data are data1, data2, data3, data4, data5, and data6, respectively, the data identifier and the value of the serial number seq1 The corresponding relationship is: the sequence numbers seq1 corresponding to data1, data2, data3, data4, data5, and data6 are 1, 2, 3, 4, 5, and 6 in order.

Therefore, the writing sequence of each written data and the serial number corresponding to the identifier of each written data can be obtained through the above-mentioned correspondence relationship. The serial number can be used as the serial number of the written data and can be recorded in the entire During the write operation, which target data is written first.

Similarly, the second record information is a second serial number, and the second serial number is used to perform an update operation according to a second update rule when the designated computer room successfully synchronizes data. Wherein, the second update rule may be the same as the above-mentioned first update rule, and the second update rule may also be to increase the second value. Specifically, the initial value of the second serial number is the second initial value, and the second update rule is to increase the second value. When synchronizing data from the computer room for the first time, the value of the second serial number becomes the second The sum of the initial value and the second value. After synchronizing the data again, the value of the second serial number becomes the sum of the second initial value and two second values, that is, it is added to the second serial number saved last time. The second value. Wherein, the first value and the second value are the same, for example, both are 1, and the first initial value and the second initial value may also be the same, for example, both may be 0.

Assuming that the second sequence number is seq2, the initial value is 0. After synchronizing data from the computer room for the first time, the value of the second sequence number seq2 becomes 1, and after the second time synchronizing data, the value of the second sequence number seq2 The value becomes 2, and so on. If the synchronized data includes multiple data, the multiple data are data1, data2, data3, data4, data5, and data6 respectively. The corresponding relationship between the data identifier and the value of the serial number seq2 is: data1 The sequence numbers seq2 corresponding to, data2, data3, data4, data5, and data6 are 1, 2, 3, 4, 5, and 6, respectively.

It should be noted that the seq2 of the slave is only changed when the data is successfully synchronized by the slave, and the size relationship of the seq1 corresponding to the target data written by the host is determined by determining the size of the current seq2 , Can determine whether the slave successfully synchronizes the target data.

Therefore, it can be determined whether the data is successfully synchronized to the designated computer room according to the size relationship between the first serial number and the second serial number. Specifically, it is determined whether the data is successfully synchronized according to the first record information and the second record information. The implementation manner of synchronizing to the designated computer room may be: obtaining the size relationship between the first serial number and the second serial number; if the size relationship satisfies a specified condition, it is determined that the data is successfully synchronized to the A designated computer room, wherein the size relationship satisfies a specified condition indicates that the designated computer room successfully synchronizes the target data after the host room is written with the target data; if the size relationship does not meet the specified condition, it is determined not to The data is successfully synchronized to the designated computer room.

Wherein, the size relationship can be whether the first serial number is less than or equal to the second serial number, or another size relationship that can characterize whether the first serial number is less than or equal to the second serial number, for example, it can also be to obtain the first serial number. The ratio between the serial number and the second serial number. If the ratio is greater than 1, it indicates that the first serial number is greater than the second serial number; if the ratio is not greater than 1, it indicates that the first serial number is less than or equal to the second serial number. No.

As an implementation manner, the magnitude relationship can be determined according to the first update rule and the second update rule. If the first update rule and the second update rule are the same, and both are increased by the same value on the basis of the initial value, The size relationship may be whether the first serial number is less than or equal to the second serial number. If the first update rule and the second update rule are the same, and the same value is subtracted from the initial value, the magnitude relationship may be whether the first serial number is greater than or equal to the second serial number. If the first update rule and the second update rule are not the same, and the change value corresponding to the first update rule is k times the change value corresponding to the second update rule, the magnitude relationship may be the second sequence number and the first sequence Whether the ratio of the number is a positive integer multiple of k.

After obtaining the size relationship between the first serial number and the second serial number, it is determined whether the size relationship satisfies a specified condition.

Wherein, that the size relationship satisfies a specified condition indicates that the specified computer room successfully synchronizes the target data after the host room is written into the target data.

Then, when the first update rule and the second update rule are the same or different, the above-mentioned magnitude relationship is also different, and therefore the corresponding designated conditions are also different.

For example, if the first update rule and the second update rule are the same, and the same value is added to the initial value, the size relationship may be whether the first serial number is less than or equal to the second serial number, then if If the first serial number is less than or equal to the second serial number, it is determined that the size relationship meets the specified condition; otherwise, it is determined that the size relationship does not meet the specified condition.

If the first update rule and the second update rule are the same, and the same value is subtracted from the initial value, the magnitude relationship can be whether the first serial number is greater than or equal to the second serial number, then if the first serial number is greater than or equal to the second serial number, If a serial number is greater than or equal to the second serial number, it is determined that the size relationship meets the specified condition; otherwise, it is determined that the size relationship does not meet the specified condition.

If the first update rule and the second update rule are not the same, and the change value corresponding to the first update rule is k times the change value corresponding to the second update rule, the magnitude relationship may be the second sequence number and the first sequence Whether the ratio of the number is a positive integer multiple of k, then if the ratio of the second serial number to the first serial number is a positive integer multiple of k, it is determined that the size relationship meets the specified condition, otherwise, it is determined that the size relationship does not meet Specify the conditions.

In the embodiment of the present application, the first update rule and the second update rule may be the same, and the same value is added to the initial value. The size relationship meets the specified condition means that if the first serial number is less than or equal to The second serial number.

As an implementation manner, the first serial number is changed according to the first update rule, and the second serial number is changed according to the second update rule. When the host room completes the write operation of the target data, the first serial number The first value of the serial number after the first update rule is updated, and when the slave completes the synchronization operation of the target data, the second serial number is the second value after the second update rule is updated, the first value and The size relationship between the second value should satisfy the relationship between the first update rule and the second update rule. For example, if the first update rule and the second update rule are the same, the size relationship between the first value and the second value should be same. If the first update rule is to increase or decrease the value a, and the second update rule is to increase the value b, then the ratio of the value a to the value b is c, then the magnitude relationship between the first value and the second value should also satisfy the value a The ratio to the value b is n times c, where n is a positive integer.

Assuming that the first update rule and the second update rule are the same, they are both increased by 1, and the initial value of the first serial number is the same as the initial value of the second serial number, and each time a write operation is performed, the host room The first serial number of the slave and the second serial number of the slave will be initialized, that is, it will be restored to the initial value. If there is only one data to be written, that is, if the target data is one data, the host room writes the After the target data, the first serial number should be 1, and after the slave successfully synchronizes the target data, the second serial number should be 1, that is, the first serial number and the second serial number are the same.

In addition, considering that when there are multiple data written in the host room, the data synchronization operation of the host room and the host room's detection of whether the target data is successfully synchronized to the designated computer room are synchronized in execution, there may be a host room When determining whether data data1 is successfully synchronized by the slave, the slave has successfully synchronized to data3, and data3 is the data written after data1. At this time, the second sequence number of the slave is greater than the first sequence Therefore, if the second serial number is greater than the first serial number, it can indicate that the data corresponding to the first serial number has been successfully synchronized to the slave.

The writing sequence of the data written in the master room, the synchronization sequence of synchronizing the written data to the slave machine can be the same, for example, the writing sequence of the data written in the master room is data1, data2, data3 , Data4, data5 and data6, the synchronization sequence of synchronizing the written data to the slave is data1, data2, data3, data4, data5 and data6.

Therefore, the first serial number corresponding to the target data written in the master room is acquired, and the current second serial number of the slave is acquired. If the first serial number is less than or equal to the second serial number, it means that the slave is in all locations. After the host room is written into the target data, the target data is successfully synchronized.

For example, the data written in the host room includes data1, data2, data3, data4, data5, and data6. At this time, data1, data2, data3, data4, data5, and data6 are successfully written into the host room, where data2 is the target data, then Referring to the above table 1, the first sequence number corresponding to the target data is seq1 equal to 2. The master room synchronizes data1, data2, data3, data4, data5, and data6 to the slave in sequence. At this time, the seq2 of the slave is 1, then It indicates that the data successfully synchronized by the slave is data1, and the first serial number is greater than the second serial number, and it can be determined that the target data has not been successfully synchronized to the designated computer room.

Then, if the seq2 of the slave machine is 2, it means that the data successfully synchronized by the slave machine is data2, that is, the data successfully synchronized by the slave machine is the target data, and the first serial number is equal to the second serial number, then it can be determined that the The target data is not successfully synchronized to the designated computer room.

Then if the seq2 of the slave is 3 at this time, it means that the data successfully synchronized by the slave is data3, that is, the data successfully synchronized by the slave is data3 after the target data, and the first serial number is less than the second serial number, then It can be determined that the target data is not successfully synchronized to the designated computer room.

Therefore, when the first serial number is less than or equal to the second serial number, it can be determined that the target data is successfully synchronized to the designated computer room.

In addition, when the master room corresponds to multiple slave rooms, and each slave room corresponds to a second serial number, the above detection method can be used to determine whether each slave room has successfully synchronized the target data, so as to determine the target data Whether to successfully synchronize to the slave computer room corresponding to the host room.

In some embodiments, the operation of detecting whether the target data is successfully synchronized to the slave computer room corresponding to the host room may be named a scanning operation, and if the first serial number is greater than the second serial number, it is determined that the target When the data is not successfully synchronized to the slave computer room corresponding to the host room, you can wait for a certain period of time before continuing to perform the scanning operation, that is, pause the scanning operation during this time period, so that the slave computer room can complete the scan within this time period. Synchronization operation of target data.

Specifically, if the size relationship does not meet the specified condition, after determining that the data is not successfully synchronized to the specified computer room, the method further includes: obtaining the first serial number and the second serial number again after a specified length of time. The size relationship between the serial numbers and the execution of subsequent operations, wherein the operation of synchronizing the target data to the specified computer room is continued to be executed within the specified length of time. The specified time length may be a preset time length. For example, the specified time length may be the same as the predetermined time length corresponding to the aforementioned predetermined time period. Specifically, the specified time length may be 10 ms.

S550: Use a slave room other than the designated machine room among the slave machine rooms as a new designated machine room.

After it is determined by the above method that the host room has synchronized all the written data to the designated computer room, a slave computer room other than the designated computer room is re-selected as the new designated computer room, and the execution of S530 is returned. , Until all slave rooms are synchronized.

Therefore, in the embodiment of the present application, mutually independent Redis clusters can be deployed in multiple computer rooms, and the read and write operations of all computer rooms can be synchronized to other computer rooms, that is, the clusters of each computer room are mirror images of each other. When part of the computer room fails (including software, hardware, and network failures), the business layer can dispatch read and write requests to other normal computer rooms to continue running, thereby achieving cross-computer room disaster recovery and ensuring high service availability and data security.

In addition, as shown in FIG. 7, the above-mentioned client may include a scanning process and a subscription process. The scanning process is used to perform the operation of judging whether the data is successfully synchronized to the designated computer room, and the subscription process is used to monitor the write operation of the master room and the slave computer room. When the write operation is detected, the The write operation obtains the target data.

Data synchronization through external programs has no perception of the business layer and is simple to use, that is, data synchronization is achieved through the above-mentioned client. Therefore, there will be no cross-machine room reading and writing, and business performance will not be affected. Supports simultaneous read and write in multiple locations, supports data synchronization between any number of clusters, and services can be deployed in multiple locations according to the situation, with strong scalability. Through the master-slave machine room mechanism, the final consistency of the data when writing in multiple places is guaranteed at the same time.

Please refer to FIG. 8, which shows a structural block diagram of a data synchronization device 800 provided by an embodiment of the present application. The device may include: an acquiring unit 801, a writing unit 802, and a synchronization unit 803.

The obtaining unit 801 is configured to obtain target data requested to be written into the data storage system.

The writing unit 802 is configured to write the target data into the host room.

The synchronization unit 803 is configured to synchronize the target data to the slave computer room after successfully writing the target data into the host room.

Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working process of the device and module described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

Please refer to FIG. 9, which shows a structural block diagram of a data synchronization device 900 provided by an embodiment of the present application. The device may include: an acquiring unit 910, a writing unit 920, and a synchronization unit 930.

The obtaining unit 910 is configured to obtain the target data requested to be written into the data storage system.

The acquiring unit 910 also includes a monitoring subunit 911 and a data determining subunit 912.

The monitoring subunit 911 is used to monitor the write operation of the host room and the slave computer room.

The data determining subunit 912 is further configured to obtain the target data according to the write operation when the write operation is detected.

Specifically, the data determining subunit 912 is further configured to, if there are multiple write operations, obtain a timestamp corresponding to each write operation, where each write operation corresponds to designated data and is used to The content to be modified is modified to designated data; the target write operation is determined from the plurality of write operations according to the timestamp; the designated data corresponding to the target write operation is used as the target data.

Further, the data determining subunit 912 is further configured to determine the latest timestamp among all the timestamps as the target timestamp if the timestamps are not all the same; use the write operation corresponding to the target timestamp as the candidate Write operation; the target write operation is determined from the to-be-selected write operation.

Further, the data determining subunit 912 is further configured to, if the number of the write operations to be selected is 1, set the write operations to be selected as the target write operation.

Further, the data determining subunit 912 is further configured to determine the operation object corresponding to each of the writing operation to be selected if the number of the writing operation to be selected is at least two, and the operation object includes the host room or the slave Computer room; take the write operation to be selected in the host room where the operation object is the target write operation.

Further, the data determining subunit 912 is further configured to determine the operation object corresponding to each of the write operations to be selected if the time stamps are all the same, and the operation object includes the host room or the slave computer room;

The writing unit 920 is configured to write the target data into the host room.

The synchronization unit 930 is configured to synchronize the target data to the slave computer room after successfully writing the target data into the host room.

Wherein, the master room and the slave room each have a message queue, the master room subscribes to the message queues of all the slave computer rooms, each of the slave computer rooms subscribes to the message queue of the master room, and the After the target data is written into the host room, synchronizing the target data to the slave computer room includes: after successfully writing the target data into the host room, passing through a message queue corresponding to the host room To synchronize the target data to the slave computer room.

Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working process of the above described device and module can be referred to the corresponding process in the foregoing method embodiment, which will not be repeated here.

Please refer to FIG. 10, which shows a structural block diagram of a data synchronization device 1000 provided by an embodiment of the present application. The device may include: an acquiring unit 1010, a writing unit 1020, and a synchronization unit 1030.

The obtaining unit 1010 is configured to obtain the target data requested to be written into the data storage system.

The writing unit 1020 is configured to write the target data into the host room.

The synchronization unit 1030 is configured to synchronize the target data to the slave computer room after successfully writing the target data into the host room.

The synchronization unit 1030 includes a selection subunit 1031, a judgment subunit 1032, and a synchronization subunit 1033.

The selection subunit 1031 is configured to synchronize the target data to a designated computer room after successfully writing the target data into the host room, and the designated computer room is one of the multiple slave computer rooms.

The judging subunit 1032 is used to judge whether the data is successfully synchronized to the designated computer room.

In some embodiments, the judging subunit 1032 is also used to detect whether the synchronization success information fed back by the designated computer room is received; if it is received, it is determined that the data is successfully synchronized to the designated computer room; if it is not received, then It is determined that the data is not successfully synchronized to the designated computer room.

In other embodiments, the judging subunit 1032 is further configured to obtain the first record information corresponding to the host room, and the first record information is used to record the operation performed by the host room to write target data; The second record information corresponding to the designated computer room, where the second record information is used to record the data synchronization operation of the target data performed by the designated computer room; determine whether or not according to the first record information and the second record information The data is successfully synchronized to the designated computer room.

Further, the judging subunit 1032 is also used to judge whether the first operation time point in the first record information is earlier than the second operation time point in the second record information; if it is earlier, it is judged that the data is successfully transferred Synchronize to the designated computer room; if it is not earlier, it is determined that the data is not successfully synchronized to the designated computer room.

Wherein, the first record information is a first serial number, and the first serial number is used to perform an update operation according to a first update rule when the host room is successfully written into target data; the second record information It is a second serial number, and the second serial number is used to perform an update operation according to a second update rule when the designated computer room successfully synchronizes data.

Further, the judging subunit 1032 is also used to obtain the size relationship between the first serial number and the second serial number; if the size relationship meets a specified condition, it is determined that the data is successfully synchronized to the specified computer room , Wherein the size relationship satisfies the specified condition indicates that the specified computer room successfully synchronizes the target data after the host room is written into the target data; if the size relationship does not meet the specified condition, it is determined that the transfer is unsuccessful The data is synchronized to the designated computer room.

Wherein, the first update rule is to increase or decrease the first serial number by a first value when the target data is successfully written in the host room, and the second update rule is to successfully synchronize data in the designated computer room, The second serial number is increased or decreased by a second value, and the first value is the same as the second value.

The fact that the size relationship satisfies a specified condition means that the first serial number is less than or equal to the second serial number.

Further, the judging subunit 1032 is further configured to obtain the size relationship between the first serial number and the second serial number again and perform subsequent operations after a specified length of time, wherein, within the specified length of time Continue to perform the operation of synchronizing the target data to the designated computer room.

The synchronization subunit 1033 is configured to, if the data is successfully synchronized to the designated computer room, set the secondary computer room, the secondary computer room outside the designated computer room as the new designated computer room, and return to execute the synchronization of the target data to the designated computer room. Computer room, until all the target data synchronization operations of the slave computer rooms are completed.

In the several embodiments provided in this application, the coupling between the modules may be electrical, mechanical or other forms of coupling.

In addition, the functional modules in the various embodiments of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or software functional modules.

Please refer to FIG. 11, which shows a structural block diagram of an electronic device provided by an embodiment of the present application. The electronic device 100 may be an electronic device capable of running application programs, such as a smart phone, a tablet computer, or an e-book. The electronic device 100 in this application may include one or more of the following components: a processor 110, a memory 120, and one or more application programs, where one or more application programs may be stored in the memory 120 and configured to be configured by One or more processors 110 execute, and one or more programs are configured to execute the methods described in the foregoing method embodiments.

The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes by running or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and calling data stored in the memory 120. Various functions and processing data of the electronic device 100. Optionally, the processor 110 may adopt at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). A kind of hardware form to realize. The processor 110 may be integrated with one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a modem, and the like. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is used for rendering and drawing of display content; the modem is used for processing wireless communication. It is understandable that the above-mentioned modem may not be integrated into the processor 110, but may be implemented by a communication chip alone.

The memory 120 may include random access memory (RAM) or read-only memory (Read-Only Memory). The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, where the program storage area may store instructions for implementing the operating system and instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.) , Instructions used to implement the following various method embodiments, etc. The data storage area can also store data (such as phone book, audio and video data, chat record data) created by the terminal 100 during use.

Please refer to FIG. 12, which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. The computer-readable medium 1200 stores program code, and the program code can be invoked by a processor to execute the method described in the foregoing method embodiment.

The computer-readable storage medium 1200 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 1200 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 1200 has storage space for the program code 1210 for executing any method steps in the above-mentioned methods. These program codes can be read from or written into one or more computer program products. The program code 1210 may be compressed in an appropriate form, for example.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions recorded in the foregoing embodiments are modified, or some of the technical features thereof are equivalently replaced; these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

A data synchronization method, characterized in that it is applied to a host room of a data storage system, the data storage system further includes a slave computer room corresponding to the host room, and the method includes:

Acquiring the target data requested to be written to the data storage system;

Writing the target data into the host room;

After successfully writing the target data into the host room, synchronize the target data to the slave computer room.
The method according to claim 1, wherein the obtaining the target data written in the data storage system by the request comprises:

Monitoring the write operations of the host room and the slave computer room;

When the write operation is detected, the target data is obtained according to the write operation.
The method according to claim 2, wherein when the writing operation is detected, obtaining target data according to the writing operation comprises:

If there are multiple write operations, obtain a time stamp corresponding to each write operation, where each write operation corresponds to designated data and is used to modify the content to be modified into designated data;

Determined as a target write operation from among the multiple write operations according to the timestamp;

The designated data corresponding to the target write operation is used as the target data.
The method according to claim 3, wherein the determining as a target write operation from a plurality of the write operations according to the timestamp comprises:

If the time stamps are not all the same, determine the latest time stamp among all time stamps as the target time stamp;

Take the write operation corresponding to the target timestamp as a write operation to be selected;

The target write operation is determined from the to-be-selected write operation.
The method according to claim 4, wherein the determining a target write operation from the to-be-selected write operation comprises:

If the number of the write operation to be selected is 1, the write operation to be selected is taken as the target write operation.
The method according to claim 4, wherein the determining a target write operation from the to-be-selected write operation comprises:

If the number of the write operations to be selected is at least two, determine an operation object corresponding to each write operation to be selected, and the operation object includes a host room or a slave computer room;

The write operation to be selected in the host room as the operation object is taken as the target write operation.
The method according to claim 3, further comprising:

If the timestamps are all the same, determine the operation object corresponding to each of the to-be-selected write operations, and the operation object includes the host room or the slave computer room;

The write operation to be selected in the host room as the operation object is taken as the target write operation.
The method according to claim 1, wherein there are multiple slave computer rooms, and after successfully writing the target data into the host room, synchronizing the target data to the slave computer room, include:

After successfully writing the target data into the host room, synchronize the target data to a designated computer room, where the designated computer room is one of the multiple slave computer rooms;

Determine whether the data is successfully synchronized to the designated computer room;

If the data is successfully synchronized to the designated machine room, the slave machine room and the slave machine room other than the designated machine room will be used as the new designated machine room, and the execution will be returned to synchronize the target data to the designated machine room until all the data is completed. The target data synchronization operation from the computer room is described.
The method according to claim 8, wherein the determining whether the data is successfully synchronized to the designated computer room comprises:

Detecting whether the synchronization success information fed back by the designated computer room is received;

If it is received, it is determined that the data is successfully synchronized to the designated computer room;

If it is not received, it is determined that the data has not been successfully synchronized to the designated computer room.
The method according to claim 8, wherein the determining whether the data is successfully synchronized to the designated computer room comprises:

Acquiring first record information corresponding to the host room, where the first record information is used to record the operation performed by the host room to write target data;

Acquiring second record information corresponding to the designated computer room, where the second record information is used to record the data synchronization operation of the target data performed by the designated computer room;

Determine whether the data is successfully synchronized to the designated computer room according to the first record information and the second record information.
The method according to claim 10, wherein the recording information is used to record the operation time point of the operation; the judging whether the data is successfully synchronized to the The designated computer room includes:

Determining whether the first operation time point in the first record information is earlier than the second operation time point in the second record information;

If it is earlier, it is determined that the data is successfully synchronized to the designated computer room;

If it is not earlier, it is determined that the data has not been successfully synchronized to the designated computer room.
The method according to claim 10, wherein the first record information is a first serial number, and the first serial number is used when the host room is successfully written into the target data, according to the first update The update operation is performed according to the rule; the second record information is a second serial number, and the second serial number is used to perform the update operation according to the second update rule when the designated computer room successfully synchronizes data.
The method according to claim 10, wherein the judging whether the data is successfully synchronized to the designated computer room according to the first record information and the second record information comprises:

Acquiring the size relationship between the first serial number and the second serial number;

If the size relationship meets the specified condition, it is determined that the data is successfully synchronized to the specified computer room, where the size relationship meets the specified condition indicates that the specified computer room successfully synchronizes after the host room is written with the target data The target data;

If the size relationship does not meet the specified condition, it is determined that the data is not successfully synchronized to the specified computer room.
The method according to claim 13, wherein the first update rule is to increase or decrease the first serial number by a first value when the host room is successfully written into the target data, and the second update rule In order to increase or decrease the second serial number by a second value when the designated computer room successfully synchronizes data, the first value and the second value are the same.
The method according to claim 14, wherein the size relationship satisfying a specified condition means that the first serial number is less than or equal to the second serial number.
The method according to claim 13, characterized in that, after determining that the data is not successfully synchronized to the specified computer room if the size relationship does not satisfy a specified condition, the method further comprises:

After the specified length of time, the size relationship between the first serial number and the second serial number is obtained again and subsequent operations are performed, wherein the synchronization of the target data to the The operation of the designated computer room.
The method according to any one of claims 1-16, wherein the master room and the slave room each have a message queue, the master room subscribes to the message queues of all the slave rooms, and each slave room Both subscribing to the message queue of the host room, and synchronizing the target data to the slave computer room after successfully writing the target data into the host room includes:

After successfully writing the target data into the host room, synchronize the target data to the slave computer room through a message queue corresponding to the host room.
A data synchronization device, characterized in that it is applied to a host room of a data storage system, the data storage system further includes a slave room corresponding to the host room, and the device includes:

An obtaining unit, configured to obtain the target data requested to be written into the data storage system;

A writing unit for writing the target data into the host room;

The synchronization unit is configured to synchronize the target data to the slave computer room after successfully writing the target data into the host room.
A data storage system is characterized in that it comprises:

The host room and the slave room corresponding to the host room;

The host is used to execute the method described in any one of claims 1-17. .
A computer-readable medium, wherein the computer-readable medium stores program code executable by a processor, and when the program code is executed by the processor, the processor executes any of claims 1-17. One of the methods described.