CN116821232A

CN116821232A - Data synchronization method and related device

Info

Publication number: CN116821232A
Application number: CN202210283849.9A
Authority: CN
Inventors: 吴双桥; 严海涛
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2022-03-22
Filing date: 2022-03-22
Publication date: 2023-09-29

Abstract

The embodiment of the application discloses a data synchronization method and a related device, wherein the method is applied to a target data center under a multi-activity data center architecture, the target data center is any one of a plurality of data centers under the architecture, and the plurality of data centers are provided with corresponding databases. The method comprises the following steps: receiving operation data to be synchronized sent by other data centers, wherein the operation data to be synchronized is configured with version information, and the version information is determined according to the generation time of the operation data to be synchronized; determining whether the operation data to be synchronized is valid or not according to the version information of the operation data to be synchronized and the version information of the reference operation data; if the operation data to be synchronized is determined to be effective, changing processing based on the target data is performed in the corresponding database according to the operation data to be synchronized; and if the operation data to be synchronized is determined to be invalid, the operation data to be synchronized is ignored. The method can solve the problem of data error coverage caused by data collision, and ensure the complete synchronization of the data between the data centers.

Description

Data synchronization method and related device

Technical Field

The present application relates to the field of database technologies, and in particular, to a data synchronization method and a related device.

Background

The multi-active data center architecture is a network architecture including a plurality of data centers, and a plurality of data centers deployed in different physical areas in the multi-active data center architecture can simultaneously provide services to the outside.

In a multi-active data center architecture, data synchronization is performed between individual data centers. FIG. 1 is a schematic diagram of an exemplary multi-active data center architecture; as shown in fig. 1, the architecture includes databases of A, B, C data centers located in different physical areas, each data center writes data into its database, and for the written data, the written data is synchronized to other data centers through a data synchronization link, so as to ensure that each data center has a full data set. Under the data synchronization mechanism, when a certain data center in the architecture has a disaster accident, the traffic of the data center can be correspondingly transferred to other data centers, so that the rapid transfer of the service is realized, and the normal operation of the service is ensured.

The most central problem in the data synchronization mechanism is that data collision cannot occur, and the data collision refers to that data to be synchronized sent by a plurality of data centers is received at the same time. If there is a data collision, there is a high possibility that a problem occurs in the data synchronization sequence, and the data generated later in time sequence is covered by the data generated earlier, thereby obtaining unexpected results. In many cases, even small data collisions may lead to serious consequences, such as, for example, a disruption of the multi-active data center architecture, which in turn affects the normal operation of the business under the multi-active data center architecture.

Disclosure of Invention

The embodiment of the application provides a data synchronization method and a related device, which can effectively solve the problem of data error coverage caused by data collision and ensure the complete synchronization of data among data centers in a multi-activity data center architecture.

In view of this, a first aspect of the present application provides a data synchronization method applied to a target data center under a multi-active data center architecture, the multi-active data center architecture including a plurality of data centers, the target data center being any one of the plurality of data centers, the plurality of data centers having respective corresponding databases thereof; the method comprises the following steps:

receiving operation data to be synchronized sent by other data centers under the multi-activity data center architecture; the operation data to be synchronized is used for indicating to change based on target data, and is configured with version information, and the version information is determined according to the generation time of the operation data to be synchronized;

determining whether the operation data to be synchronized is valid or not according to the version information of the operation data to be synchronized and the version information of the reference operation data; the reference operation data includes at least one of operation data that has been executed by the target data center and operation data that has been received by the target data center but has not been executed;

If the operation data to be synchronized is determined to be effective, changing processing based on the target data is performed in a database corresponding to the target data center according to the operation data to be synchronized; and if the operation data to be synchronized is determined to be invalid, the operation data to be synchronized is ignored.

The second aspect of the present application provides a data synchronization device, disposed in a target data center under a multi-activity data center architecture, where the multi-activity data center architecture includes a plurality of data centers, the target data center being any one of the plurality of data centers, the plurality of data centers having databases corresponding to the target data centers; the device comprises:

the data receiving module is used for receiving the operation data to be synchronized sent by other data centers under the multi-activity data center architecture; the operation data to be synchronized is used for indicating to change based on target data, and is configured with version information, and the version information is determined according to the generation time of the operation data to be synchronized;

the data detection module is used for determining whether the operation data to be synchronized is effective or not according to the version information of the operation data to be synchronized and the version information of the reference operation data; the reference operation data includes at least one of operation data that has been executed by the target data center and operation data that has been received by the target data center but has not been executed;

The data changing module is used for carrying out changing processing based on the target data in a database corresponding to the target data center according to the operation data to be synchronized if the operation data to be synchronized is determined to be effective; and if the operation data to be synchronized is determined to be invalid, the operation data to be synchronized is ignored.

A third aspect of the application provides a computer apparatus comprising a processor and a memory:

the memory is used for storing a computer program;

the processor is configured to perform the steps of the data synchronization method as described in the first aspect above according to the computer program.

A fourth aspect of the present application provides a computer readable storage medium storing a computer program for performing the steps of the data synchronization method of the first aspect described above.

A fifth aspect of the application provides a computer program product or computer program comprising computer instructions stored on a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the data synchronization method described in the first aspect.

From the above technical solutions, the embodiment of the present application has the following advantages:

the embodiment of the application provides a data synchronization method, which is applied to a target data center under a multi-activity data center architecture, wherein the multi-activity data center architecture comprises a plurality of data centers, the target data can be any one of the plurality of data centers, and each data center under the multi-activity data center architecture is provided with a corresponding database. The target data center can receive to-be-synchronized operation data sent by other data centers under the multi-activity data center architecture, the to-be-synchronized operation data is used for indicating to change based on the target data, and is configured with version information, and the version information is determined according to the generation time of the to-be-synchronized operation data; then, the target data center may determine whether the operation data to be synchronized is valid according to version information of the operation data to be synchronized and version information of reference operation data, where the reference operation version data may include at least one of operation data that the target data center has performed and operation data that the target data center has received but has not performed; if the data to be synchronized is determined to be effective, the change processing based on the target data can be performed in the database corresponding to the target data center according to the data to be synchronized, and if the data to be synchronized is determined to be ineffective, the data to be synchronized can be ignored. Accordingly, corresponding version information is configured for each operation data according to the generation time of each operation data, and data synchronization is performed based on the version information corresponding to each operation data, so that each data center under the multi-activity data center architecture can realize data synchronization operation based on the operation data of the latest version.

Drawings

FIG. 1 is a schematic diagram of an exemplary multi-active data center architecture;

FIG. 2 is a schematic diagram of an exemplary scenario in which operation data is wrongly written into other data centers;

FIG. 3 is a schematic diagram illustrating the working principle of a multi-active data center architecture according to an embodiment of the present application;

fig. 4 is a schematic flow chart of a data synchronization method according to an embodiment of the present application;

FIG. 5 is a schematic diagram of data synchronization in one embodiment of the present application;

FIG. 6 is a schematic diagram illustrating a synchronization principle of a multi-active data center architecture according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a data synchronization device according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

In order to make the present application better understood by those skilled in the art, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Cloud technology (Cloud technology) is a generic term of network technology, information technology, integration technology, management platform technology, application technology and the like based on Cloud computing business model application, and can form a resource pool, so that the Cloud computing business model application system is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.

The Database (Database), which can be considered as an electronic filing cabinet, is a place for storing electronic files, and users can perform operations such as adding, inquiring, updating, deleting and the like on the data in the files. A "database" is a collection of data stored together in a manner that can be shared with multiple users, with as little redundancy as possible, independent of the application.

The database management system (Database Management System, DBMS) is a computer software system designed for managing databases, and generally has basic functions of storage, interception, security, backup, and the like. The database management system may classify according to the database model it supports, e.g., relational, XML (Extensible Markup Language ); or by the type of computer supported, e.g., server cluster, mobile phone; or by the query language used, such as SQL (structured query language (Structured Query Language), XQuery, or by the energy impact emphasis, such as maximum-scale, maximum-speed, or other classification means, regardless of which classification means is used, some DBMSs can cross-category, for example, while supporting multiple query languages.

The data synchronization method provided by the embodiment of the application relates to a database technology in cloud technology.

For a multi-activity data center architecture, when data synchronization is performed between data centers based on a data synchronization mechanism, the most central problem is that data collision cannot occur, and serious consequences are possibly caused once the data collision occurs, so that the overall normal operation of the multi-activity data center architecture is affected.

In the related art, there are two main ways to avoid data collision: one is to set up the data record table corresponding to it for different data centers, write the data related to this data center (such as the operation data changing for the data stored in this data center, etc.) into the data record table corresponding to this data center, each data center in the multi-activity data center architecture carries on the data synchronization among each data center based on its corresponding data record table. Another is to assign different key value intervals to different data centers in a single data record table, for example, assign key value intervals 1, 4, 7, … … to data center a, assign key value intervals 2, 5, 8, … … to data center B, and assign key value intervals 3, 6, 9, … … to data center C; and writing data related to the data center into a corresponding key value interval for each data center, and performing data synchronization among the data centers based on the key value intervals corresponding to the data centers in the data record table by the data centers in the multi-activity data center architecture.

However, the above manner of avoiding data collision brings great challenges to the use and operation of the whole multi-active data center architecture, and the business accessing different data centers needs to sense which data center is used, which table or which key value interval needs to be written with data, so as to avoid data writing errors, which brings a lot of inconveniences to the design and deployment of the business and increases the complexity of using the database and operation and maintenance business by the business. Furthermore, in extreme cases, if the service deployment is wrong, the whole multi-active data center architecture may be confused, and human access may be required to recover the whole system; since the link configuration itself is complex in the multi-active data center architecture, the restoration to normal data is also very complex, resulting in a longer service restoration time.

FIG. 2 is a schematic diagram of an exemplary scenario for misreading operation data to other data centers. Specifically, it is assumed that the service side triggers two update operations for the data stored in the data center a, where the first time is to modify (1, a) to (1, b) and the second time is to modify (1, b) to (1, c); in this scenario, the first update operation (i.e., (1, a) → (1, B)) is sent to the data center B by mistake, resulting in the data center a receiving only the operation data of the second update operation, and directly performing the second update to modify the data a stored therein to the data c. After receiving the operation data of modifying (1, a) into (1, B), the data center B synchronizes the operation data to the data center A; in the case where the data center a has modified its stored data a to data c based on the operation data of the second update operation, after receiving the operation data of the data center B for modifying (1, a) to (1, B), the data center a may erroneously modify the modified data c to data B again, i.e., make the data B cover the data c, which is inconsistent with the data result expected by the service side, thereby causing confusion of data under the architecture of the multi-activity data center.

In order to solve the problems of the related art, and effectively solve the problem of data collision under a multi-activity data center architecture, the embodiment of the application provides a data synchronization method. The method is applied to a target data center under a multi-activity data center architecture, wherein the multi-activity data center architecture comprises a plurality of data centers, the target data can be any one of the plurality of data centers, and each data center under the multi-activity data center architecture has a corresponding database.

In the data synchronization method, the target data center may receive to-be-synchronized operation data sent by other data centers under the multi-activity data center architecture, the to-be-synchronized operation data is used for indicating to perform modification processing based on the target data, and the to-be-synchronized operation data is configured with version information, and the version information is determined according to generation time of the to-be-synchronized operation data. Then, the target data center may determine whether the operation data to be synchronized is valid according to version information of the operation data to be synchronized and version information of reference operation data, where the reference operation version data may include at least one of operation data that the target data center has performed and operation data that the target data center has received but has not performed. If the operation data to be synchronized is determined to be effective, changing processing based on target data can be performed in a database corresponding to the target data center according to the operation data to be synchronized; if the operation data to be synchronized is determined to be invalid, the operation data to be synchronized can be ignored.

According to the data synchronization method, corresponding version information is configured for each item of operation data according to the generation time of each item of operation data, and data synchronization is performed based on the version information corresponding to each item of operation data, so that each data center under the multi-activity data center architecture can realize data synchronization operation based on the operation data of the latest version; in this case, even if a data collision occurs, the data centers can be synchronized based on the latest version of operation data, so that the data consistency among the data centers can be ensured. That is, even if a data conflict occurs under the multi-active data center architecture, each data center under the architecture can automatically resolve the data conflict according to a pre-configured policy, thereby improving the robustness of the whole multi-active data center architecture. In addition, compared with the scheme of setting corresponding data record tables for different data centers or distributing different key value intervals in the data record tables for different data centers in the related art, the scheme provided by the embodiment of the application does not increase the complexity of using the database and the operation and maintenance service on the service side.

In the embodiment of the present application, each data center under the multi-activity data center architecture may be a server, where the server may be an independent physical server, or may be a server cluster or a distributed system formed by multiple physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligent platforms. Each data center under the multi-activity data center architecture can be in butt joint with a terminal, and a corresponding server is provided for the terminal, wherein the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart sound box, a smart watch and the like, but is not limited to the terminal. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the present application is not limited herein.

In order to facilitate understanding of the data synchronization method provided by the embodiment of the present application, an application scenario to which the data synchronization method provided by the embodiment of the present application is applicable is first described in the following by way of example.

Referring to fig. 3, fig. 3 is a schematic diagram of a working principle of a multi-activity data center architecture 300 provided by an embodiment of the present application, that is, an application scenario schematic diagram of a data synchronization method provided by an embodiment of the present application. As shown in fig. 3, the multi-activity data center architecture 300 includes a plurality of data centers, namely, a data center 310, a data center 320, data centers 330, … …, and a data center 3n0, each having its corresponding database, namely, the data center 310 corresponds to the database 311, the data center 320 corresponds to the database 321, the data center 330 corresponds to the databases 331, … …, and the data center 3n0 corresponds to the database 3n1; it should be understood that the database corresponding to the data center may be a storage device independent of the data center, or may be a storage component integrated within the data center.

In this embodiment, taking the data center 310 as a target data center as an example, a scenario exemplary description is performed on the data synchronization method provided in the embodiment of the present application.

As shown in fig. 3, the data center 310 may receive operation data to be synchronized, which is transmitted by the data center 320 through a data synchronization link; the operation data to be synchronized is used for indicating that the change process is performed based on the target data, for example, the operation data to be synchronized may be used for indicating that the target data is inserted into the database 311, for example, the operation data to be synchronized may be used for indicating that the target data originally stored in the database 311 is deleted, for example, the operation data to be synchronized may be used for indicating that the target data originally stored in the database 311 is updated to the target update data, and so on. In addition, the operation data to be synchronized is configured with version information, which is determined according to the generation time of the operation data to be synchronized; it should be understood that the later the generation time of the operation data to be synchronized, the higher the version corresponding to the version information of the operation data to be synchronized.

Further, the data center 310 may determine whether the operation data to be synchronized is valid according to the version information of the operation data to be synchronized and the version information of the reference operation data. The reference operation data herein may include at least one of operation data that has been performed by the data center 310 and operation data that has been received by the data center 310 but has not been performed yet; wherein, the operation data executed by the data center 310 can be understood as having changed the data in the database 311 according to the operation data; operation data that has been received by data center 310 but has not yet been executed may be understood as not having been altered from the operation data in database 311, which may be from any data center in a multi-active data center architecture. Specifically, when determining whether the to-be-synchronized operation data is valid or not according to the version information of the to-be-synchronized operation data and the version information of the reference operation data, the data center 310 may determine whether the version information of the to-be-synchronized operation data is higher than the version information of the reference operation data, if so, it indicates that the generation time of the to-be-synchronized operation data is relatively late, and if not, it indicates that the generation time of the to-be-synchronized operation data is relatively early, and the to-be-synchronized operation data is invalid.

If the data center 310 determines that the operation data to be synchronized is valid through the above operation, a modification process based on the target data, such as insertion, update, or deletion of the target data, may be performed in the database 311 according to the operation data to be synchronized. On the contrary, if the data center 310 determines that the operation data to be synchronized is invalid through the above operation, the operation data to be synchronized may be ignored, i.e. the operation indicated by the operation data to be synchronized is not performed on the database 311.

It should be understood that the application scenario shown in fig. 3 is merely an example, and in practical application, the data synchronization method provided in the embodiment of the present application may also be applied to other scenarios, which is not limited in any way.

The data synchronization method provided by the application is described in detail by the method embodiment.

Referring to fig. 4, fig. 4 is a flowchart illustrating a data synchronization method according to an embodiment of the present application. The data synchronization method is applied to a target data center under a multi-activity data center architecture, as shown in fig. 4, and comprises the following steps:

step 401: receiving operation data to be synchronized sent by other data centers under the multi-activity data center architecture; the operation data to be synchronized is used for indicating to change based on target data, and is configured with version information, and the version information is determined according to the generation time of the operation data to be synchronized.

Under the multi-activity data center architecture, in order to ensure the data synchronization among the data centers, the data centers need to be synchronized through a data synchronization link, so that the databases corresponding to the data centers respectively store the total data under the multi-activity data center architecture.

For a target data center (which may be any data center under the multi-active data center architecture) under the multi-active data center architecture, it may receive to-be-synchronized operation data sent by other data centers under the multi-active data center architecture through a data synchronization link; it should be understood that the target data center may receive the data to be synchronized sent by a plurality of data centers at the same time, or may receive only the data to be synchronized sent by one data center.

It should be noted that, in the embodiment of the present application, the operation data to be synchronized is used to instruct to perform the change processing based on the target data; for example, the operation data to be synchronized may be used to indicate that the target data is inserted into the database corresponding to the target data center, for another example, the operation data to be synchronized may be used to indicate that the target data originally stored in the database corresponding to the target data center is deleted, for another example, the operation data to be synchronized may be used to indicate that the target data originally stored in the database corresponding to the target data center is updated into target update data, and so on. The data to be synchronized may be embodied as a data manipulation language (Data Manipulation Language, DML) in an SQL language set, where the DML is an instruction set responsible for performing data access tasks on database objects.

In addition, in the embodiment of the application, the operation data to be synchronized is configured with version information, and the version information is determined according to the generation time of the operation data to be synchronized; it should be understood that the higher the generation time of the operation data to be synchronized, the higher the version information of the operation data to be synchronized should be. For example, the version information of the operation data to be synchronized may be specifically expressed as a time stamp, and the later the generation time of the operation data to be synchronized is, the later the time indicated by the time stamp of the operation data to be synchronized is. For example, the version information of the data to be synchronized may be specifically expressed as a version number, and the later the generation time of the data to be synchronized is, the larger the version number of the data to be synchronized is. The present application is not limited in any way herein to the form of version information of the operation data to be synchronized.

In addition, in the embodiment of the application, the data to be synchronized can be configured with an active data center identifier, and the data center corresponding to the active data center identifier is the data center which initially generates the data to be synchronized. For example, a corresponding data center identifier may be configured for each data center in the multi-active data center architecture in advance, and when a certain data center sends operation data to be synchronized to a target data center, a source data center identifier may be configured for the operation data to be synchronized, where the source data center identifier indicates a source data center that originally generates the operation data to be synchronized, in addition to version information may be configured for the operation data to be synchronized.

It should be understood that, in the embodiment of the present application, each operation data generated in the multi-active data center architecture may be configured with corresponding version information and source data center identification. That is, each operation data generated under the multi-activity data center architecture belongs to operation data to be synchronized with other data centers, that is, operation data to be synchronized in the above, and accordingly, for each operation data generated in the multi-activity data center architecture, corresponding version information and source data center identification need to be configured.

It should be noted that, the method provided by the embodiment of the present application may be applied to a multi-active data center architecture that records operation data based on a data record table. The data record table is the data record table in the related art; the data record table may be configured separately for different data centers for recording operational data associated with the data centers; the data record table may be configured uniformly for each data center in the multi-activity data center architecture, wherein the data record table includes a key value interval corresponding to each data center, and operation data related to the data center is recorded in the corresponding key value interval for each data center.

When the method provided by the embodiment of the application is applied to a multi-activity data center architecture for recording operation data based on the data record table, two columns can be added in the data record table, one column is used for bearing version information of the operation data, and the other column is used for bearing source data center identification of the operation data.

Step 402: determining whether the operation data to be synchronized is valid or not according to the version information of the operation data to be synchronized and the version information of the reference operation data; the reference operation data includes at least one of operation data that has been executed by the target data center and operation data that has been received by the target data center but not executed.

After receiving the operation data to be synchronized sent by other data centers, the target data center can determine whether the operation data to be synchronized is valid or not according to the version information of the operation data to be synchronized and the version information of the reference operation data.

It should be noted that the reference operation data may include at least one of operation data that has been executed by the target data center and operation data that has been received by the target data center but not executed. The operation data executed by the target data center may be understood as that the target data center has changed the data stored in the corresponding database according to the operation data, for example, the target data center has executed a change operation such as data insertion, data deletion, data update, etc. according to the operation data. For operation data that the target data center has received but has not performed, it is understood that the target data center has received the operation data, but has not changed the data stored in its corresponding database according to the operation data.

It should be appreciated that the reference operation data is also configured with version information, which is determined based on the time at which the reference operation data was originally generated. That is, the reference operation data is essentially the operation data acquired by the target data center before the operation data to be synchronized is received, and such operation data may be directly written into the target data center by the service side, or may be transmitted to the target data center by other data centers in the multi-activity data center architecture.

In the embodiment of the application, the later the generation time of the operation data is, the higher the version information of the operation data is. On the basis, when the target data center specifically determines whether the operation data to be synchronized is valid, whether the version information of the operation data to be synchronized is higher than the version information of the reference operation data can be judged. If the version information of the operation data to be synchronized is higher than that of the reference operation data, determining that the operation data to be synchronized is valid; and if the version information of the operation data to be synchronized is lower than that of the reference operation data, determining that the operation data to be synchronized is invalid.

Specifically, if the version information of the operation data to be synchronized is higher than the version information of the reference operation data, the generation time of the operation data to be synchronized is later than the generation time of the reference operation data, and the operation data to be synchronized is still valid, that is, the operation data to be synchronized is valid. Otherwise, if the version information of the operation data to be synchronized is lower than the version information of the reference operation data, the generation time of the operation data to be synchronized is earlier than the generation time of the reference operation data, and the validity of the operation data to be synchronized is replaced by the reference operation data, so that the invalidation of the operation data to be synchronized can be determined.

As described in step 401 above, in the embodiment of the present application, besides the version information may be configured for the to-be-synchronized operation data, the source data center identifier may be configured for the to-be-synchronized operation data, and in some possible cases, the target data center may not be able to determine whether the to-be-synchronized operation data is valid according to the version information of the to-be-synchronized operation data and the version information of the reference operation data, and at this time, whether the to-be-synchronized operation data is valid may be determined according to the source data center identifier of the to-be-synchronized operation data and the source data center identifier of the reference operation data.

In practical applications, a large amount of operation data may be generated on each data center in the running process of the multi-activity data center architecture, so that a situation that a plurality of operation data are generated at the same time point easily occurs, and for a plurality of operation data generated at the same time, the generation time is the same, so that the corresponding version information of each operation data is the same. When the version information of the reference operation data is the same as the version information of the operation data to be synchronized, the target data center cannot simply rely on the version information of the reference operation data and the version information of the operation data to be synchronized to determine the validity of the operation data to be synchronized.

At this time, the target data center may further determine the validity of the operation data to be synchronized by means of the source data center identification of the operation data to be synchronized and the source data center identification of the reference operation data.

Specifically, for the multi-activity data center architecture, each data center is configured with a corresponding priority, and all the priorities corresponding to each data center have a full-order relationship, that is, the size relationship between the priorities corresponding to each data center is fixed and comparable. On the basis, the target data center can determine the data center corresponding to the source data center identifier of the operation data to be synchronized, and the data center corresponding to the source data center identifier of the reference operation data is determined as a first data center and a second data center; furthermore, the target data center may determine whether the priority of the first data center is higher than the priority of the second data center, if so, it may determine that the operation data to be synchronized is valid, and if not, it may determine that the operation data to be synchronized is invalid.

That is, in the embodiment of the present application, the size relationship between the priorities corresponding to the respective data centers in the multi-active data center architecture may be specified in advance, where the size relationship between the specified priorities needs to ensure uniqueness and comparability in the entire synchronous link (i.e., the link composed of the respective data centers in the multi-active data center architecture). Based on this, the target data center further determines the validity of the operation data to be synchronized by means of the priority of the data center generating the operation data to be synchronized and the priority of the data center generating the reference operation data in the case where the validity of the operation data to be synchronized cannot be determined based on only the version information of the operation data to be synchronized and the version information of the reference operation data, determines the validity of the operation data to be synchronized if the priority of the data center generating the operation data to be synchronized is higher than the priority of the data center generating the reference operation data, and determines the invalidity of the operation data to be synchronized if the priority of the data center generating the operation data to be synchronized is lower than the priority of the data center generating the reference operation data.

Therefore, based on the two layers of reference information, namely the version information and the priority of the data center corresponding to the source data center identification, the validity of the received data to be synchronized is determined, and whether the data to be synchronized is valid or not can be determined uniquely and reliably.

Step 403: if the operation data to be synchronized is determined to be effective, changing processing based on the target data is performed in a database corresponding to the target data center according to the operation data to be synchronized; and if the operation data to be synchronized is determined to be invalid, the operation data to be synchronized is ignored.

Through the operation, if the target data center determines that the operation data to be synchronized is valid, the change processing based on the target data can be performed in the database corresponding to the target data center according to the operation data to be synchronized. Otherwise, if the target data center determines that the operation data to be synchronized is invalid, the operation data to be synchronized can be ignored, that is, the change process indicated by the operation data to be synchronized is not executed in the database corresponding to the target data center.

The operation data in the DML mainly includes an INSERT (INSERT) statement, an UPDATE (UPDATE) statement, and a DELETE (DELETE) statement, and data synchronization for the DML is typically implemented according to a binary log. For INSERT statements, only the back mirror image, i.e. the inserted data, is in the binary log; for the UPDATE statement, there is both a front mirror image and a back mirror image in the binary log, i.e. there is data before and after updating; for DELETE statements, there is only a front image in the binary log and no back image, as the data has already been deleted.

In the embodiment of the present application, the version information of the operation data depends on the generation time of the operation data, so that the operation data is necessarily present in the post-mirror image, but the DELETE statement is not post-mirror image, so that the corresponding version information cannot be configured for the DELETE statement in the conventional DML, which also affects the implementation of the method provided by the embodiment of the present application. Fig. 5 is an exemplary schematic diagram corresponding to this case provided in the embodiment of the present application.

As shown in fig. 5, the data center a and the data center M are included, the time axes under the data center a and the data center M in fig. 5 are all from bottom to top, insert 1A indicates that the data center a inserts data with version number 1 in the corresponding database, insert 2A indicates that the data center a inserts data with version number 2 in the corresponding database, and delete 3M indicates that the data center M performs a delete operation on the data in the corresponding database. By observing the time axis below data center a and data center M shown in fig. 5, it can be seen that there is eventually no data on data center a, and the final data version on data center M is 2A.

The data center A and the data center M are asynchronous, because DELETE 3M belongs to DELETE operation, and DELETE operation has no method for carrying post-mirror image, and has no method for carrying corresponding version information; accordingly, when insert 1A is synchronized to data center M, data center M may misunderstand that there is no higher version of operation data operation, and perform the data modification process indicated by insert 1A, further, when insert 2A is synchronized to data center M, since its version information is higher than that of insert 1A, it is also performed.

In order to avoid the occurrence of the above situation, the embodiment of the application adjusts the original DELETE statement so that the operation data corresponding to the DELETE statement can carry the corresponding version information. Specifically, the operation data corresponding to the DELETE statement after adjustment may include a front mirror (i.e., the DELETE statement indicates deleted data, i.e., deleted data), a DELETE identifier (for identifying the front mirror as deleted data), version information, and a source data center identifier. The operation data corresponding to the adjusted DELETE statement can be understood as operation data corresponding to another form of UPDATE statement, that is, the post-mirror image is adjusted to DELETE the identifier based on the operation data corresponding to the original UPDATE statement.

In the case where the operation data to be synchronized is valid, when the operation data to be synchronized indicates deletion of the target data, that is, when the operation data to be synchronized is the operation data corresponding to a DELETE statement indicating deletion of the target data, the target data center may mark a deletion identifier for the target data in its corresponding database, while marking version information of the operation data to be synchronized, and marking the operation to be synchronized, where the marked deletion identifier is used to characterize that the current target data in the database is invalid. And when the preset time condition is met later, the target data center can delete the target data marked with the deletion mark.

Specifically, in the embodiment of the application, in addition to the adjustment of the operation data corresponding to the DELETE statement, the operation mode corresponding to the DELETE statement is also adjusted to a certain extent, so that the deleted data of the DELETE statement can be still reflected after the DELETE statement is executed, the execution time of the DELETE statement is conveniently marked, and further the DELETE statement is conveniently compared with the subsequently received operation data to be synchronized, so that the validity of the subsequently received operation data to be synchronized is determined.

That is, when the operation data to be synchronized is the operation data corresponding to the DELETE statement for indicating deletion of the target data, the target data center may mark the deletion identifier for the target data in the corresponding database when the target data center performs the change processing indicated by the operation data to be synchronized, and by marking the deletion identifier, it indicates that the target data in the database has failed, and does not support any further processing of the target data, and by marking the deletion identifier, the DELETE statement may be further embodied, thereby conveniently marking version information and source data center identifier of the operation data to be synchronized. This way of embodying DELETE statements by tagging DELETE identifications can also be understood as soft DELETE operations.

When the preset time condition is met, for example, after a preset time length from the execution time interval of the soft deletion operation, the target data center can execute a real deletion operation on the target data, namely, delete the target data marked with the deletion mark. It should be understood that after the above-mentioned preset time condition is satisfied, there is no more data to be synchronized with similar version information, that is, the validity of the subsequently received data to be synchronized is not required to be detected by using the version information of the target data and the source data center identifier, so that a real deleting operation can be performed to delete the target data.

In the case where the operation data to be synchronized is valid, when the operation data to be synchronized indicates insertion of the target data, that is, when the operation data to be synchronized is the operation data corresponding to an INSERT statement indicating insertion of the target data in the database, the target data center may INSERT the target data in its corresponding database, and mark version information of the operation data to be synchronized and mark a source data center identification of the operation data to be synchronized for the inserted target data.

For the operation data corresponding to the INSERT statement, when the target data center executes the INSERT operation indicated by the operation data, the difference is just similar to the way of executing the INSERT operation indicated by the conventional INSERT statement, in the embodiment of the application, the target data center executes the INSERT operation indicated by the operation data to be synchronized, after inserting the target data into the corresponding database, version information of the operation data to be synchronized and a source data center identifier are configured for the target data, so that the target data is conveniently used as reference operation data to be compared with the operation data to be synchronized subsequently received by the target data center, and the effectiveness of the operation data to be synchronized subsequently received is detected.

For INSERT operations, the underlying operation code is rewritten as follows:

INSERT IGNORE INTO table1 VALUES(id_src,c1_src,ts_src,node_name_src)

UPDATE table1 SET c1＝c1_src,ts＝ts_src,node_name＝node_name_src WHERE id＝id_src AND(ts,node_name)<＝(ts_src,node_name_src)

when the operation data to be synchronized is effective, the target data center may UPDATE the target data in the corresponding database as target UPDATE data when the operation data to be synchronized indicates that the target data is updated as target UPDATE data, that is, when the operation data to be synchronized is operation data corresponding to an UPDATE statement indicating that the target data originally stored in the database is updated as target UPDATE data; and, for the target update data, marking version information of the data to be synchronized, and marking a source data center identification of the data to be synchronized.

For the operation data corresponding to the UPDATE statement, when the target data center executes the UPDATE operation indicated by the operation data, the difference is just that in the embodiment of the application, the target data center executes the UPDATE operation indicated by the operation data to be synchronized, and after updating the target data into the target UPDATE data in the corresponding database, version information and source data center identification of the operation data to be synchronized are configured for the target UPDATE data, so that the target UPDATE data is conveniently used as reference operation data to be compared with the operation data to be synchronized subsequently received by the target data center, and the effectiveness of the operation data to be synchronized subsequently received is detected.

For the UPDATE operation, the underlying operation code is rewritten as follows:

INSERT IGNORE INTO table1 VALUES(id_src_new,c1_src_new,ts_src_new,node_name_src_new)UPDATE table1 SET c1＝c1_src_new,ts＝ts_src_new,node_name＝node_name_src_new WHERE id＝id_src_new AND(ts,node_name)<＝(ts_src_new,node_name_src_new)

the schematic diagram of the synchronization principle of the multi-active data center architecture working based on the data synchronization method provided by the embodiment of the application is shown in fig. 6, wherein when data synchronization is performed among the data center a, the data center B and the data center C, the data synchronization is realized based on a condition coverage, namely, whether the data to be synchronized are synchronized is determined according to the version information and the source data center identifier of the reference operation data and the data to be synchronized.

According to the data synchronization method, corresponding version information is configured for each item of operation data according to the generation time of each item of operation data, and data synchronization is performed based on the version information corresponding to each item of operation data, so that each data center under the multi-activity data center architecture can realize data synchronization operation based on the operation data of the latest version; in this case, even if a data collision occurs, the data centers can be synchronized based on the latest version of operation data, so that the data consistency among the data centers can be ensured. That is, even if a data conflict occurs under the multi-active data center architecture, each data center under the architecture can automatically resolve the data conflict according to a pre-configured policy, thereby improving the robustness of the whole multi-active data center architecture.

The application also provides a corresponding data synchronization device for the data synchronization method described above, so that the data synchronization method can be practically applied and realized.

Referring to fig. 7, fig. 7 is a schematic structural diagram of a data synchronization apparatus 700 corresponding to the data synchronization method shown in fig. 4 above, which is deployed in a target data center under a multi-active data center architecture including a plurality of data centers, the target data center being any one of the plurality of data centers having their respective corresponding databases. As shown in fig. 7, the data synchronization apparatus 700 includes:

a data receiving module 701, configured to receive operation data to be synchronized sent by other data centers under the multi-activity data center architecture; the operation data to be synchronized is used for indicating to change based on target data, and is configured with version information, and the version information is determined according to the generation time of the operation data to be synchronized;

a data detection module 702, configured to determine whether the operation data to be synchronized is valid according to the version information of the operation data to be synchronized and the version information of the reference operation data; the reference operation data includes at least one of operation data that has been executed by the target data center and operation data that has been received by the target data center but has not been executed;

A data changing module 703, configured to, if it is determined that the operation data to be synchronized is valid, perform, according to the operation data to be synchronized, a change process based on the target data in a database corresponding to the target data center; and if the operation data to be synchronized is determined to be invalid, the operation data to be synchronized is ignored.

Optionally, the later the generation time of the operation data, the higher the version information of the operation data; the data detection module 702 is specifically configured to:

if the version information of the operation data to be synchronized is higher than the version information of the reference operation data, determining that the operation data to be synchronized is valid; and if the version information of the operation data to be synchronized is lower than the version information of the reference operation data, determining that the operation data to be synchronized is invalid.

Optionally, the data to be synchronized is further configured with an active data center identifier, and a data center corresponding to the active data center identifier is a data center for initially generating the data to be synchronized; the data detection module 702 is further configured to:

and under the condition that whether the operation data to be synchronized is effective or not can not be determined according to the version information of the operation data to be synchronized and the version information of the reference operation data, determining whether the operation data to be synchronized is effective or not according to the source data center identification of the operation data to be synchronized and the source data center identification of the reference operation data.

Optionally, each data center in the multi-activity data center architecture is configured with a corresponding priority; the data detection module 702 is specifically configured to:

determining a first data center corresponding to the source data center identifier of the operation data to be synchronized, and determining a second data center corresponding to the source data center identifier of the reference operation data;

if the priority corresponding to the first data center is higher than the priority corresponding to the second data center, determining that the data to be synchronized are valid; and if the priority corresponding to the first data center is lower than the priority corresponding to the second data center, determining that the operation data to be synchronized is invalid.

Optionally, when the operation data to be synchronized indicates to delete the target data, the data change module 703 is specifically configured to:

marking a deletion identifier, version information of the data to be synchronized and a source data center identifier of the data to be synchronized aiming at the target data in a database corresponding to the target data center; the deletion identifier is used for representing the current target data failure in the database;

and deleting the target data marked with the deletion mark when a preset time condition is met.

Optionally, when the operation data to be synchronized indicates insertion of the target data, the data modification module 703 is specifically configured to:

inserting the target data into a database corresponding to the target data center;

and marking version information of the operation data to be synchronized according to the target data, and marking a source data center identifier of the operation data to be synchronized.

Optionally, when the operation data to be synchronized indicates that the target data is updated to target update data, the data change module 703 is specifically configured to:

updating the target data in the database corresponding to the target data center into the target updated data;

and aiming at the target updating data, marking version information of the data to be synchronized, and marking a source data center identifier of the data to be synchronized.

The data synchronization device correspondingly configures corresponding version information for each item of operation data according to the generation time of each item of operation data, and performs data synchronization based on the version information corresponding to each item of operation data so as to ensure that each data center under the multi-activity data center architecture realizes data synchronization operation based on the operation data of the latest version; in this case, even if a data collision occurs, the data centers can be synchronized based on the latest version of operation data, so that the data consistency among the data centers can be ensured. That is, even if a data conflict occurs under the multi-active data center architecture, each data center under the architecture can automatically resolve the data conflict according to a pre-configured policy, thereby improving the robustness of the whole multi-active data center architecture.

The embodiment of the application also provides a computer device for realizing data synchronization, which can be a server in particular, and the server provided by the embodiment of the application is introduced from the aspect of hardware materialization.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a server 800 according to an embodiment of the present application. The server 800 may vary considerably in configuration or performance and may include one or more central processing units (central processing units, CPUs) 822 (e.g., one or more processors) and memory 832, one or more storage media 830 (e.g., one or more mass storage devices) storing applications 842 or data 844. Wherein the memory 832 and the storage medium 830 may be transitory or persistent. The program stored in the storage medium 830 may include one or more modules (not shown), each of which may include a series of instruction operations on a server. Still further, the central processor 822 may be configured to communicate with the storage medium 830 to execute a series of instruction operations in the storage medium 830 on the server 800.

The Server 800 may also include one or more power supplies 826, one or more wired or wireless network interfaces 850, one or more input/output interfaces 858, and/or one or more operating systems, such as Windows Server ^TM ，Mac OS X ^TM ，Unix ^TM ,Linux ^TM ，FreeBSD ^TM Etc.

The steps performed by the server in the above embodiments may be based on the server structure shown in fig. 8.

CPU 822, among other things, is configured to perform the following steps:

Optionally, CPU 822 may also be used to perform the steps of any one implementation of the data synchronization method provided by embodiments of the present application.

The embodiments of the present application also provide a computer readable storage medium storing a computer program for executing any one of the data synchronization methods described in the foregoing embodiments.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform any one of the data synchronization methods described in the foregoing respective embodiments.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.

In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: u disk, mobile hard disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk, etc. various media for storing computer program.

It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "and/or" for describing the association relationship of the association object, the representation may have three relationships, for example, "a and/or B" may represent: only a, only B and both a and B are present, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.

The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. The data synchronization method is characterized by being applied to a target data center under a multi-activity data center architecture, wherein the multi-activity data center architecture comprises a plurality of data centers, the target data center is any one of the plurality of data centers, and the plurality of data centers are provided with corresponding databases; the method comprises the following steps:

2. The method according to claim 1, wherein the later the generation time of the operation data is, the higher the version information of the operation data is; determining whether the operation data to be synchronized is valid according to the version information of the operation data to be synchronized and the version information of the reference operation data, including:

3. The method of claim 1, wherein the data to be synchronized is further configured with an active data center identification, the data center to which the active data center identification corresponds being the data center that originally generated the data to be synchronized; the method further comprises the steps of:

4. The method of claim 3, wherein each data center in the multi-active data center architecture is configured with a respective priority; the determining whether the operation data to be synchronized is valid according to the source data center identifier of the operation data to be synchronized and the source data center identifier of the reference operation data comprises the following steps:

5. The method according to claim 3, wherein when the operation data to be synchronized indicates deletion of the target data, the performing, according to the operation data to be synchronized, a change process based on the target data in a database corresponding to the target data center includes:

6. The method according to claim 3, wherein when the operation data to be synchronized indicates insertion of the target data, the performing, according to the operation data to be synchronized, a change process based on the target data in a database corresponding to the target data center includes:

7. The method according to claim 3, wherein when the operation data to be synchronized indicates that the target data is updated to target update data, the performing, according to the operation data to be synchronized, a change process based on the target data in a database corresponding to the target data center includes:

8. A data synchronization device, characterized by a target data center deployed under a multi-active data center architecture, the multi-active data center architecture comprising a plurality of data centers, the target data center being any one of the plurality of data centers, the plurality of data centers having their respective databases; the device comprises:

9. A computer device, the device comprising a processor and a memory;

the memory is used for storing a computer program;

the processor is configured to perform the data synchronization method of any one of claims 1 to 7 according to the computer program.

10. A computer readable storage medium for storing a computer program for executing the data synchronization method of any one of claims 1 to 7.

11. A computer program product comprising a computer program or instructions which, when executed by a processor, implements the data synchronization method of any one of claims 1 to 7.