CN113486109A

CN113486109A - Data synchronization method and device of heterogeneous database and electronic equipment

Info

Publication number: CN113486109A
Application number: CN202110608441.XA
Authority: CN
Inventors: 雷周纯
Original assignee: Peoples Insurance Company of China
Current assignee: Peoples Insurance Company of China
Priority date: 2021-06-01
Filing date: 2021-06-01
Publication date: 2021-10-08

Abstract

The application discloses a data synchronization method and device of heterogeneous databases and electronic equipment, and aims to improve data synchronization efficiency among the heterogeneous databases. The method comprises the following steps: acquiring data operation information corresponding to data change information in a source database, wherein the data operation information comprises data and an execution sequence aimed at by data operation; performing conflict detection on an insertion operation in the data operation corresponding to the data change information to identify whether a primary key conflict exists between the data targeted by the insertion operation and existing data in a target database; reconstructing the target insertion operation into a corresponding deletion operation and a new insertion operation based on the data targeted by the target insertion operation with the primary key conflict so as to obtain reconstructed data operation information; and executing corresponding data operation on the target database based on the reconstructed data operation information so as to synchronize the data change information to the target database.

Description

Data synchronization method and device of heterogeneous database and electronic equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to a data synchronization method and apparatus for a heterogeneous database, and an electronic device.

Background

Any changes to the database are recorded in the logical log file in the form of a binary file. At present, data synchronization between heterogeneous databases is usually based on synchronization of logical log files of a source database, that is, the logical log files of the source database are analyzed into corresponding data operations and transmitted to a consumption program of a data synchronization end, and the consumption program executes the corresponding data operations on a target database according to a received sequence, thereby realizing data synchronization between the heterogeneous databases.

Due to the unreliability of the transmission process, the consuming program needs to reconstruct all the insertion operations to ensure the idempotency of the data, and obviously, this approach has the problem of inefficient data synchronization, especially when a large number of insertion operations are received. Therefore, how to improve the efficiency of data synchronization between heterogeneous databases still needs further solutions.

Disclosure of Invention

An embodiment of the application aims to provide a data synchronization method and device for heterogeneous databases and electronic equipment, so as to improve data synchronization efficiency between the heterogeneous databases.

In order to solve the technical problem, the embodiment of the application adopts the following technical scheme:

in a first aspect, a data synchronization method for heterogeneous databases is provided, including:

acquiring data operation information corresponding to data change information in a source database, wherein the data operation information comprises data and an execution sequence aiming at data operation;

performing conflict detection on an insertion operation in the data operation corresponding to the data change information to identify whether a primary key conflict exists between the data targeted by the insertion operation and existing data in a target database;

reconstructing the target insertion operation into a corresponding deletion operation and a new insertion operation based on the data targeted by the target insertion operation with the primary key conflict so as to obtain reconstructed data operation information;

and executing corresponding data operation on the target database based on the reconstructed data operation information so as to synchronize the data change information to the target database.

In a second aspect, a data synchronization apparatus for heterogeneous databases is provided, including:

the acquisition module acquires data operation information corresponding to data change information in a source database, wherein the data operation information comprises data and an execution sequence aiming at data operation;

the conflict detection module is used for carrying out conflict detection on the insertion operation in the data operation corresponding to the data change information so as to identify whether the data aimed by the insertion operation has a primary key conflict with the existing data in the target database;

the reconstruction module reconstructs the target insertion operation into a corresponding deletion operation and a new insertion operation based on the data targeted by the target insertion operation with the primary key conflict so as to obtain reconstructed data operation information;

and the synchronization module executes corresponding data operation on the target database based on the reconstructed data operation information so as to synchronize the data change information to the target database.

In a third aspect, an electronic device is provided, including: memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the method according to the first aspect.

In a fourth aspect, a computer-readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, carries out the steps of the method according to the first aspect.

In one or more embodiments provided in the embodiments of the present application, conflict detection is performed on an insert operation corresponding to data change information of a source database, a target insert operation in which a primary key conflict exists is reconfigured as a delete operation and a new insert operation, and then the reconfigured data operation is performed on the target database. Because the occurrence probability of the data with the main key conflict is less, the insertion operation with the main key conflict is reconstructed only under the condition that the data is repeatedly consumed generally, through the scheme provided by the embodiment of the application, on one hand, the reconstructed data operation amount can be reduced, on the other hand, the insertion operation without the main key conflict is not reconstructed and is executed according to the original insertion strategy, and compared with the reconstructed strategy, the original insertion strategy has higher data synchronization performance, so that the data synchronization efficiency between heterogeneous databases can be improved. Moreover, by reconstructing the insertion operation with the main key conflict into the deletion operation and the new insertion operation, the influence on the correctness of the data synchronization result due to repeated consumption of data change can be avoided, the idempotent characteristic of the data in the target database is ensured, and the performance of the target database is improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a schematic flowchart illustrating a data synchronization method for heterogeneous databases according to an embodiment of the present application;

fig. 2 is a schematic diagram illustrating an implementation of a data synchronization method for heterogeneous databases according to an embodiment of the present application;

fig. 3 is a schematic diagram illustrating an implementation of a data synchronization method for heterogeneous databases according to another embodiment of the present application;

fig. 4 is a schematic diagram illustrating an implementation of a data synchronization method for heterogeneous databases according to another embodiment of the present application;

fig. 5 is a schematic diagram illustrating an implementation of a data synchronization method for heterogeneous databases according to yet another embodiment of the present application;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a data synchronization apparatus for heterogeneous databases according to an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more clear, the technical solutions of the present application will be clearly and completely described below with reference to the specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In order to improve data synchronization efficiency between heterogeneous databases, one or more embodiments of the present application provide a data synchronization method for heterogeneous databases, where collision detection is performed on an insert operation corresponding to data change information of a source database, a target insert operation with a primary key collision is reconstructed into a delete operation and a new insert operation, and then the reconstructed data operation is performed on the target database. Due to the fact that the occurrence probability of data with main key conflict is low, the reconstruction data operation amount can be reduced through the scheme provided by the embodiment of the application only under the condition that the data are repeatedly consumed, and therefore the data synchronization efficiency between heterogeneous databases is improved. Moreover, by reconstructing the insertion operation with the main key conflict into the deletion operation and the new insertion operation, the influence on the correctness of the data synchronization result due to repeated consumption of data change can be avoided, the idempotent characteristic of the data in the target database is ensured, and the performance of the target database is improved.

It should be understood that the data synchronization method for the heterogeneous databases provided in the embodiments of the present application may be executed by an electronic device or software installed in the electronic device, and specifically may be executed by a terminal device or a server device. More specifically, the electronic device may be a destination to which the target database belongs.

For convenience of description, the following describes an embodiment of the method as an example of the method being executed by a destination. It is to be understood that the implementation of the method is merely an exemplary illustration and should not be construed as a limitation of the method.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Referring to fig. 1, a schematic flow chart of a data synchronization method for heterogeneous databases according to an embodiment of the present application is provided, where the method includes:

s102, acquiring data operation information corresponding to data change information in a source database, wherein the data operation information comprises data and an execution sequence aimed at by data operation.

In the embodiment of the present application, the data change information refers to information for characterizing data change. The data operation information corresponding to the data change information refers to information of a data operation that can reflect a data change, and may specifically include data targeted by the data operation and an execution order of the data operation. Wherein the data operation may include at least one of an insert operation (insert), a delete operation (delete), and an modify operation (update). The data targeted by the data operation includes a primary key (id) and a key value (value) corresponding to the primary key.

Specifically, the manner of acquiring the data operation information corresponding to the data change information of the source database may include, but is not limited to, the following two manners, that is, manner 1: the destination end acquires the data change information in the source database from the source end and generates corresponding data operation information based on the acquired data change information. Mode 2: the destination receives data operation information corresponding to the data change information in the source database, where the data operation information is generated by the source based on the data change information in the source database. It should be noted that, with the acquisition method provided in the manner 2, the destination does not need to perform local analysis by itself, and thus the workload of the destination can be reduced.

More specifically, the data change in the database is recorded in the logical log file in a binary file manner, so that the data change in the source database can be parsed into the corresponding data operation by parsing the logical log file of the source database.

For example, as shown in the data table in fig. 2, the data table obtained after the data in the source database is changed, and the data operation that can be obtained by analyzing the data change information in the source database includes:

insert(id＝1,value＝hello),insert(id＝2,value＝hey),insert(id＝3,value＝alloha),inser t(id＝4,value＝morning),insert(id＝5,value＝whatsup),insert(id＝6,value＝hi),update(id＝ 4,value＝howdy),update(id＝4,value＝nihao),delete(id＝5),delete(id＝1)。

and S104, performing conflict detection on the inserting operation in the data operation corresponding to the data change information to identify whether the data aimed by the inserting operation has a primary key conflict with the existing data in the target database.

Because the data change repeated for many times is a main factor influencing the idempotency of the data in the database, and the data change repeated for many times is usually caused by the insertion operation, if the data targeted by the insertion operation has a primary key conflict with the existing data in the target database, the execution of the insertion operation will cause the repeated data change, thereby influencing the idempotency of the data in the target database. Based on the above, conflict detection can be performed on the insertion operation corresponding to the data change information in the source database, so as to identify whether the data targeted by the insertion operation has a primary key conflict with the existing data in the target database.

It should be noted that, any appropriate manner commonly used in the art may be adopted for the manner of performing collision detection on the insertion operation, and this is not specifically limited in this application embodiment.

And S106, reconstructing the target insertion operation into a corresponding deletion operation and a new insertion operation based on the data targeted by the target insertion operation with the primary key conflict so as to obtain the reconstructed data operation information.

In order to avoid the influence of the target insertion operation with the primary key conflict on the idempotency of the data in the target database after the target insertion operation is executed, the target insertion operation can be reconstructed into a corresponding deletion operation and a new insertion operation, and the deletion operation is performed before the new insertion operation, so that the primary key conflict can be eliminated through the reconstructed deletion operation, and then the data change information in the source database is synchronized into the target database through the new insertion operation.

Specifically, before a target insertion operation in which a primary key conflict exists, a deletion operation for deleting a primary key and a corresponding key value for the target operation may be added, thereby obtaining a deletion operation and a new insertion operation corresponding to the target insertion operation.

For example, as shown in fig. 2, before a target insertion operation insert (id-1, value-hello), insert (id-2, value-hey), insert (id-3, value-alloha), insert (id-4, value-moving), insert (id-5, value-whatsu), a corresponding deletion operation delete (id-1), delete (id-2), delete (id-3), delete (id-4, value-4), delete (id-5).

Further, if there are a plurality of consecutive target insertion operations, a deletion operation for deleting the primary key and the corresponding key value respectively targeted by the consecutive target insertion operations may be added before a first target insertion operation among the plurality of consecutive target insertion operations. For example, as shown in fig. 3, a delete operation delete (

id

1,2,3,4,5) may be inserted into a target insert operation insert (id 1, value hello) in which a primary key collision exists.

And S108, executing corresponding data operation on the target database based on the reconstructed data operation information so as to synchronize the data change information in the source database to the target database.

After the target insertion operation with the primary key conflict is reconstructed into the deletion operation and the new insertion operation, new data operation information corresponding to the data change information in the source database can be obtained, and further, corresponding data operation can be executed on the target database based on the data and the execution sequence of the data operation in the new data operation information, so that the data change information in the source database is synchronized to the target database.

By adopting the data synchronization method of the heterogeneous database provided by the embodiment of the application, the conflict detection is carried out on the insertion operation corresponding to the data change information of the source database, the target insertion operation with the main key conflict is reconstructed into the deletion operation and the new insertion operation, and then the reconstructed data operation is executed on the target database. Because the occurrence probability of the data with the primary key conflict is less, the insertion operation with the primary key conflict is reconstructed only under the condition that the data is repeatedly consumed generally, through the scheme provided by the embodiment of the application, on one hand, the reconstructed data operation amount can be reduced, on the other hand, the insertion operation without the primary key conflict is not reconstructed and is executed according to the original insertion strategy, and compared with the reconstructed strategy, the original insertion strategy has higher data synchronization performance, so that the data synchronization efficiency between heterogeneous databases can be improved. Moreover, by reconstructing the insertion operation with the main key conflict into the deletion operation and the new insertion operation, the influence on the correctness of the data synchronization result due to repeated consumption of data can be avoided, the idempotent characteristic of the data in the target database is ensured, and the performance of the target database is improved.

The following describes in detail a data synchronization method for a heterogeneous database according to an embodiment of the present application.

In the scheme of the embodiment of the application, conflict detection and reconstruction can be performed for a single insertion operation.

In order to further improve the data synchronization efficiency, as a preferred embodiment, the data operations corresponding to the data change information in the source database may be grouped, conflict detection and reconstruction may be performed in units of groups, and each group of data operations may be sequentially performed on the target database, and one group of data operations is completed in one transaction, so as to ensure atomicity of the transaction. Specifically, in S104, performing collision detection on an insertion operation in a data operation corresponding to the data change information includes:

step a1, grouping data operations corresponding to the data change information based on the execution order of the data operations corresponding to the data change information to obtain multiple sets of data operations.

The number of data operations included in each group of data operations can be determined according to actual needs. In order to ensure that the performance of the target database is better and simultaneously consider reducing the possibility of target database deadlock caused by long transactions, the number n of data operations included in each group of data operations can be any value in the following value ranges: n is more than or equal to 10 and less than or equal to 500. For example, as shown in fig. 4, by grouping data operations corresponding to data change information in the source database, two sets of data operations as shown in fig. 4 can be obtained.

Step A2, allocating corresponding transactions for the multiple groups of data operations respectively and executing the allocated transactions to the target database in sequence.

After obtaining a plurality of groups of data operations, corresponding transactions can be allocated to each group of data operations, and the allocated transactions are executed in sequence according to the execution sequence of each group of data operations, so that the same group of data operations can be executed in the same transaction, and the data operations are either all executed successfully or all executed unsuccessfully, thereby ensuring the atomicity of the transaction.

Step A3, during the execution of the current transaction, performing conflict detection on the insert operation in the current transaction to identify whether there is a primary key conflict between the data targeted by the insert operation in the current transaction and the existing data in the target database.

Accordingly, in the above S106, if the data targeted by the insert operation in the current transaction has a primary key conflict with the existing data in the target database, the current transaction fails to execute, and the target insert operation in the current transaction having the primary key conflict is further reconstructed into a corresponding delete operation and a new insert operation, so as to obtain a new transaction. Accordingly, in S108 described above, a new transaction is performed on the target database.

Further, in order to further improve the accuracy of the data synchronization result, before the step S106, the method further includes: and if the data targeted by the insert operation in the current transaction is in primary key conflict with the existing data in the target database, rolling back the target database to the state before the current transaction is processed.

Further, if there is no primary key conflict between the data targeted by the insert operation in the current transaction and the existing data in the target database, the current transaction will be executed successfully, the next transaction is further executed, and the foregoing process is repeated until the execution of all transactions is completed.

For example, as shown in fig. 4, in the process of executing the transaction corresponding to the group 1 data operation on the target database, if it is detected that all the insert operations included in the transaction have a primary key conflict, the insert operations in the transaction are reconstructed to obtain a new transaction, and then the new transaction is executed. And after the new transaction corresponding to the 1 st group of data operation is successfully executed, executing the transaction corresponding to the 2 nd group of data operation, and if the inserting operation with the primary key conflict is not detected in the executing process of the transaction, the transaction is successfully executed.

It can be understood that by grouping the data operations corresponding to the data change information, allocating corresponding transactions to each group of data operations and executing the allocated transactions, the execution and collision detection of batch data operations can be realized, and thus the efficiency of data synchronization between heterogeneous databases can be further improved.

Optionally, in another embodiment, before allocating corresponding transactions for the multiple groups of data operations respectively and executing the allocated transactions to the target database in sequence, the method may further include: and detecting whether the same group of data operations contains the same continuous multiple data operations, and if so, merging the continuous multiple data operations of the same type.

As an alternative embodiment, considering that a plurality of insertion operations in succession in the same group of data operations can be driven by a modern database, if a plurality of insertion operations in succession are included in the same group of data operations, the plurality of insertion operations in succession are merged for processing to obtain a new insertion operation. Specifically, a consecutive plurality of insertion operations insert may be combined into a new insertion operation such as insert value (…), (…), (…).

Considering that a plurality of deletion operations in succession in the same group of data operations can also be driven by the modern database, if the plurality of deletion operations in succession are included in the same group of data operations, the deletion operations in succession are merged to obtain a new deletion operation. For example, as shown in fig. 5, the 2 nd group of data operations includes two consecutive deletion operations delete (id ═ 1) and delete (id ═ 5), and the two deletion operations may be merged into delete (id ═ 1, 2).

Considering that the final key value of the primary key depends on the last modification when the key values of the same primary key are continuously modified, for this reason, if a plurality of continuous modification operations for the same primary key are included in the same group of data operations, the modification operations except the modification operation which is executed last in the plurality of continuous modification operations are deleted. Therefore, when the same group of data operations comprises a plurality of continuous modification operations aiming at the same main key, only the last modification operation needs to be executed, and the data synchronization efficiency among heterogeneous databases can be further improved. For example, as shown in fig. 5, if two consecutive modification operations update (id 4, value 4) and update (id 4, value nhao) for the primary key id 4 are included in the 2 nd group of data operations, the update (id 4, value howdy) and the packet retention update (id 4, value nhao) may be deleted.

It can be understood that, in this embodiment, by merging a plurality of consecutive data operations of the same type, batch detection, reconstruction, and execution of the data operations can be implemented, so as to further improve data synchronization efficiency and performance between heterogeneous databases.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Referring to fig. 6, at a hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory, such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services.

The processor, the network interface, and the memory may be connected to each other via an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 6, but that does not indicate only one bus or one type of bus.

And the memory is used for storing programs. In particular, the program may include program code comprising computer operating instructions. The memory may include both memory and non-volatile storage and provides instructions and data to the processor.

The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form the data synchronization device of the heterogeneous database on a logic level. The processor executes the program stored in the memory and is specifically used for executing the following operations:

The method performed by the data synchronization apparatus for heterogeneous databases according to the embodiment shown in fig. 1 of the present application may be applied to or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.

The electronic device may further execute the method in fig. 1, and implement the functions of the data synchronization apparatus for a heterogeneous database in the embodiment shown in fig. 1, which are not described herein again in this embodiment of the present application.

Of course, besides the software implementation, the electronic device of the present application does not exclude other implementations, such as a logic device or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or a logic device.

Embodiments of the present application also provide a computer-readable storage medium storing one or more programs, where the one or more programs include instructions, which when executed by a portable electronic device including a plurality of application programs, enable the portable electronic device to perform the method of the embodiment shown in fig. 1, and are specifically configured to:

Fig. 7 is a schematic structural diagram of a data synchronization apparatus for heterogeneous databases according to an embodiment of the present application. Referring to fig. 7, in a software implementation, a database synchronization apparatus 700 may include:

an obtaining module 710, configured to obtain data operation information corresponding to data change information in a source database, where the data operation information includes data and an execution sequence targeted by a data operation;

a conflict detection module 720, configured to perform conflict detection on an insertion operation in the data operations corresponding to the data change information, so as to identify whether a primary key conflict exists between data targeted by the insertion operation and existing data in a target database;

the reconstructing module 730 reconstructs a target insertion operation into a corresponding deletion operation and a new insertion operation based on the data targeted by the target insertion operation with the primary key conflict, so as to obtain reconstructed data operation information;

the synchronization module 740 executes corresponding data operation on the target database based on the reconstructed data operation information, so as to synchronize the data change information to the target database. Optionally, the collision detection module 720 is specifically configured to:

grouping the data operations corresponding to the data change information based on the execution sequence of the data operations corresponding to the data change information to obtain a plurality of groups of data operations;

distributing corresponding transactions for the multiple groups of data operations respectively and executing the distributed transactions to the target database in sequence;

during the execution process of the current transaction, performing conflict detection on the insertion operation in the current transaction to identify whether primary key conflict exists between the data targeted by the insertion operation in the current transaction and the existing data in the target database.

Optionally, the apparatus further comprises:

a rollback module configured to, before the reconstructing module 730 reconstructs the target insert operation into a corresponding delete operation and a new insert operation based on the data targeted by the target insert operation having the primary key conflict, if the data targeted by the insert operation in the current transaction has the primary key conflict with the existing data in the target database, rollback the target database to a state before processing the current transaction;

the reconstructing module 730 is specifically configured to reconstruct, based on the data targeted by the target insert operation in which the primary key conflict exists in the current transaction, the target insert operation into a corresponding delete operation and a new insert operation, so as to obtain a new transaction;

the synchronization module 740 is specifically configured to execute the new transaction on the target database.

Optionally, the apparatus further comprises:

a similar detection module, configured to detect whether a same set of data operations includes multiple consecutive data operations of the same type before the conflict detection module 720 allocates corresponding transactions to the multiple sets of data operations and sequentially executes the allocated transactions on the target database;

and the merging processing module is used for merging the continuous multiple data operations of the same type if the same group of data operations contains the continuous multiple data operations of the same type.

Optionally, the merging processing module is specifically configured to:

if the same group of data operation comprises a plurality of continuous insert operations, combining the continuous insert operations to obtain a new insert operation; and/or the presence of a gas in the gas,

and if the same group of data operation comprises a plurality of continuous deleting operations, combining the plurality of continuous deleting operations to obtain a new deleting operation.

Optionally, the merging processing module is specifically configured to:

and if the same group of data operation comprises a plurality of continuous modification operations aiming at the same main key, deleting the other modification operations except the modification operation executed last in the plurality of continuous modification operations.

Optionally, the obtaining module 710 is specifically configured to:

and acquiring the data operation information provided by a source end, wherein the data operation information is generated by the source end based on data change information in the source database.

In short, the above description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The embodiments in the present description are described in a progressive manner, and similar parts between the embodiments are referred to each other, and each embodiment focuses on different points from other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

Claims

1. A data synchronization method for heterogeneous databases is characterized by comprising the following steps:

acquiring data operation information corresponding to data change information in a source database, wherein the data operation information comprises data and an execution sequence aimed at by data operation;

2. The method according to claim 1, wherein the performing collision detection on the insertion operation in the data operation corresponding to the data change information comprises:

3. The method according to claim 2, before reconstructing the target insert operation into a corresponding delete operation and a new insert operation based on data targeted by the target insert operation for which there is a primary key conflict, further comprising:

if the data targeted by the insert operation in the current transaction has a primary key conflict with the existing data in the target database, rolling back the target database to a state before the current transaction is processed;

reconstructing the target insertion operation into a corresponding deletion operation and a new insertion operation based on the data targeted by the target insertion operation with the primary key conflict, wherein the reconstructing comprises the following steps:

reconstructing the target insertion operation into a corresponding deletion operation and a new insertion operation based on the data targeted by the target insertion operation with the primary key conflict in the current transaction to obtain a new transaction;

the executing corresponding data operation on the target database based on the reconstructed data operation information includes:

executing the new transaction against the target database.

4. The method of claim 2, further comprising, before allocating respective transactions for the plurality of sets of data operations and executing the allocated transactions in sequence on the target database, respectively:

detecting whether a plurality of continuous data operations of the same type are contained in the same group of data operations;

and if so, merging the continuous multiple data operations of the same type.

5. The method of claim 4, wherein the merging the consecutive data operations of the same type comprises:

if the same group of data operation comprises a plurality of continuous insert operations, combining the plurality of continuous insert operations to obtain a new insert operation; and/or the presence of a gas in the gas,

6. The method of claim 4, wherein the merging the consecutive data operations of the same type comprises:

7. The method of claim 1, wherein the obtaining data operation information corresponding to data change information in the source database comprises:

and acquiring the data operation information provided by a source end, wherein the data operation information is generated by the source end based on the data change information in the source database.

8. A data synchronization apparatus for heterogeneous databases, comprising:

the reconstruction module reconstructs target insertion operation into corresponding deletion operation and new insertion operation based on the data targeted by the target insertion operation with the primary key conflict so as to obtain reconstructed data operation information;

9. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, carries out the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.