WO2021237704A1

WO2021237704A1 - Data synchronization method and related device

Info

Publication number: WO2021237704A1
Application number: PCT/CN2020/093374
Authority: WO
Inventors: 郭子亮
Original assignee: 深圳市欢太科技有限公司; Oppo广东移动通信有限公司
Priority date: 2020-05-29
Filing date: 2020-05-29
Publication date: 2021-12-02
Also published as: CN115104092A

Abstract

A data synchronization method, comprising: first, processing first binary logs in a first database in batches according to a preset batching rule to obtain N sub binary logs, N being a positive integer (201); then, inserting a data marker into a preset position of each sub binary log to obtain second binary logs (202); and finally, completing the data synchronization between the first database and a second database according to the second binary logs (203). Binary logs can be processed in batches and marked, so that which data is generated by the system is automatically identified during data synchronization between remote databases, to avoid repeated circular replication of the binary logs in the two databases, thereby improving the data synchronization effect.

Description

Data synchronization method and related device

Technical field

The invention relates to the technical field of database storage, in particular to a data synchronization method and related devices.

Background technique

With the progress of society, more and more data are generated every day, and many applications need to process massive amounts of data, which puts forward very high requirements for data storage and reading. As a tool for processing structured data, databases face huge challenges. Nowadays, independent data centers are generally established in different cities. In order to ensure the consistency of the data in the databases in various regions, two-way data synchronization is required.

In the two-way synchronization scheme, because there will be data written in the two databases, in some cases, if the same data is updated with different values in the two places, it will cause the binary log to be continuously transferred in the two databases. Local data will also be replicated continuously, which affects the effect of data synchronization.

Summary of the invention

Based on the above-mentioned problems, this application proposes a data synchronization method and related device. By performing batch processing and marking processing on binary logs, when data synchronization between remote databases is performed, it is possible to identify which data is generated by the system and avoid The binary log is replicated continuously in the two databases, which greatly improves the effect of data synchronization.

In the first aspect, an embodiment of the present application provides a data synchronization method, and the method includes:

The first binary log in the first database is processed in batches according to a preset batching rule to obtain N sub-binary logs, where N is a positive integer;

Insert a data mark at the preset position of each sub-binary log to obtain the second binary log;

The data synchronization between the first database and the second database is completed according to the second binary log.

In a second aspect, an embodiment of the present application provides a data synchronization device, and the device includes:

The batching unit is used to process the first binary log in the first database in batches according to a preset batching rule to obtain N sub-binary logs, where N is a positive integer;

The marking unit is used to insert a data mark at the preset position of each sub-binary log to obtain the second binary log;

The synchronization unit is configured to complete data synchronization between the first database and the second database according to the second binary log.

In a third aspect, an embodiment of the present application provides an electronic device, including an application processor, a memory, and one or more programs. The one or more programs are stored in the memory and configured by the The application processor executes the program, and the program includes instructions for executing the steps in the method according to any one of claims 1-7.

In the fourth aspect, in the fourth aspect, an embodiment of the present application provides a computer storage medium, the computer storage medium stores a computer program, and the computer program includes program instructions that, when executed by a processor, cause all The processor executes the method according to any one of the first aspects of the embodiments of the present application.

In a fifth aspect, an embodiment of the present application provides a computer product, wherein the above-mentioned computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the above-mentioned computer program is operable to cause a computer to execute operations as described in the embodiments of the present application. Part or all of the steps described in any method of the first aspect. The computer program product may be a software installation package.

It can be seen that through the above method, the first binary log in the first database is first processed in batches according to the preset batching rule to obtain N sub-binary logs, where N is a positive integer; then, in the preset of each sub-binary log Insert a data mark at the position to obtain a second binary log; finally, complete data synchronization between the first database and the second database according to the second binary log. The binary log can be processed in batches and marked, so that when data is synchronized between remote databases, it can automatically identify which data is generated by the system, avoiding the continuous circular replication of the binary log in the two databases, greatly improving data synchronization Effect.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a system architecture diagram of a data synchronization method provided by an embodiment of this application;

2 is a schematic flowchart of a data synchronization method provided by an embodiment of this application;

FIG. 3 is a schematic diagram of an application scenario of a data synchronization method provided by an embodiment of the application;

4A is a schematic diagram of a truncated transaction fragment provided by an embodiment of the application;

4B is a schematic diagram of another truncated transaction fragment provided by an embodiment of the application;

4C is a schematic diagram of another truncated transaction fragment provided by an embodiment of the application;

FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of this application;

6 is a block diagram of the functional unit composition of a data synchronization device provided by an embodiment of the application;

FIG. 7 is a block diagram of the functional unit composition of another data synchronization device provided by an embodiment of the application.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of this application, the technical solutions in the embodiments of this application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only These are a part of the embodiments of this application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific sequence. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes unlisted steps or units, or optionally also includes Other steps or units inherent in these processes, methods, products or equipment.

Reference to "embodiments" herein means that a specific feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art clearly and implicitly understand that the embodiments described herein can be combined with other embodiments.

For ease of understanding, the background technology and related terms involved in the embodiments of this application are first explained. The current remote database synchronization generally involves establishing independent data centers in different cities, and the databases of these data centers will work at the same time. All databases accept the writing of corresponding business data and synchronize and update data to other databases to achieve two-way data synchronization between databases with multiple activities in different places. The above databases can be relational database management systems, such as MySQL, and data update SQL Statement information will be stored in the binary log binlog. The above binlog is a binary log that records all database table structure changes (such as CREATE, ALTER TABLE...) and table data modifications (INSERT, UPDATE, DELETE...). It should be noted that binlog can represent a transaction, and a transaction is a sequence of database operations, which is the smallest unit of work in the database. A transaction consists of all database operations performed between the start of the transaction and the end of the transaction. How to solve the problem of data loopback and ensure the efficiency of the two-way synchronization during the two-way synchronization between multi-active databases in different places has become the focus of the embodiments of the present application.

The related background technology is described above, and the system architecture of the data synchronization method in the embodiment of the present application is introduced below.

As shown in FIG. 1, FIG. 1 is a system architecture diagram of a data synchronization method provided by an embodiment of this application. The system architecture 100 includes a first database 110, a second database 120, a replication module 130, a first terminal 141, and a second terminal 142. The above-mentioned first database 110 is connected to the second database 120 through the replication module 130, The first terminal 141 in the location of the first database 110 interacts with the first database 110, and the second terminal 142 in the location of the second database 120 interacts with the second database 120.

Wherein, the first database 110 and the second database 120 may be MySQL databases, and data is stored in the first database 110 and the second database 120 in the form of tables.

Wherein, the above-mentioned replication module 130 may be mounted on a server, and is used to read data from the above-mentioned first database 110 and synchronize to the above-mentioned second database 120, and at the same time, read data from the above-mentioned second database 120 and synchronize to the above-mentioned first database. 110. The replication module 130 may also process the read data in batches, so that the data synchronization between the first data 110 and the second database 120 is performed in batches.

The aforementioned user terminal 140 may be installed with various communication client applications, such as database management applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, and so on. The user terminal 140 may be various electronic devices with a display screen, including but not limited to smart phones, tablet computers, portable computers, desktop computers, and so on. There is no specific limitation here.

It should be noted that the first database 110 and the second database 120 in the foregoing system architecture only represent different databases, and it does not mean that the embodiment of the present application can only be applied to two databases. The same applies to the foregoing replication module 130 and the user terminal. There may be one or more than 140.

Through the above system architecture, binary logs can be processed in batches and marked, so that when data is synchronized between remote databases, it can automatically identify which data is generated by the system, and avoid the continuous circular replication of binary logs in the two databases. Greatly improve the effect of data synchronization.

The system architecture of a data synchronization method in an embodiment of the application is described above, and a data synchronization method in an embodiment of the application is described in detail below with reference to FIG. 2. FIG. 2 is a data synchronization method provided by an embodiment of the application. The schematic flow diagram of the synchronization method includes the following steps:

Step 201: Perform batch processing of the first binary log in the first database according to a preset batching rule to obtain N sub-binary logs.

Wherein, N is a positive integer, and the first binary log records the database operation sequence in the first database.

Specifically, the size of the first binary log is the first amount of data. The first amount of data can be set according to user needs. First, the first amount of data is read from the first database through the replication module in the system architecture. Then, the second data volume corresponding to the second database is obtained. Because there is two-way synchronization between the first database and the second database, the second data volume can be used for the subsequent first A binary log is processed in batches for data reference, and finally, the first two-level system log is processed in batches according to the first data volume and the second data volume to obtain N sub-binary logs.

Further, in conjunction with FIG. 3, how to perform batch processing on the first two-level system log to obtain N sub-binary logs according to the foregoing first data volume and the foregoing second data volume will be described in detail. FIG. 3 is provided for an embodiment of the application. An application scenario diagram of a data synchronization method, including a first database, a second database, and a replication (Repetition) module, a_select_size represents the first data volume, b_select_size represents the second data volume, and a_split_size represents the batch data volume, through the following pseudo The code can determine the size of a_split_size:

If(a_select_size>b_select_size){

a_split_size=b_select_size-2; // means a head tag and a tail tag

}Else{

a_split_size=b_select_size;

}

It can be seen that the replication module reads the first binary log of the first amount of data from the first database, performs batch processing on the first binary log, and then synchronizes to the second database, and reads from the second database at the same time. The binary log of the second amount of data is synchronized to the above-mentioned first database. It should be noted that FIG. 3 only shows the specific batch processing steps of synchronizing data from the above-mentioned first database to the above-mentioned second database. The above-mentioned first database synchronization data is also the same processing step, which will not be repeated here.

When the above-mentioned first data amount is greater than the above-mentioned second data amount, since the number of data tags in each batch is 2, that is, a head tag and a tail tag, it can be determined that the batch data amount is the second data amount minus 2 , That is, the size of each sub-binary log is the above-mentioned second data volume minus 2. For easy distinction, the batch data volume when the first data volume is greater than the second data volume is named the first batch data volume; When the data volume is less than or equal to the second data volume, the batch data volume can be determined to be the second data volume, that is, the size of each sub-binary log is the second data volume. In order to facilitate the distinction, the first data volume is smaller than The batch data volume when it is equal to the above-mentioned second data volume is named the second batch data volume.

By processing the first binary log in the first database in batches according to a preset batching rule to obtain N sub-binary logs, the size of one-time synchronization data can be reduced, the data volume pressure during synchronization is reduced, and the data volume pressure during synchronization is reduced. A deadlock situation occurs.

Step 202: Insert a data mark at a preset position of each sub-binary log to obtain a second binary log.

Wherein, the data mark includes a head mark and a tail mark. Inserting a data mark at a preset position means inserting the head mark at the head of each sub-binary log and inserting the tail mark at the end of each sub-binary log, inserting the data mark The size of each sub-binary log of is equal to the above-mentioned second data volume, so when the second database synchronizes data with the above-mentioned first database, at least one head mark or tail mark can be read.

Specifically, each of the above-mentioned first database and the above-mentioned second database has a formulation table. The binary log of the operation of generating data tags is stored in the formulation table. By operating the formulation table, a header can be inserted after each transaction header. Part mark, insert a tail mark before the end of each transaction.

The second binary log is obtained by inserting the data mark in the preset position of each sub-binary log, which can ensure the visibility of the data mark.

Step 203: Complete data synchronization between the first database and the second database according to the second binary log.

Wherein, the system input segment and service input segment of the second binary log may be determined according to the head tag and the tail tag, that is, the segment between the head tag and the tail tag is determined as the The system input fragments are determined to be the business input fragments other than the system input fragments. The system input fragments represent data generated by the system. Recognizing the system input fragments during synchronization can avoid the phenomenon of data loopback. The fragments represent the data entered into the database by the external terminal. Finally, the data synchronization between the first database and the second database is completed according to the system input fragment and the service input fragment.

Specifically, due to batch processing, the data may be truncated when the batch-processed data is written into the second database. For example, in conjunction with Figure 4A, Figure 4B and Figure 4C, Figure 4A is a kind of Schematic diagram of truncated transaction fragments, dml represents the operation sequence command for operating the database, transaction_begin represents the transaction head, transaction_end represents the transaction end, flag_begin represents the head tag, and flag_end represents the tail tag. In Figure 4A, only one head tag is read, so the head The dml sequence after the part mark belongs to the system input fragment, and the dml sequence before the head mark is input to the business input fragment; Figure 4B only reads a tail mark, so the dml sequence before the tail mark belongs to the system input fragment, after the tail mark The dml sequence belongs to the business input segment; Figure 4C reads the complete head and tail tags, so the dml sequence between the head tag and the tail tag belongs to the system input segment. It should be noted that the foregoing is only illustrative of possible truncation situations, and does not represent a limitation to the embodiments of the present application.

By completing the data synchronization between the first database and the second database according to the second binary log, the binary log can be processed in batches and marked, reducing the data volume pressure of data synchronization, and making it remote When synchronizing data between databases, it automatically recognizes which data is generated by the system, avoids the continuous circular replication of binary logs in the two databases, and greatly improves the effect of data synchronization.

The foregoing mainly introduces the solution of the embodiment of the present application from the perspective of the execution process on the method side. It can be understood that the above methods can be implemented independently or in combination. In order to implement the above functions, the electronic device includes hardware structures and/or software modules corresponding to each function. Those skilled in the art should easily realize that in combination with the units and algorithm steps of the examples described in the embodiments provided herein, this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed by hardware or computer software-driven hardware depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

The following describes the hardware devices that apply the data synchronization method in the embodiments of the present application.

FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of its own application. The electronic device 500 may be an electronic device with communication capabilities. The electronic device may include various handheld devices with wireless communication functions, vehicle-mounted devices, and wearables. Equipment, computing equipment or other processing equipment connected to a wireless modem, as well as various forms of user equipment (User Equipment, UE), mobile station (Mobile Station, MS), terminal equipment (terminal device), and so on. The electronic device 500 in this application may include one or more of the following components: a processor 510, a memory 520, and an input/output device 530.

The processor 510 may include one or more processing cores. The processor 510 uses a communication interface to connect various parts of the entire terminal 500, and executes the operations of the electronic device 500 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 520, and calling data stored in the memory 520. Various functions and processing data. The processor 510 may include one or more processing units. For example, the processor 510 may include a central processing unit (CPU), an application processor (AP), a modem processor, and a graphics processor ( graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. The CPU is mainly used to process the operating system, user interface, and application programs; the GPU is used to render and draw the display content; the modem is used to process wireless communication. Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. In the embodiment of this application, the central processing unit CPU is taken as an example for description. For example, when the electronic device 500 is running an application, the CPU can obtain drawing data of the UI thread and other threads for the application, and determine the current drawing data based on the drawing data. Whether the running application program has dropped frames, etc., after that, based on the performance usage of the CPU itself, the frequency of the CPU can be dynamically controlled.

The processor 510 may be provided with a memory 520 for storing instructions and data. In some embodiments, the memory 520 in the processor 510 is a cache memory. The memory 520 can store instructions or data that the processor 510 has just used or used cyclically. If the processor 510 needs to use the instruction or data again, it can be directly called from the memory 520. Avoid repeated accesses, reduce the waiting time of the processor 510, and improve system efficiency.

It is understandable that the above-mentioned processor 510 can be mapped to a system-on-chip (SOC) in an actual product, and the above-mentioned processing unit and/or interface may not be integrated into the processor 510 and pass through a communication chip alone. Or electronic components realize the corresponding function. The above-mentioned interface connection relationship between the modules is only a schematic description, and does not constitute the only limitation on the structure of the electronic device 500.

The memory 520 may include random access memory (Random Access Memory, RAM), and may also include read-only memory (Read-Only Memory). Optionally, the memory 520 includes a non-transitory computer-readable storage medium. The memory 520 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 520 may include a storage program area and a storage data area, where the storage program area may store instructions for implementing the operating system and instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.) , Instructions used to implement the following various method embodiments, etc., the operating system can be the Android system (including the system based on the in-depth development of the Android system), the IOS system developed by Apple (including the system based on the in-depth development of the IOS system) ) Or other systems. The data storage area can also store data created by the electronic device 500 during use.

The input and output device 530 may include a touch display screen, which is used to receive a user's touch operation on or near any suitable object such as a finger, a touch pen, etc., and to display a user interface of each application program. The touch screen is usually arranged on the front panel of the terminal 500. The touch screen can be designed as a full screen, curved screen or special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiment of the present application.

The embodiment of the present application may divide the electronic device into functional units according to the foregoing method examples. For example, each functional unit may be divided corresponding to each function, or two or more functions may be integrated into one processing unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit. It should be noted that the division of units in the embodiments of the present application is illustrative, and is only a logical function division, and there may be other division methods in actual implementation.

In the case of dividing each functional module corresponding to each function, FIG. 6 is a block diagram of the functional unit composition of a data synchronization device provided in an embodiment of the application. As shown in FIG. 6, the data synchronization device 600 includes:

The batching unit 610 is configured to perform batch processing of the first binary log in the first database according to a preset batching rule to obtain N sub-binary logs, where N is a positive integer;

The marking unit 620 is configured to insert a data mark at the preset position of each sub-binary log to obtain the second binary log;

The synchronization unit 630 is configured to complete data synchronization between the first database and the second database according to the second binary log.

Among them, all relevant content of the steps involved in the foregoing method embodiments can be cited in the functional description of the corresponding functional module, and will not be repeated here.

In the case of using an integrated unit, FIG. 7 is a functional unit composition block diagram of another central processor frequency control device 700 involved in an embodiment of the present application. The device 700 for controlling the frequency of the central processing unit is applied to electronic equipment that supports display functions. The electronic device includes a processor, a display module, etc., and the device for controlling the frequency of the central processing unit 700 includes a processing unit 701 and a communication unit 702, wherein , The processing unit 701 is configured to perform any step in the above method embodiment, and when performing data transmission such as sending, it can optionally call the communication unit 702 to complete the corresponding operation.

Wherein, the device 700 for controlling the frequency of the central processing unit may further include a storage unit 703 for storing program codes and data of the electronic device. The processing unit 701 may be a central processing unit, the communication unit 702 may be a touch screen or a transceiver, and the storage unit 703 may be a memory.

It is understandable that because the method embodiment and the device embodiment are different presentation forms of the same technical concept, the content of the method embodiment part of this application should be synchronized to the device embodiment part, and will not be repeated here. Both the data synchronization device 600 and the data synchronization device 700 described above can execute all the data synchronization methods included in the foregoing embodiment.

An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any method as described in the above method embodiment , The above-mentioned computer includes electronic equipment.

The embodiments of the present application also provide a computer program product. The above-mentioned computer program product includes a non-transitory computer-readable storage medium storing a computer program. The above-mentioned computer program is operable to cause a computer to execute any of the methods described in the above-mentioned method embodiments. Part or all of the steps of the method. The computer program product may be a software installation package, and the above-mentioned computer includes electronic equipment.

It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that this application is not limited by the described sequence of actions. Because according to this application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

In the above-mentioned embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed device may be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of the above-mentioned units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or integrated. To another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical or other forms.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the aforementioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable memory. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, A number of instructions are included to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the foregoing methods of the various embodiments of the present application. The aforementioned memory includes: U disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable memory, and the memory can include: a flash disk , Read-only memory (English: Read-Only Memory, abbreviation: ROM), random access device (English: Random Access Memory, abbreviation: RAM), magnetic disk or optical disc, etc.

The embodiments of the application are described in detail above, and specific examples are used in this article to illustrate the principles and implementation of the application. The descriptions of the above examples are only used to help understand the methods and core ideas of the application; at the same time, for Persons of ordinary skill in the art, based on the idea of the application, will have changes in the specific implementation and the scope of application. In summary, the content of this specification should not be construed as limiting the application.

Claims

A data synchronization method, characterized in that the method includes:

The first binary log in the first database is processed in batches according to a preset batching rule to obtain N sub-binary logs, where N is a positive integer;

Insert a data mark at the preset position of each sub-binary log to obtain the second binary log;

The data synchronization between the first database and the second database is completed according to the second binary log.
The method according to claim 1, wherein the step of processing the first binary log in the first database in batches according to a preset batching rule to obtain N sub-binary logs comprises:

Read the first binary log of a first amount of data from the first database;

Acquiring a second amount of data corresponding to the second database, where the second amount of data represents the size of the synchronization data of the second database with the first database;

Perform batch processing on the first two-level system log according to the first data volume and the second data volume to obtain N sub-binary logs.
The method according to claim 2, wherein the batch processing of the first two-level system log according to the first data volume and the second data volume to obtain N sub-binary logs comprises:

When the first data volume is greater than the second data volume, the first batch data volume is determined according to the second data volume and the data mark, and the first batch data volume represents the amount of each sub-binary log size;

The first binary log is processed in batches according to the amount of the first batch of data to obtain the N sub-binary logs.
The method according to claim 2, wherein the batch processing of the first two-level system log according to the first data volume and the second data volume to obtain N sub-binary logs comprises:

When the first data amount is less than or equal to the second data amount, determining the second data amount as a second batch data amount;

The first binary log is processed in batches according to the second batch data amount to obtain the N sub-binary logs.
The method according to claim 1, wherein the inserting a data mark in a preset position of each sub-binary log to obtain the second binary log comprises:

Inserting a head mark at the head of each sub-binary log, and inserting a tail mark at the end of each sub-binary log to obtain the second binary log.
The method according to claim 5, wherein the completing data synchronization between the first database and the second database according to the second binary log comprises:

Determining a system input segment and a business input segment of the second binary log according to the head tag and the tail tag;

Complete data synchronization between the first database and the second database according to the system input fragment and the service input fragment.
The method according to claim 6, wherein the determining the system input segment and the service input segment of the second binary log according to the head tag and the tail tag comprises:

Determining the segment between the head tag and the tail tag as the system input segment;

A segment other than the system input segment is determined as the service input segment.
A data synchronization device, characterized in that the device includes:

The batching unit is used to process the first binary log in the first database in batches according to a preset batching rule to obtain N sub-binary logs, where N is a positive integer;

The marking unit is used to insert a data mark at the preset position of each sub-binary log to obtain the second binary log;

The synchronization unit is configured to complete data synchronization between the first database and the second database according to the second binary log.
An electronic device, characterized by comprising an application processor, a memory, and one or more programs, the one or more programs are stored in the memory and configured to be executed by the application processor, so The program includes instructions for executing the steps in the method according to any one of claims 1-7.
A computer storage medium, wherein the computer storage medium stores a computer program, the computer program includes program instructions, and when executed by a processor, the program instructions cause the processor to execute as claimed in claims 1 to 7. Any one of the methods.