CN115599797A - Bidirectional synchronization method and device based on operation log and electronic equipment - Google Patents
Bidirectional synchronization method and device based on operation log and electronic equipment Download PDFInfo
- Publication number
- CN115599797A CN115599797A CN202211502437.6A CN202211502437A CN115599797A CN 115599797 A CN115599797 A CN 115599797A CN 202211502437 A CN202211502437 A CN 202211502437A CN 115599797 A CN115599797 A CN 115599797A
- Authority
- CN
- China
- Prior art keywords
- data
- databases
- synchronization
- bidirectional synchronization
- bidirectional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002457 bidirectional effect Effects 0.000 title claims abstract description 117
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000012544 monitoring process Methods 0.000 claims abstract description 18
- 238000004806 packaging method and process Methods 0.000 claims abstract description 6
- 238000012545 processing Methods 0.000 claims description 10
- 238000012217 deletion Methods 0.000 claims description 4
- 230000037430 deletion Effects 0.000 claims description 4
- 230000001360 synchronised effect Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000002085 persistent effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000002355 dual-layer Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses a bidirectional synchronization method and device based on an operation log and an electronic device, which can sense data updating in time based on the operation log of a database and perform bidirectional data synchronization, improve data synchronization efficiency and keep data consistency of cloud data and local data. The bidirectional synchronization method comprises the following steps: monitoring operation logs of a cloud database and an operation log of a local database; when data updating occurs in any one of the two databases, determining data information related to the data updating according to the corresponding operation log, packaging the data information into a queue message, and sending the queue message to an application program, wherein the application program is a program of a consumer serving as the queue message; and performing bidirectional synchronization on the two databases by using the data information according to a bidirectional synchronization rule in the databases through an application program, wherein the bidirectional synchronization rule in the databases is preset.
Description
Technical Field
The present application relates to the field of database technologies, and in particular, to a bidirectional synchronization method and apparatus based on an operation log, and an electronic device.
Background
Read-write separation is a common technique for solving the performance of relational databases. The application needing to read and write accesses the master library, and the application needing to read only accesses the slave library. The master library and the slave libraries are synchronized on data through data replication, and the data consistency of the master library and the slave library is ensured. Here, data synchronization is a key technology for read-write separation.
At present, a technical scheme for performing bidirectional synchronization on a cloud and a local database does not exist in a related data synchronization technology, and when the bidirectional synchronization scheme can ensure that the cloud and the local database can keep synchronous in real time, the bidirectional synchronization method is of great significance to system double activities.
Disclosure of Invention
Based on this, in order to solve the above problems, the application provides a bidirectional synchronization method and apparatus based on an operation log, and an electronic device, which can sense data update in time based on the operation log of a database and perform bidirectional data synchronization, thereby improving data synchronization efficiency and maintaining data consistency of cloud data and local data.
In a first aspect, the present application provides a bidirectional synchronization method based on an operation log, including:
monitoring operation logs of a cloud database and a local database;
when data updating occurs in any one of the two databases, determining data information related to the data updating according to the corresponding operation log, packaging the data information into a queue message, and sending the queue message to an application program, wherein the application program is a program of a consumer serving as the queue message;
and performing bidirectional synchronization on the two databases by using the data information according to a bidirectional synchronization rule in the databases through an application program, wherein the bidirectional synchronization rule in the databases is preset.
Optionally, in an implementation manner of the first aspect, the data information includes rule-class data and synchronization-class data, the rule-class data includes a bidirectional synchronization rule, and the synchronization-class data includes target data that needs bidirectional synchronization; the method further comprises the following steps:
updating the bidirectional synchronization rule in the database by using the bidirectional synchronization rule in the data information;
according to the bidirectional synchronization rule in the database, the two databases are synchronized bidirectionally by using the data information, which comprises the following steps:
and performing bidirectional synchronization on the two databases by using the target data according to a bidirectional synchronization rule in the data information.
Optionally, in an implementation manner of the first aspect, the data information is a database table, and before performing bidirectional synchronization on the two databases using the data information, the method further includes:
when an application program is started, acquiring a bidirectional synchronization rule in a database;
when a database table exists and the unique key of the database table is correct, an operation of bi-directionally synchronizing the two databases is performed.
Optionally, in an implementation manner of the first aspect, the queue message is a rabbitmq message, and a message type of the rabbitmq message includes: input, update, and delete.
Further optionally, when the data in the rabbitmq message is an input type or an updated type, performing bidirectional synchronization on the two databases by using the data information according to a bidirectional synchronization rule in the databases, including:
for any one of the two databases, judging whether data in the rabbitmq message exists in the database;
if not, performing input operation to input the data in the rabbitmq message into the database;
if so, an update operation is performed to update the data in the rabbitmq message to the database.
Further optionally, when the data in the rabbitmq message is a deletion type, performing bidirectional synchronization on the two databases by using the data information according to a bidirectional synchronization rule in the databases, where the bidirectional synchronization includes:
for any one of the two databases, judging whether data in the rabbitmq message exists in the database;
if so, a delete operation is performed to delete the data in the rabbitmq message from the database.
Optionally, in an implementation manner of the first aspect, the method further includes:
when the bidirectional synchronization is completed, acquiring corresponding synchronization duration;
and if the synchronization time length is greater than or equal to the preset synchronization threshold value, sending out alarm information in a WeChat mode.
In a second aspect, the present application provides an operation log-based bidirectional synchronization apparatus, including:
the device comprises a monitoring module, a processing module and an updating module;
the monitoring module is used for: monitoring operation logs of a cloud database and a local database;
the processing module is used for: when data updating occurs in any one of the two databases, determining data information related to the data updating according to the corresponding operation log, packaging the data information into a queue message, and sending the queue message to an application program, wherein the application program is a program of a consumer serving as the queue message;
the update module is to: and performing bidirectional synchronization on the two databases by using the data information according to bidirectional synchronization rules in the databases, wherein the bidirectional synchronization rules in the databases are preset.
In a third aspect, the present application provides an electronic device, comprising:
a processor and a memory having executable code stored thereon;
when the executable code is invoked by the processor, the electronic device is caused to perform the steps of the oplog based bi-directional synchronization method as described in any of the first aspect and its implementations.
In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon executable code, which when called by a processor of an electronic device, causes the electronic device to perform the steps in the oplog-based bidirectional synchronization method as described in any one of the first aspect and its implementation.
The technical scheme provided by the application has the following beneficial effects:
according to the technical scheme, by monitoring the operation logs of the cloud database and the local database, when data updating occurs in the cloud database or any local database, the data information related to the data updating is determined through the corresponding operation logs, the data information is packaged into queue messages and sent to an application program, the application program uses the data information to carry out bidirectional synchronization on the cloud database and the local database, the data updating can be timely sensed based on the operation logs of the databases, data in the cloud database and the local database are guaranteed to be consistent in real time, the data synchronization efficiency is improved, and the data consistency of the cloud data and the local data is kept.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The foregoing and other objects, features and advantages of the application will be apparent from the following more particular descriptions of exemplary embodiments of the application, as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts throughout the exemplary embodiments of the application.
FIG. 1 is a schematic flow chart of a bidirectional synchronization method based on an operation log according to an embodiment of the present application;
FIG. 2 is a schematic structural diagram of an oplog-based bidirectional synchronization apparatus in an embodiment of the present application;
fig. 3 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While embodiments of the present application are illustrated in the accompanying drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms "first," "second," "third," etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
The embodiment of the application provides a bidirectional synchronization method and device based on an operation log and an electronic device, the operation log based on a database can timely sense data updating and perform bidirectional data synchronization, the data synchronization efficiency is improved, and the data consistency of cloud data and local data is maintained.
In order to facilitate understanding of the bidirectional synchronization method based on the operation log in the embodiment of the present application, a specific technical solution is described below with reference to the accompanying drawings, which are specifically as follows:
fig. 1 is a schematic flowchart of a bidirectional synchronization method based on an operation log in an embodiment of the present application.
101. And monitoring operation logs of the cloud database and the local database.
In the embodiment of the application, the cloud database is a database deployed on a cloud platform, for example, a cloud database, and the local database is a database deployed in a local machine room by an enterprise or a utility.
Accordingly, oplogs are used to record data updates or potential updates in the database, for example, mysql bin log is one of the oplogs, and binlog is a binary log file.
The monitoring mode may be real-time monitoring or periodic monitoring, which is not limited in this application.
102. And when data updating occurs in any one of the two databases, determining data information related to the data updating according to the corresponding operation log.
In the embodiment of the application, since the operation log can record the data update or the potential update, the data information related to the data update can be acquired according to the record of the operation log. An application refers to a program that is a consumer of a queued message.
Optionally, in an implementation manner of the embodiment of the present application, data in the data information may be divided into rule-class data and synchronization-class data, where the rule-class data represents a bidirectional synchronization rule, and the synchronization-class data includes target data that needs bidirectional synchronization.
Optionally, the data information may be presented in a form of a table, and specifically, the data information is a database table.
It should be noted that bidirectional synchronization refers to bidirectional real-time data synchronization between two databases, for example, bidirectional data synchronization between two MySQL databases, such as bidirectional data real-time synchronization between RDS MySQL and self-built MySQL.
103. And encapsulating the data information into a queue message and sending the queue message to the application program.
In this embodiment, the queue message is also referred to as a Message Queue (MQ), specifically, the message queue may be a rabbitmq message, the application program is a program that is a consumer of the rabbitmq message, and the message queue may also be other messages besides the rabbitmq message, which is not limited in this application.
Further, the message types of the rabbitmq message include: input, update, and delete.
Further optionally, when the data in the rabbitmq message is an input type or an update type, the two databases are bi-directionally synchronized by using the data information according to a bi-directional synchronization rule in the databases, which includes the following steps:
for any one of the two databases, judging whether data in the rabbitmq message exists in the database;
if not, performing input operation to input the data in the rabbitmq message into the database;
if so, an update operation is performed to update the data in the rabbitmq message to the database.
Further optionally, when the data in the rabbitmq message is a deletion type, performing bidirectional synchronization on the two databases by using the data information according to a bidirectional synchronization rule in the databases, including the following steps:
for any one of the two databases, judging whether data in the rabbitmq message exists in the database;
if so, a delete operation is performed to delete the data in the rabbitmq message from the database.
104. And performing bidirectional synchronization on the two databases by using the data information according to the bidirectional synchronization rule in the databases through the application program.
In the embodiment of the present application, the bidirectional synchronization rule in the database is pre-established and is used as the initialized bidirectional synchronization rule.
The data information comprises rule data and synchronization data, and the following steps can be specifically executed to realize bidirectional synchronization of the cloud database and the local database: firstly, updating a bidirectional synchronization rule in a database by using the bidirectional synchronization rule in the data information; secondly, the target data is synchronized to the two databases in a bidirectional mode according to a bidirectional synchronization rule in the data information.
Further, if the data information is a database table, before performing bidirectional synchronization on the two databases using the data information, the method further includes: when an application program is started, acquiring a bidirectional synchronization rule in a database; when the database table exists and the unique key of the database table is correct, the operation of performing bidirectional synchronization on the two databases is executed.
Optionally, in some implementation manners of the embodiment of the present application, the scheme may further provide data synchronization duration monitoring, delay a certain time for the system to perform a WeChat alarm, and enable the operation and maintenance staff to perform access processing. Specifically, the bidirectional synchronization method further includes the following steps:
when the bidirectional synchronization is completed, acquiring corresponding synchronization duration;
and if the synchronization time length is greater than or equal to the preset synchronization threshold value, sending alarm information in a WeChat mode.
In the technical scheme of the embodiment of the application, by monitoring the operation logs of the cloud and the local databases, when data updating occurs in the cloud or any local database, data information related to the data updating is determined through the corresponding operation logs, the data information is packaged into queue messages and sent to an application program, then the application program uses the data information to carry out bidirectional synchronization on the cloud and the local databases, the data updating can be timely sensed based on the operation logs of the databases, data in the cloud and the local databases are ensured to be consistent in real time, the data synchronization efficiency is improved, and the data consistency of the cloud data and the local data is maintained.
Corresponding to the embodiment of the application function implementation method, the application also provides a bidirectional synchronization device based on the operation log, an electronic device and a corresponding embodiment.
Fig. 2 is a schematic structural diagram of an operation log-based bidirectional synchronization apparatus in an embodiment of the present application.
As shown in fig. 2, the bidirectional synchronization apparatus 20 based on the operation log in the embodiment of the present application includes:
a monitoring module 201, a processing module 202 and an updating module 203;
the listening module 201 is configured to: monitoring operation logs of a cloud database and a local database;
the processing module 202 is configured to: when data updating occurs in any one of the two databases, determining data information related to the data updating according to the corresponding operation log, packaging the data information into a queue message, and sending the queue message to an application program, wherein the application program is a program of a consumer serving as the queue message;
the update module 203 is configured to: and performing bidirectional synchronization on the two databases by using the data information according to bidirectional synchronization rules in the databases, wherein the bidirectional synchronization rules in the databases are preset.
Optionally, in an implementation manner of the embodiment of the present application, the data information includes rule-class data and synchronization-class data, the rule-class data includes a bidirectional synchronization rule, and the synchronization-class data includes target data that needs bidirectional synchronization; the update module 203 is further configured to: updating the bidirectional synchronization rule in the database by using the bidirectional synchronization rule in the data information; furthermore, the update module 203 performs the following operations for bidirectional synchronization: the updating module 203 performs bidirectional synchronization on the two databases by the target data according to the bidirectional synchronization rule in the data information.
Optionally, in an implementation manner of the embodiment of the present application, the data information is a database table, and before the updating module 203 performs bidirectional synchronization on the two databases using the data information, the updating module 203 is further configured to perform the following steps: when the application program is started, the updating module 203 acquires the bidirectional synchronization rule in the database; when a database table exists and the unique key of the database table is correct, the update module 203 performs an operation of bi-directionally synchronizing the two databases.
Optionally, in an implementation manner of the embodiment of the present application, the queue message used for sending in the processing module 202 is an rabbitmq message, and a message type of the rabbitmq message includes: input, update, and delete.
Optionally, in an implementation manner of the embodiment of the present application, when data in the rabbitmq message is an input type or an update type, the update module 203 performs the following operation specifically by performing bidirectional synchronization on the two databases using the data information according to a bidirectional synchronization rule in the databases: for any one of the two databases, the updating module 203 judges whether data in the rabbitmq message exists in the database; if not, the updating module 203 executes input operation to input the data in the rabbitmq message into the database; if so, the update module 203 performs an update operation to update the data in the rabbitmq message to the database.
Optionally, in an implementation manner of the embodiment of the present application, when data in the rabbitmq message is a deletion type, the updating module 203 specifically performs the following steps to perform bidirectional synchronization on the two databases by using the data information according to a bidirectional synchronization rule in the databases: for any one of the two databases, the updating module 203 judges whether data in the rabbitmq message exists in the database; if so, the update module 203 performs a delete operation to delete the data in the rabbitmq message from the database.
Optionally, in an implementation manner of the embodiment of the present application, the updating module 203 is further configured to perform the following operations for alarming: after the bidirectional synchronization is completed, the updating module 203 acquires a corresponding synchronization duration; if the synchronization duration is greater than or equal to the preset synchronization threshold, the update module 203 sends the alarm information in a WeChat mode.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs operations and the advantages thereof have been described in detail in the embodiment related to the method, and will not be elaborated upon herein.
Fig. 3 is a schematic structural diagram of an electronic device in an embodiment of the present application.
As shown in fig. 3, the electronic device 30 in the embodiment of the present application includes a memory 301 and a processor 302. The memory has stored thereon executable code that, when executed by the processor, causes the processor to perform the method of any of the embodiments described above.
The Processor 302 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 301 may include various types of storage units such as system memory, read Only Memory (ROM), and permanent storage. Wherein the ROM may store static data or instructions for the processor 302 or other modules of the computer. The persistent storage device may be a read-write storage device. The persistent storage may be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered down. In some embodiments, the persistent storage device employs a mass storage device (e.g., magnetic or optical disk, flash memory) as the persistent storage device. In other embodiments, the permanent storage may be a removable storage device (e.g., floppy disk, optical drive). The system memory may be a read-write memory device or a volatile read-write memory device, such as a dynamic random access memory. The system memory may store instructions and data that some or all of the processors require at run-time. In addition, the memory 301 may comprise any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic and/or optical disks, may also be employed. In some embodiments, memory 301 may include a removable storage device that is readable and/or writable, such as a Compact Disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, dual layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, min SD card, micro-SD card, etc.), a magnetic floppy disc, or the like. Computer-readable storage media do not contain carrier waves or transitory electronic signals transmitted by wireless or wired means.
The memory 301 has stored thereon executable code that, when processed by the processor 302, may cause the processor 302 to perform some or all of the methods described above.
Furthermore, the method according to the present application may also be implemented as a computer program or computer program product comprising computer program code instructions for performing some or all of the steps of the above-described method of the present application.
Alternatively, the present application may also be embodied as a computer-readable storage medium (or machine-readable storage medium) having stored thereon executable code (or a computer program, or computer instruction code) which, when executed by a processor of an electronic device (or an electronic device, a server, etc.), causes the processor to perform part or all of the steps of the above-described method according to the present application.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should be further noted that, in this document, relationships such as first and second, etc., are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Also, the terms include, or any other variation is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements does not include only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application.
Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (10)
1. A bidirectional synchronization method based on operation logs is characterized by comprising the following steps:
monitoring operation logs of a cloud database and an operation log of a local database;
when data updating occurs in any one of the two databases, determining data information related to the data updating according to the corresponding operation log, packaging the data information into a queue message, and sending the queue message to an application program, wherein the application program is a program of a consumer serving as the queue message;
and performing bidirectional synchronization on the two databases by using the data information according to bidirectional synchronization rules in the databases through the application program, wherein the bidirectional synchronization rules in the databases are preset.
2. The bidirectional synchronization method of claim 1, wherein the data information includes rule class data and synchronization class data, the rule class data including bidirectional synchronization rules, the synchronization class data including target data requiring bidirectional synchronization; the method further comprises the following steps:
updating the bidirectional synchronization rule in the database by using the bidirectional synchronization rule in the data information;
the bidirectional synchronization of the two databases by using the data information according to the bidirectional synchronization rule in the databases comprises:
and performing bidirectional synchronization on the two databases by the target data according to a bidirectional synchronization rule in the data information.
3. The bidirectional synchronization method according to claim 1 or 2, wherein the data information is a database table, and before the bidirectional synchronization of the two databases using the data information, the method further comprises:
when the application program is started, acquiring a bidirectional synchronization rule in the database;
when the database table exists and the unique key of the database table is correct, performing the operation of performing bidirectional synchronization on the two databases.
4. The bidirectional synchronization method of claim 1, wherein the queue message is a rabbitmq message, and a message type of the rabbitmq message comprises: input, update, and delete.
5. The bidirectional synchronization method of claim 4, wherein when the data in the rabbitmq message is input or updated, the bidirectional synchronization of the two databases using the data information according to the bidirectional synchronization rule in the databases comprises:
for any one of the two databases, judging whether the database has data in the rabbitmq message;
if the rabbitmq message does not exist, performing input operation to input the data in the rabbitmq message into a database;
and if so, executing an updating operation to update the data in the rabbitmq message to a database.
6. The bidirectional synchronization method of claim 4, wherein when the data in the rabbitmq message is deleted, the bidirectional synchronization of the two databases using the data information according to the bidirectional synchronization rule in the databases comprises:
for any one of the two databases, judging whether the database has data in the rabbitmq message;
and if so, executing a deletion operation to delete the data in the rabbitmq message from the database.
7. The bi-directional synchronization method of claim 1, further comprising:
when the bidirectional synchronization is completed, acquiring corresponding synchronization duration;
and if the synchronization time length is greater than or equal to a preset synchronization threshold value, sending out alarm information in a WeChat mode.
8. An operation log based bidirectional synchronization apparatus, comprising:
the device comprises a monitoring module, a processing module and an updating module;
the monitoring module is used for: monitoring operation logs of a cloud database and a local database;
the processing module is used for: when data updating occurs in any one of the two databases, determining data information related to the data updating according to the corresponding operation log, packaging the data information into a queue message, and sending the queue message to an application program, wherein the application program is a program of a consumer serving as the queue message;
the update module is to: and performing bidirectional synchronization on the two databases by using the data information according to bidirectional synchronization rules in the databases, wherein the bidirectional synchronization rules in the databases are preset.
9. An electronic device, comprising:
a processor and a memory, the memory having executable code stored thereon;
the executable code, when invoked by the processor, causes the electronic device to perform the steps in the oplog-based two-way synchronization method as claimed in any one of claims 1-7.
10. A computer-readable storage medium having stored thereon executable code which, when invoked by a processor of an electronic device, causes the electronic device to perform the steps in the oplog-based bi-directional synchronization method as claimed in any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211502437.6A CN115599797A (en) | 2022-11-29 | 2022-11-29 | Bidirectional synchronization method and device based on operation log and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211502437.6A CN115599797A (en) | 2022-11-29 | 2022-11-29 | Bidirectional synchronization method and device based on operation log and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115599797A true CN115599797A (en) | 2023-01-13 |
Family
ID=84852262
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211502437.6A Pending CN115599797A (en) | 2022-11-29 | 2022-11-29 | Bidirectional synchronization method and device based on operation log and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115599797A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6779002B1 (en) * | 2000-06-13 | 2004-08-17 | Sprint Communications Company L.P. | Computer software framework and method for synchronizing data across multiple databases |
| US20200409977A1 (en) * | 2017-09-08 | 2020-12-31 | Guangdong Construction Information Center | Generic Multi-Source Heterogeneous Large-Scale Data Synchronization Client-Server Method |
| CN112182001A (en) * | 2020-09-27 | 2021-01-05 | 浪潮云信息技术股份公司 | Method, apparatus and medium for incremental synchronization of database to dynamic ES index library |
| CN112270600A (en) * | 2020-10-29 | 2021-01-26 | 广东通莞科技股份有限公司 | Multi-source data processing method, system and related device |
| CN113407638A (en) * | 2021-07-16 | 2021-09-17 | 上海通联金融服务有限公司 | Method for realizing real-time relational database data synchronization |
| CN115309827A (en) * | 2022-08-30 | 2022-11-08 | 成都中科大旗软件股份有限公司 | Data differentiation synchronization method, system, device and medium |
-
2022
- 2022-11-29 CN CN202211502437.6A patent/CN115599797A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6779002B1 (en) * | 2000-06-13 | 2004-08-17 | Sprint Communications Company L.P. | Computer software framework and method for synchronizing data across multiple databases |
| US20200409977A1 (en) * | 2017-09-08 | 2020-12-31 | Guangdong Construction Information Center | Generic Multi-Source Heterogeneous Large-Scale Data Synchronization Client-Server Method |
| CN112182001A (en) * | 2020-09-27 | 2021-01-05 | 浪潮云信息技术股份公司 | Method, apparatus and medium for incremental synchronization of database to dynamic ES index library |
| CN112270600A (en) * | 2020-10-29 | 2021-01-26 | 广东通莞科技股份有限公司 | Multi-source data processing method, system and related device |
| CN113407638A (en) * | 2021-07-16 | 2021-09-17 | 上海通联金融服务有限公司 | Method for realizing real-time relational database data synchronization |
| CN115309827A (en) * | 2022-08-30 | 2022-11-08 | 成都中科大旗软件股份有限公司 | Data differentiation synchronization method, system, device and medium |
Non-Patent Citations (2)
| Title |
|---|
| 郑伟等: "一种高性能的多级异构电能质量数据库同步方法", 《电力科学与工程》 * |
| 马越等: "WebSphere MQ在零售业系统数据传输中的应用", 《上海师范大学学报(自然科学版)》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11256715B2 (en) | Data backup method and apparatus | |
| US10496685B2 (en) | Redis key management method and system | |
| EP3474516B1 (en) | Data processing method and device | |
| US20180181751A1 (en) | Anomaly Detection in Distributed Ledger Systems | |
| CN106201771B (en) | Data-storage system and data read-write method | |
| EP3258396A1 (en) | Data synchronization method, device and system | |
| CN106933823B (en) | Data synchronization method and device | |
| EP3944556B1 (en) | Block data storage method and apparatus, and block data access method and apparatus | |
| CN109344316B (en) | News popularity calculation method and device | |
| CN105824846B (en) | Data migration method and device | |
| US10609141B2 (en) | Methods and apparatuses for cluster switching | |
| WO2019037617A1 (en) | Data transaction processing method, device, and electronic device | |
| CN106897342B (en) | Data verification method and equipment | |
| JP6987340B2 (en) | Database data change request processing method and equipment | |
| CN110442648A (en) | Method of data synchronization and device | |
| CN109684410A (en) | A kind of system, method and the storage medium of determining master-slave database synchronization delayed time | |
| CN111459948A (en) | Data block deleting method based on centralized block chain type account book | |
| CN110895545B (en) | Shared data synchronization method and device | |
| CN110121694A (en) | A kind of blog management method, server and Database Systems | |
| CN109471901B (en) | Data synchronization method and device | |
| CN115599797A (en) | Bidirectional synchronization method and device based on operation log and electronic equipment | |
| CN107111532A (en) | Calculate it is uncertain in the case of recovery and fault-tolerant | |
| CN113434476B (en) | Data synchronization method, device, equipment, system, storage medium and program product | |
| TWI754837B (en) | Method, apparatus, device and computer-readable medium for database migration | |
| CN111177077B (en) | Data storage and data query method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230113 |