CN112883118A - Method and system for synchronously acquiring incremental data based on sql - Google Patents
Method and system for synchronously acquiring incremental data based on sql Download PDFInfo
- Publication number
- CN112883118A CN112883118A CN202110345061.1A CN202110345061A CN112883118A CN 112883118 A CN112883118 A CN 112883118A CN 202110345061 A CN202110345061 A CN 202110345061A CN 112883118 A CN112883118 A CN 112883118A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- database
- logic
- acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
Abstract
The invention discloses a method and a system for synchronously acquiring incremental data based on sql, belonging to the field of database full and incremental data synchronization; the method comprises the following specific steps: s1 creating a logic copy slot at the database acquisition end through the execution database sql; s2 acquiring the increment data of the table structure and logic decoding acquired by the program in real time; s3, carrying out secondary analysis and serialization on the character string data set subjected to logic decoding, and then transmitting through cmsp; s4, the warehouse entry end acquires data according to the deserialization of the database type and writes the data into the database; s5, when the collection requirement is finished, the created logic copy slot is cleared; s6 obtaining incremental data of the database; the method can not acquire the condition of table structure change through a timestamp or increment field mode, and improves the accuracy and efficiency of analysis by utilizing database analysis; and sending the acquired data to a destination terminal, starting corresponding conversion by the destination terminal according to the type of the database, and writing the incremental data into the destination database in real time by the warehousing module.
Description
Technical Field
The invention discloses an increment data synchronous acquisition method and system based on sql, and relates to the technical field of database full and increment data synchronization.
Background
The postgreSQL provides the functions of logic copy and logic decoding from the 9.4+ version, the logic copy can realize the data synchronization between the main and standby, the logic decoding is the analysis of the structure set of incremental data realized by the built-in plug-in, and the plug-in test _ decoding exemplifies the result of the incremental acquisition and feeds back the result to the user in the form of a character string representation result set; the CMSP is a high-performance message middleware in cloud and big data era, provides high-performance and high-reliability message queue service to the outside in a cloud service and cluster mode, and supports a big data acquisition, transmission, aggregation and exchange, big data real-time processing and micro-service processing architecture, and the functions realized by the CMSP are shown in figure 3;
with the fact that Postgresql is familiar and used by more and more users, at present, the functions of the Postgresql cannot realize full-scale and incremental data synchronization with other databases, in order to realize the incremental data synchronization between the PG and other databases, the secondary development is completed by utilizing the logic copying and decoding technology behind the PG9.4 version on the basis of the analysis of a built-in test _ decoding plug-in, a dml change record is obtained, data is serialized at an acquisition end, transmitted through a cmsp, data is obtained at a storage end, deserialized, and self-adaption is carried out according to the type of a target database, so that the incremental data synchronization is completed.
Based on the existing collection mode of the timestamp or the increment field, the collected data cannot meet the service requirements, such as: the acquisition of deleted records and records before the timestamp/incremental point cannot be satisfied, so that an incremental data synchronous acquisition method and system based on sql are provided to solve the problems.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides an increment data synchronous acquisition method and system based on sql, and the adopted technical scheme is as follows: an increment data synchronous acquisition method based on sql comprises the following specific steps:
s1 creating a logic copy slot at the database acquisition end through the execution database sql;
s2 acquiring the increment data of the table structure and logic decoding acquired by the program in real time;
s3, carrying out secondary analysis and serialization on the character string data set subjected to logic decoding, and then transmitting through cmsp;
s4, the warehouse entry end acquires data according to the deserialization of the database type and writes the data into the database;
s5, when the collection requirement is finished, the created logic copy slot is cleared;
s6 obtains database delta data.
The specific step of S1 creating a logical copy slot by executing the database sql at the database acquisition end includes:
s101, creating a logic copy slot aiming at a table needing synchronization;
s102, determining the quantity of data synchronization information of the logical slot management table according to the service requirement.
The specific step of S3 transmitting the logically decoded character string data set through a cmsp after performing secondary parsing and serialization includes:
s301, acquiring acquisition list information through the structural change of a lookup table;
s302, a table needing synchronization and a field needing synchronization are taken from a result set and combined;
s303 serializes the obtained result set and puts the result set into a cmsp for transmission.
The specific steps of S6 include:
s601, analyzing the incremental data to judge whether the incremental data is table information
S602, analyzing the character string to obtain the pattern name and the table name, and simultaneously obtaining the increment type of dml;
s603, circularly analyzing the column names and the column data contents according to the character string storage format;
and S604, receiving and assembling data by the warehousing end.
An increment data synchronous acquisition system based on sql specifically comprises a logic decoding module, a data acquisition module, a data analysis module, a data writing module, a slot cleaning module and a code realization module:
a logic decoding module: creating a logic copy slot at a database acquisition end by executing a database sql;
a data acquisition module: acquiring the increment data of the table structure and the logic decoding by the acquisition program in real time;
a data analysis module: secondarily analyzing and serializing the character string data set subjected to logic decoding, and transmitting the character string data set through cmsp;
a data writing module: the warehousing end obtains data according to the deserialization of the database type and writes the data into the database;
a slot cleaning module: clearing the created logic copy slot after the acquisition requirement is finished;
a code implementation module: and acquiring incremental data of the database.
The logic decoding module establishes a logic copy slot at a database acquisition end through an execution database sql, and specifically comprises an establishing module and a management module:
a creation module: creating a logic copy slot aiming at a table needing synchronization;
a management module: and determining the quantity of the data synchronization information of the copy slot management table according to the service requirement.
The data analysis module analyzes and serializes the character string data set subjected to logic decoding for the second time and transmits the character string data set through a cmsp, and the data analysis module specifically comprises a structure acquisition module, a result processing module and a data transmission module:
a structure acquisition module: acquiring acquisition list information through the structural change of the query table;
a result processing module: combining a table to be synchronized and a field to be synchronized in a result set;
a data transmission module: and serializing the obtained result set into a cmsp for transmission.
The code implementation module is used for acquiring the incremental data of the database and specifically comprises an information processing module, a character string processing module, a circulation analysis module and a data assembly module:
an information processing module: analyzing the incremental data to judge whether the incremental data is table information;
a character string processing module: analyzing the character string to obtain a mode name and a table name, and simultaneously obtaining an increment type of dml;
a cycle analysis module: circularly analyzing the column names and the column data contents according to a character string storage format;
a data assembling module: and the warehousing end receives and assembles the data.
The invention has the beneficial effects that: according to the method, the ddl change can be obtained by obtaining the table structure information for comparison each time, and the defect that most databases cannot synchronize ddl change is overcome;
incremental data are obtained in a logical copy slot mode, and the log is not needed to be analyzed wal by collecting and embedding the code into a database plug-in, so that the analysis efficiency and accuracy are improved;
and the logic decoding result set is analyzed for the second time, so that the table names and the column names which need to be collected can be filtered, the subsequent expansion is more convenient, and the coupling is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a flow chart of the method of the present invention; FIG. 2 is a schematic diagram of the system of the present invention; figure 3 is a functional diagram of a CMSP implementation.
Detailed Description
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
Since PostgreSQL was version 9.4, a test _ decoding plug-in was provided in the source code (default automatic installation); the test _ decor function is to convert the incremental dml records of the database into a character string for storing incremental data, as shown in a picture in the background art, and the technical scheme is to create a database connection at a program acquisition end and acquire the data character string of the incremental result at regular time. Analyzing the character string, filtering the column name and the data content, converting the column name and the data content into a corresponding format, transmitting the corresponding format to a warehouse-in end through the high efficiency of cmsp, and realizing the purpose of incremental data synchronization by assembling data at the warehouse-in end;
the first embodiment is as follows:
an increment data synchronous acquisition method based on sql comprises the following specific steps:
s1 creating a logic copy slot at the database acquisition end through the execution database sql;
s2 acquiring the increment data of the table structure and logic decoding acquired by the program in real time;
s3, carrying out secondary analysis and serialization on the character string data set subjected to logic decoding, and then transmitting through cmsp;
s4, the warehouse entry end acquires data according to the deserialization of the database type and writes the data into the database;
s5, when the collection requirement is finished, the created logic copy slot is cleared;
s6 obtaining incremental data of the database;
firstly, a logical copy slot is created by executing a database sql at a database acquisition end according to S1, and then incremental data after table structure and logical decoding are acquired in real time according to an acquisition program of S2:
select*from pg_logical_slot_peek_changes('zcj_test_slot',NULL,NULL)；
secondly, analyzing and serializing the character string data set subjected to logic decoding for the second time according to S3, and then transmitting the character string data set through cmsp; obtaining data in a deserializing mode according to the type of the database at the database entry end of S4, and writing the data into the database;
after the collection requirement is finished, the created logic copy slot is cleaned up by executing the following sql according to the step S5, so that the pressure on the database caused by continuous management of wal logs is prevented:
select pg_drop_replication_slot('zcj_test_slot')；
finally, according to S6, obtaining database increment data by executing select data from pg _ local _ slot _ seek _ changes ();
the method can not acquire the condition of table structure change through a timestamp or increment field mode, and improves the accuracy and efficiency of analysis by utilizing database analysis; the acquisition module is combined with the efficient transmission characteristic of the CMSP to send the acquired data to a destination terminal, the destination terminal starts corresponding conversion according to the type of the database, and the warehousing module writes the incremental data into a destination database in real time;
further, the specific step of S1 creating the logical copy slot by executing the database sql at the database acquisition end includes:
s101, creating a logic copy slot aiming at a table needing synchronization;
s102, determining the quantity of data synchronization information of a logic slot management table according to service requirements;
creating a logic copy slot by executing a database sql at a database acquisition end:
select*from pg_create_logical_replication_slot('zcj_test_slot','test_decoding','FALSE')；
according to the service requirement, one slot can manage data synchronization information of 1 or more tables, and because character information after logic decoding is adopted contains table name information, the table to which each log record belongs can be screened; one slot manages synchronous information of a plurality of tables, so that the pressure of logical decoding of a database can be relieved, and the influence on service production is reduced;
further, the step S3 of transmitting the logically decoded character string data set through cmsp after parsing and serializing it twice includes:
s301, acquiring acquisition list information through the structural change of a lookup table;
s302, a table needing synchronization and a field needing synchronization are taken from a result set and combined;
s303, serializing the obtained result set and putting the result set into a cmsp for transmission;
the obtained result is as shown in a character string result set described in the background technology, the list information of the acquisition list is obtained through the structural change of the query table, the table to be synchronized and the field to be synchronized are obtained from the result set, the ddl change result and the dml change result of the table can be obtained through the combination of the table to be synchronized and the field to be synchronized, and the result set is serialized and is transmitted in a cmsp;
still further, the specific step of S6 includes:
s601, analyzing the incremental data to judge whether the incremental data is table information;
s602, analyzing the character string to obtain the pattern name and the table name, and simultaneously obtaining the increment type of dml;
s603, circularly analyzing the column names and the column data contents according to the character string storage format;
s604, receiving and assembling data at a warehousing end;
analyzing whether the first mark of the incremental data is 'table', judging whether the mark is table information,
parsing the string obtains the schema name + table name, while obtaining the increment type of dml is INSERT, DELETE, UPDATE,
circularly analyzing the column names and the column data contents according to a character string storage format, serializing the data into a cmsp, and sending the data through the high efficiency of the cmsp;
then the warehouse-in end receives and assembles the data to realize the synchronization of the incremental data;
the table to be collected by the method of the invention must contain a main key field column, otherwise, the logic decoding fails when the record is analyzed and deleted, and the method prompts: DELETE (no-tuple-data), while the result of using the truncate command to clear the table cannot be analyzed, after the command is executed, the logic decoding function of the database cannot acquire the analysis result set;
the method is based on the logic replication and the secondary development of a logic decoding result set, and realizes the incremental data acquisition of the PostgreSQL database; filtering incremental data needing to be collected by setting a collected table name or a collected user name; analyzing character strings of the result set in different modes according to the list types; the invention carries out efficient transmission of messages by means of CMSP, thereby greatly improving the transmission efficiency of data; the invention improves the accuracy of data processing by means of a logic decoding result set.
Example two:
an increment data synchronous acquisition system based on sql specifically comprises a logic decoding module, a data acquisition module, a data analysis module, a data writing module, a slot cleaning module and a code realization module:
a logic decoding module: creating a logic copy slot at a database acquisition end by executing a database sql;
a data acquisition module: acquiring the increment data of the table structure and the logic decoding by the acquisition program in real time;
a data analysis module: secondarily analyzing and serializing the character string data set subjected to logic decoding, and transmitting the character string data set through cmsp;
a data writing module: the warehousing end obtains data according to the deserialization of the database type and writes the data into the database;
a slot cleaning module: clearing the created logic copy slot after the acquisition requirement is finished;
a code implementation module: acquiring incremental data of a database;
firstly, a logic decoding module is utilized to establish a logic copy slot at a database acquisition end through an execution database sql, and then a data acquisition module acquisition program acquires a table structure and logic decoded incremental data in real time:
select*from pg_logical_slot_peek_changes('zcj_test_slot',NULL,NULL)；
secondly, analyzing and serializing the character string data set subjected to logic decoding for the second time through a data analysis module, and then transmitting the character string data set through a cmsp; the data writing module is used for performing deserialization on the data to obtain data according to the type of the database, and the data are written into the database;
after the collection requirement is finished, the created logic copy slot is cleaned by the slot cleaning module through executing the following sql, so that the pressure on the database caused by continuous management of wal logs is prevented:
select pg_drop_replication_slot('zcj_test_slot')；
finally, a code implementation module is used for acquiring database increment data by executing select data from pg _ local _ slot _ seek _ changes ();
the system can not acquire the condition of table structure change through a timestamp or increment field mode, and the accuracy and the efficiency of analysis are improved by utilizing database analysis; the acquisition module is combined with the efficient transmission characteristic of the CMSP to send the acquired data to a destination terminal, the destination terminal starts corresponding conversion according to the type of the database, and the warehousing module writes the incremental data into a destination database in real time;
further, the logic decoding module creates a logic copy slot at the database acquisition end by executing the database sql, and specifically comprises a creating module and a management module:
a creation module: creating a logic copy slot aiming at a table needing synchronization;
a management module: determining the quantity of data synchronization information of the copy slot management table according to the service requirement;
further, the data analysis module secondarily analyzes and serializes the character string data set after the logical decoding and then transmits the character string data set through a cmsp, and specifically includes a structure acquisition module, a result processing module and a data transmission module:
a structure acquisition module: acquiring acquisition list information through the structural change of the query table;
a result processing module: combining a table to be synchronized and a field to be synchronized in a result set;
a data transmission module: serializing the obtained result set and putting the result set into a cmsp for transmission;
further, the code implementation module acquiring the database incremental data specifically comprises an information processing module, a character string processing module, a circular analysis module and a data assembly module:
an information processing module: analyzing the incremental data to judge whether the incremental data is table information;
a character string processing module: analyzing the character string to obtain a mode name and a table name, and simultaneously obtaining an increment type of dml;
a cycle analysis module: circularly analyzing the column names and the column data contents according to a character string storage format;
a data assembling module: and the warehousing end receives and assembles the data.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (8)
1. An increment data synchronous acquisition method based on sql is characterized in that the method comprises the following specific steps:
s1 creating a logic copy slot at the database acquisition end through the execution database sql;
s2 acquiring the increment data of the table structure and logic decoding acquired by the program in real time;
s3, carrying out secondary analysis and serialization on the character string data set subjected to logic decoding, and then transmitting through cmsp;
s4, the warehouse entry end acquires data according to the deserialization of the database type and writes the data into the database;
s5, when the collection requirement is finished, the created logic copy slot is cleared;
s6 obtains database delta data.
2. The method as claimed in claim 1, wherein the step S1 of creating the logical copy slot by executing the database sql at the database collection end comprises:
s101, creating a logic copy slot aiming at a table needing synchronization;
s102, determining the quantity of data synchronization information of the logical slot management table according to the service requirement.
3. The method as claimed in claim 2, wherein the step of S3 parsing and serializing the logically decoded string data set twice for transmission through cmsp comprises:
s301, acquiring acquisition list information through the structural change of a lookup table;
s302, a table needing synchronization and a field needing synchronization are taken from a result set and combined;
s303 serializes the obtained result set and puts the result set into a cmsp for transmission.
4. The method as claimed in claim 3, wherein the step of S6 comprises:
s601, analyzing the incremental data to judge whether the incremental data is table information
S602, analyzing the character string to obtain the pattern name and the table name, and simultaneously obtaining the increment type of dml;
s603, circularly analyzing the column names and the column data contents according to the character string storage format;
and S604, receiving and assembling data by the warehousing end.
5. An increment data synchronous acquisition system based on sql is characterized by specifically comprising a logic decoding module, a data acquisition module, a data analysis module, a data writing module, a slot cleaning module and a code realization module:
a logic decoding module: creating a logic copy slot at a database acquisition end by executing a database sql;
a data acquisition module: acquiring the increment data of the table structure and the logic decoding by the acquisition program in real time;
a data analysis module: secondarily analyzing and serializing the character string data set subjected to logic decoding, and transmitting the character string data set through cmsp;
a data writing module: the warehousing end obtains data according to the deserialization of the database type and writes the data into the database;
a slot cleaning module: clearing the created logic copy slot after the acquisition requirement is finished;
a code implementation module: and acquiring incremental data of the database.
6. The system as claimed in claim 5, wherein the logical decoding module creates the logical copy slot by executing the database sql at the database collection side specifically includes a creation module and a management module:
a creation module: creating a logic copy slot aiming at a table needing synchronization;
a management module: and determining the quantity of the data synchronization information of the copy slot management table according to the service requirement.
7. The system as claimed in claim 6, wherein the data parsing module parses and serializes the logically decoded string data set for the second time and then transmits the parsed string data set through a cmsp, and specifically includes a structure acquisition module, a result processing module, and a data transmission module:
a structure acquisition module: acquiring acquisition list information through the structural change of the query table;
a result processing module: combining a table to be synchronized and a field to be synchronized in a result set;
a data transmission module: and serializing the obtained result set into a cmsp for transmission.
8. The system of claim 7, wherein the code implementation module for obtaining the database incremental data specifically comprises an information processing module, a string processing module, a loop parsing module, and a data assembling module:
an information processing module: analyzing the incremental data to judge whether the incremental data is table information;
a character string processing module: analyzing the character string to obtain a mode name and a table name, and simultaneously obtaining an increment type of dml;
a cycle analysis module: circularly analyzing the column names and the column data contents according to a character string storage format;
a data assembling module: and the warehousing end receives and assembles the data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110345061.1A CN112883118A (en) | 2021-03-31 | 2021-03-31 | Method and system for synchronously acquiring incremental data based on sql |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110345061.1A CN112883118A (en) | 2021-03-31 | 2021-03-31 | Method and system for synchronously acquiring incremental data based on sql |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112883118A true CN112883118A (en) | 2021-06-01 |
Family
ID=76040355
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110345061.1A Pending CN112883118A (en) | 2021-03-31 | 2021-03-31 | Method and system for synchronously acquiring incremental data based on sql |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112883118A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113377851A (en) * | 2021-06-11 | 2021-09-10 | 上海妙一生物科技有限公司 | Data processing system and method |
| CN116860898A (en) * | 2023-09-05 | 2023-10-10 | 建信金融科技有限责任公司 | Data processing method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160314046A1 (en) * | 2015-04-21 | 2016-10-27 | Commvault Systems, Inc. | Content-independent and database management system-independent synthetic full backup of a database based on snapshot technology |
| CN110879813A (en) * | 2019-11-20 | 2020-03-13 | 浪潮软件股份有限公司 | Binary log analysis-based MySQL database increment synchronization implementation method |
| CN111198914A (en) * | 2019-12-12 | 2020-05-26 | 山西云时代技术有限公司 | Whole database real-time data acquisition method based on oracle database filing log |
| CN111459724A (en) * | 2020-03-06 | 2020-07-28 | 中国人民财产保险股份有限公司 | Node switching method, device, equipment and computer readable storage medium |
| CN112269823A (en) * | 2020-10-30 | 2021-01-26 | 浪潮云信息技术股份公司 | Method and system for realizing synchronization of PostgreSQL incremental data |
-
2021
- 2021-03-31 CN CN202110345061.1A patent/CN112883118A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160314046A1 (en) * | 2015-04-21 | 2016-10-27 | Commvault Systems, Inc. | Content-independent and database management system-independent synthetic full backup of a database based on snapshot technology |
| CN110879813A (en) * | 2019-11-20 | 2020-03-13 | 浪潮软件股份有限公司 | Binary log analysis-based MySQL database increment synchronization implementation method |
| CN111198914A (en) * | 2019-12-12 | 2020-05-26 | 山西云时代技术有限公司 | Whole database real-time data acquisition method based on oracle database filing log |
| CN111459724A (en) * | 2020-03-06 | 2020-07-28 | 中国人民财产保险股份有限公司 | Node switching method, device, equipment and computer readable storage medium |
| CN112269823A (en) * | 2020-10-30 | 2021-01-26 | 浪潮云信息技术股份公司 | Method and system for realizing synchronization of PostgreSQL incremental data |
Non-Patent Citations (1)
| Title |
|---|
| 付莎: "PostgreSQL热备原理研究及流复制运用", 《电脑知识与技术》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113377851A (en) * | 2021-06-11 | 2021-09-10 | 上海妙一生物科技有限公司 | Data processing system and method |
| CN116860898A (en) * | 2023-09-05 | 2023-10-10 | 建信金融科技有限责任公司 | Data processing method and device |
| CN116860898B (en) * | 2023-09-05 | 2024-04-23 | 建信金融科技有限责任公司 | Data processing method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102591964B (en) | Implementation method and device for data reading-writing splitting system | |
| CN112883118A (en) | Method and system for synchronously acquiring incremental data based on sql | |
| CN109829011B (en) | Data synchronization method and device for distributed heterogeneous database | |
| CN111125260A (en) | Data synchronization method and system based on SQL Server | |
| CN112286941B (en) | Big data synchronization method and device based on Binlog + HBase + Hive | |
| CN101807210B (en) | Database data synchronic method, system and device | |
| CN111209344A (en) | Data synchronization method and device | |
| CN111046100B (en) | Method and system for synchronizing relational database to non-relational database | |
| CN110968629A (en) | Cross-hierarchy heterogeneous data aggregation-based unified information resource management method and system | |
| CN112269823A (en) | Method and system for realizing synchronization of PostgreSQL incremental data | |
| CN113468170A (en) | System for automatically realizing real-time data synchronization | |
| CN109145155A (en) | High-concurrency warehousing processing method for mass remote sensing image metadata | |
| CN115269630A (en) | Method for performing multi-source heterogeneous database joint query based on custom SQL script | |
| CN114691704A (en) | Metadata synchronization method based on MySQL binlog | |
| CN112000684B (en) | Method and tool for realizing bidirectional synchronization of database of loop control based on TRXID | |
| CN115098567B (en) | Low-code platform data transmission method based on BI platform | |
| CN108228756A (en) | Data based on the PG databases of daily record analytic technique to Hadoop platform synchronize clone method | |
| CN111913933A (en) | Power grid historical data management method and system based on unified support platform | |
| CN102006156A (en) | Method and system for synchronizing configuration data among boards | |
| CN116089545A (en) | Method for collecting storage medium change data into data warehouse | |
| CN113641765B (en) | Unified logic model organization method and device for massive multi-source remote sensing data | |
| CN113342890A (en) | Method and system for realizing cross-security isolation gatekeeper database real-time synchronization | |
| CN115033646A (en) | Method for constructing real-time warehouse system based on Flink and Doris | |
| CN112416944A (en) | Method and equipment for synchronizing service data | |
| CN114896196A (en) | Data migration method and system for domestic database |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210601 |