CN111506556A - Multi-source heterogeneous structured data synchronization method - Google Patents
Multi-source heterogeneous structured data synchronization method Download PDFInfo
- Publication number
- CN111506556A CN111506556A CN202010276827.0A CN202010276827A CN111506556A CN 111506556 A CN111506556 A CN 111506556A CN 202010276827 A CN202010276827 A CN 202010276827A CN 111506556 A CN111506556 A CN 111506556A
- Authority
- CN
- China
- Prior art keywords
- data
- source
- synchronization
- configuration
- record
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The invention discloses a multisource heterogeneous structured data synchronization method, which comprises the following steps: data source configuration: setting a dynamic interface and defining database connection; data table configuration: configuring a source table and a target table; and field configuration: acquiring fields according to a source table and a target table, and marking a main key field and an increment field; data queue management: inserting the synchronous record and operation type into an intermediate table, and inserting or updating the data of the source table into a target table by jointly checking the intermediate table; and (3) synchronous frequency management: different synchronization modes are selected for synchronization of different frequencies; reading the configuration information and executing query operation; and (3) carrying out data synchronization: selecting program processing or stored procedure processing or other processing modes; and displaying the synchronous progress. The invention realizes data replication among different structure tables and task execution with different time requirements based on the trigger tracking database record, so that the data of different database servers in the local area network are kept synchronous.
Description
Technical Field
The invention relates to a multi-source heterogeneous structured data synchronization method, and belongs to the technical field of data processing.
Background
The heterogeneous data refers to data with different structures, and the heterogeneity of the data is mainly embodied in the aspects of the computer system structure, the physical storage of the data is from computers with different system structures, the heterogeneity of the operating system, the heterogeneity of the data format, different storage management mechanisms of the data, a relational database system, such as Oracle, MySQ L and DB2, or a file-type two-dimensional data, such as txt, CSV and X L S, the heterogeneity of data storage places, the data are stored in scattered physical positions, the logic model of the data storage is heterogeneous, the data are stored and maintained in different business logics respectively, so that the data with the same meaning have the heterogeneity, such as the inconsistency of codes of departments in independent sales systems and independent procurement systems, and the heterogeneous data are not heterogeneous in one level, but are heterogeneous in multiple levels.
The existing heterogeneous data synchronization is realized by data transmission between a source database and a target database, and due to different data structures, various types, or same semantics and different data expression forms, the throughput of the synchronous data of massive heterogeneous data is limited, the efficiency of heterogeneous data synchronization is low, and the synchronization technical scheme has certain technical limitations.
Although many databases (such as Oracle, MySQ L, etc.) provide a database data synchronization function, sometimes the application requirements cannot be well met due to the diversity of the actual applications, taking the real estate as an example, data collection and sharing involve multiple different databases, the structures are different, and the real-time requirements are different.
Disclosure of Invention
Aiming at the defects of the method, the invention provides a multi-source heterogeneous structured data synchronization method which can realize data synchronization of different database servers in a local area network.
The technical scheme adopted for solving the technical problems is as follows:
the embodiment of the invention provides a multi-source heterogeneous structured data synchronization method, which comprises the following steps:
step 1, data source configuration: setting a dynamic interface and defining database connection;
step 2, data table configuration: configuring a source table and a target table;
step 3, field configuration: acquiring fields according to a source table and a target table, and marking a main key field and an increment field;
step 4, data queue management: inserting the synchronous record and operation type into an intermediate table, and inserting or updating the data of the source table into a target table by jointly checking the intermediate table;
step 5, synchronous frequency management: different synchronization modes are selected for synchronization of different frequencies;
step 6, reading the configuration information and executing query operation;
and 7, carrying out data synchronization: selecting program processing or stored procedure processing or other processing modes;
and 8, synchronous progress display.
As a possible implementation manner of this embodiment, the data source configuration process includes: setting a scheme name, a source database connection definition, a target database definition, an increment value, a synchronization time and a synchronization state dynamic interface; and performing database connection definition on the dynamic interface, wherein the format of the dynamic interface is as follows: username/password/database address/database instance/port/database type.
As a possible implementation manner of this embodiment, the source table configuration adopts a tree structure, a first node configures a main table, a second node configures a sub table associated with the first node, and so on; the target table not only configures table names, but also can be logically inserted, updated and deleted.
As a possible implementation manner of this embodiment, the primary key field is used to find whether the target table has the record; the increment field is used for judging whether the source table has new adding, updating or deleting operation.
As a possible implementation manner of this embodiment, the specific process of data queue management is as follows: inserting the synchronized records and operation types into an intermediate table through a trigger, sequencing the intermediate table through a main key, reading data by a program according to a corresponding scheme, marking the unsynchronized state as 0 and the synchronizing record as 1; inserting or updating the data of the source table into the target table by jointly checking the intermediate table; when the synchronization is successful, the record with the middle table of 1 is deleted.
As a possible implementation manner of this embodiment, the synchronization manner is divided into an active manner and a passive manner; actively scanning a precise data source according to interval time, wherein a user can set time by himself in milliseconds; and passive, receiving the user notification, and actively synchronizing the service data by the socket or the Webservice through the program.
As a possible implementation manner of this embodiment, for a table that requires real-time sharing of data, a passive type is selected during configuration; the selection initiative with the requirement on the time interval is used for setting the interval time and timing to execute the task.
As a possible implementation manner of this embodiment, the process of reading the configuration information and executing the query operation includes: and reading database connection configuration, source table configuration and field configuration, executing query reading operation through configuration information, and performing relevant processing on fields needing conversion or encryption/decryption.
As a possible implementation manner of this embodiment, the data synchronization process is: reading the name of a program method, and calling a related processing method;
if the synchronization mode is newly added after deletion, when writing in the target table, checking whether the table has the same record according to the primary key, if so, deleting the record and then inserting the record; if not, the insertion is direct. If the synchronization mode is updating, when writing in the target table, checking whether the table has the same record according to the primary key, if so, directly updating; if not, adding the new file after deletion;
if the synchronous mode is deleting, when writing in the target table, checking whether the table has the same record according to the primary key, if so, directly deleting.
As a possible implementation manner of this embodiment, the process of displaying the synchronous progress includes: and controlling the running state of the program through the program running interface, and displaying the running condition of each synchronous scheme.
The technical scheme of the embodiment of the invention has the following beneficial effects:
the method and the device realize data replication among different structure tables and task execution with different time requirements by means of data source configuration, field configuration and rule configuration based on trigger tracking database records, so that data of different database servers in a local area network are kept synchronous.
Description of the drawings:
FIG. 1 is a flow diagram illustrating a method for multi-source heterogeneous structured data synchronization in accordance with an exemplary embodiment;
FIG. 2 is a flow diagram illustrating an implementation of a multi-source heterogeneous structured data synchronization method in accordance with an illustrative embodiment;
FIG. 3 is a flowchart illustrating a programming according to an exemplary embodiment;
FIG. 4 is a diagram illustrating a synchronization rule, according to an example embodiment.
Detailed Description
The invention is further illustrated by the following examples in conjunction with the accompanying drawings:
in order to clearly explain the technical features of the present invention, the following detailed description of the present invention is provided with reference to the accompanying drawings. The following disclosure provides many different embodiments, or examples, for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. It should be noted that the components illustrated in the figures are not necessarily drawn to scale. Descriptions of well-known components and processing techniques and procedures are omitted so as to not unnecessarily limit the invention.
FIG. 1 is a flow diagram illustrating a method for multi-source heterogeneous structured data synchronization in accordance with an exemplary embodiment. As shown in fig. 1, a multi-source heterogeneous structured data synchronization method provided in an embodiment of the present invention includes the following steps:
step 1, data source configuration: setting a dynamic interface and defining database connection;
step 2, data table configuration: configuring a source table and a target table;
step 3, field configuration: acquiring fields according to a source table and a target table, and marking a main key field and an increment field;
step 4, data queue management: inserting the synchronous record and operation type into an intermediate table, and inserting or updating the data of the source table into a target table by jointly checking the intermediate table;
step 5, synchronous frequency management: different synchronization modes are selected for synchronization of different frequencies;
step 6, reading the configuration information and executing query operation;
and 7, carrying out data synchronization: selecting program processing or stored procedure processing or other processing modes;
and 8, synchronous progress display.
As a possible implementation manner of this embodiment, the data source configuration process includes: setting a scheme name, a source database connection definition, a target database definition, an increment value, a synchronization time and a synchronization state dynamic interface; and performing database connection definition on the dynamic interface, wherein the format of the dynamic interface is as follows: username/password/database address/database instance/port/database type.
As a possible implementation manner of this embodiment, the source table configuration adopts a tree structure, a first node configures a main table, a second node configures a sub table associated with the first node, and so on; the target table not only configures table names, but also can be logically inserted, updated and deleted.
As a possible implementation manner of this embodiment, the primary key field is used to find whether the target table has the record; the increment field is used for judging whether the source table has new adding, updating or deleting operation.
As a possible implementation manner of this embodiment, the specific process of data queue management is as follows: inserting the synchronized records and operation types into an intermediate table through a trigger, sequencing the intermediate table through a main key, reading data by a program according to a corresponding scheme, marking the unsynchronized state as 0 and the synchronizing record as 1; inserting or updating the data of the source table into the target table by jointly checking the intermediate table; when the synchronization is successful, the record with the middle table of 1 is deleted.
As a possible implementation manner of this embodiment, the synchronization manner is divided into an active manner and a passive manner; actively scanning a precise data source according to interval time, wherein a user can set time by himself in milliseconds; and passive, receiving the user notification, and actively synchronizing the service data by the socket or the Webservice through the program.
As a possible implementation manner of this embodiment, for a table that requires real-time sharing of data, a passive type is selected during configuration; the selection initiative with the requirement on the time interval is used for setting the interval time and timing to execute the task.
As a possible implementation manner of this embodiment, the process of reading the configuration information and executing the query operation includes: and reading database connection configuration, source table configuration and field configuration, executing query reading operation through configuration information, and performing relevant processing on fields needing conversion or encryption/decryption.
As a possible implementation manner of this embodiment, the data synchronization process is: reading the name of a program method, and calling a related processing method;
if the synchronization mode is newly added after deletion, when writing in the target table, checking whether the table has the same record according to the primary key, if so, deleting the record and then inserting the record; if not, the insertion is direct. If the synchronization mode is updating, when writing in the target table, checking whether the table has the same record according to the primary key, if so, directly updating; if not, adding the new file after deletion;
if the synchronous mode is deleting, when writing in the target table, checking whether the table has the same record according to the primary key, if so, directly deleting.
As a possible implementation manner of this embodiment, the process of displaying the synchronous progress includes: and controlling the running state of the program through the program running interface, and displaying the running condition of each synchronous scheme.
The method and the device realize data replication among different structure tables and task execution with different time requirements by means of data source configuration, field configuration and rule configuration based on the trigger tracking database record, and realize data synchronization of different database servers in the local area network.
As shown in fig. 2, the specific implementation steps of the present invention are as follows.
1. Data source configuration
The method comprises the steps of setting dynamic interfaces such as scheme name, source database connection definition, target database definition, increment value, synchronization time, synchronization state and the like, wherein the database connection definition format is user name/password/database address/database instance/port/database type, and informing the JDBC program to connect through the UR L address so as to access a source library and a target library.
2. Data table configuration
The method comprises the steps of configuring a main table by a first node, configuring sub-tables (a plurality of sub-tables) associated with the first node by a second node, and so on, setting query conditions for each table, inputting SQ L for retrieval, wherein the relationship between the main table and the sub-tables is also queried and associated through SQ L.
The target table enables methods of logical insertion, update, and deletion in addition to table name configuration, by correlating SQ L conditions.
3. Field configuration
And acquiring fields according to the table name, and marking a primary key field and an increment field. Looking up whether the target table has the record or not through the primary key field; and judging whether the source table has operations of adding, updating or deleting through the increment field. And performing list correspondence on the source table field and the target table field.
Considering different table structures and the synchronization of different types of fields, the conversion of the value of the field to be inserted is carried out through key value pairs or dictionary tables, and the conversion method is realized through SQ L reading, considering the safety of partial field reading, the program adopts DES (data Encryption Standard) symmetric Encryption/decryption, and the most obvious difference from other algorithms is that a working key is used for Encryption and decryption.
4. Data queue management
The synchronized records and operation types are inserted into the intermediate table through the trigger, the intermediate table is sorted through the main key, the program reads data according to the corresponding scheme, the unsynchronized state is marked as 0, and the records being synchronized are marked as 1. The source table data is inserted or updated into the target table by cross-referencing the intermediate table. When the synchronization is successful, the record with the middle table of 1 is deleted. The adoption of the queue management can flexibly control the synchronization condition and avoid the repeated synchronization of data.
5. Synchronous frequency management
The working mode of the program is divided into an active mode and a passive mode, the active mode actively scans a precise data source according to the interval time, a user can set the time by himself by taking millisecond as a unit, the passive mode receives the notification of the user, and the socket or the Webservice actively synchronizes service data through the program, wherein an access control list-AC L of the database is selected to realize the passive mode.
The synchronization of different frequencies may select different synchronization modes. The advantage of the passive synchronization method is mainly embodied in real-time. For the table with data required to be shared in real time, the passive type can be selected during configuration; the initiative can be selected when the time interval is required, and the task is executed in a timing mode by setting the interval time.
6. Programming
And developing synchronous software by combining the configuration management information. In order to realize synchronous efficient and orderly operation and quick abnormal positioning, the method can be developed by being divided into a plurality of modules, and the program has flexibility and expansibility as much as possible. As shown in fig. 3, the programming process is as follows:
(1) start-up/shut-down synchronization
The user can control the program running by clicking the button for starting synchronization/stopping synchronization on the running interface, and simultaneously, according to the setting of the frequency, the next synchronization task can be executed at a fixed time or at the next time interval.
(2) Checking configuration content
Reading database connection configuration, source table configuration and field configuration, and checking the setting of necessary fields, such as a main key field and an increment field; whether the content format is correct; whether the links of the data source and the target library are real or not; whether synchronization tables and fields exist, etc.
(2) Reading configuration information and executing query operation
And reading database connection configuration, source table configuration and field configuration, executing query reading operation through configuration information, and performing relevant processing on fields needing conversion or encryption/decryption.
(3) Executing synchronization rules
The synchronous processing mode comprises the following steps: selecting program processing or stored procedure processing or other processing modes. A synchronous processing program: and reading the name of the program method and calling the related processing method. If the synchronization mode is newly added after deletion, when writing in the target table, checking whether the same record exists in the table according to the primary key, and if so, deleting the record and then inserting the record; if not, the insertion is direct. If the synchronization mode is updating, when writing in the target table, checking whether the table has the same record according to the primary key, if so, directly updating; if not, the new file is added after deletion. If the synchronization mode is deletion, when writing in the target table, checking whether the same record exists in the table according to the primary key, and if so, directly deleting the record. See figure 4 for details.
(4) Synchronous progress display
The program running interface not only controls the running state of the program, but also can clearly show the running condition of each synchronization scheme. The running state of the monitoring program can be clearly monitored by the display fields of task state, time interval, data type, increment key value, working mode and synchronous time
(5) Synchronizing logs, exception log records
In order to facilitate subsequent error positioning, inquiry and solve the problem of synchronization efficiency, log tables are recorded for the access start time, the access end time, the total completion time, the access number, abnormal information and the like of each scheme, and log tables are recorded for the synchronization table, the increment value, the primary key value, the start synchronization time, the end synchronization time, the total synchronization time, the execution SQ L, the abnormal information and the like of each scheme.
(6) Monitoring management and exception handling
The running state of the program is controlled on the running interface of the program, the running condition of each synchronous scheme can be clearly shown, and the running state of the program is monitored. And storing the abnormal information into a database, regularly scanning an abnormal and wrong information table of the database when designing a timing task, and finally pushing the abnormal and wrong information table to a developer in advance in a mail form for processing.
Selecting a copied source database and a copied target database by a user; configuring relevant information of a source table and a target table; acquiring fields and completing field correspondence and field conversion; generating a corresponding synchronous task table for each scheme; configuring a synchronous rule and a synchronous mode (a timing or periodic synchronous mode can be adopted) by a user; and finally, recording the synchronization process in a log mode. The process is realized by combining database design and program design, wherein the operations of inserting, updating and deleting data are recorded in a table through a trigger, and synchronization is carried out in a queue management mode.
The foregoing is only a preferred embodiment of the present invention, and it will be apparent to those skilled in the art that various modifications and improvements can be made without departing from the principle of the present invention, and these modifications and improvements are also considered to be within the scope of the present invention.
Claims (10)
1. A multi-source heterogeneous structured data synchronization method is characterized by comprising the following steps:
step 1, data source configuration: setting a dynamic interface and defining database connection;
step 2, data table configuration: configuring a source table and a target table;
step 3, field configuration: acquiring fields according to a source table and a target table, and marking a main key field and an increment field;
step 4, data queue management: inserting the synchronous record and operation type into an intermediate table, and inserting or updating the data of the source table into a target table by jointly checking the intermediate table;
step 5, synchronous frequency management: different synchronization modes are selected for synchronization of different frequencies;
step 6, reading the configuration information and executing query operation;
and 7, carrying out data synchronization: selecting program processing or stored procedure processing or other processing modes;
and 8, synchronous progress display.
2. The multi-source heterogeneous structured data synchronization method according to claim 1, wherein the data source configuration process is as follows: setting a scheme name, a source database connection definition, a target database definition, an increment value, a synchronization time and a synchronization state dynamic interface; and performing database connection definition on the dynamic interface, wherein the format of the dynamic interface is as follows: username/password/database address/database instance/port/database type.
3. The multi-source heterogeneous structured data synchronization method according to claim 1, wherein the source table configuration adopts a tree structure, a first node configures a main table, a second node configures a sub table associated therewith, and so on; the target table not only configures table names, but also can be logically inserted, updated and deleted.
4. The multi-source heterogeneous structured data synchronization method according to claim 1, wherein said primary key field is used to look up whether the target table has the record; the increment field is used for judging whether the source table has new adding, updating or deleting operation.
5. The multi-source heterogeneous structured data synchronization method according to claim 1, wherein the specific process of the data queue management is as follows: inserting the synchronized records and operation types into an intermediate table through a trigger, sequencing the intermediate table through a main key, reading data by a program according to a corresponding scheme, marking the unsynchronized state as 0 and the synchronizing record as 1; inserting or updating the data of the source table into the target table by jointly checking the intermediate table; when the synchronization is successful, the record with the middle table of 1 is deleted.
6. The multi-source heterogeneous structured data synchronization method according to claim 1, wherein the synchronization mode is divided into an active mode and a passive mode; actively scanning a precise data source according to interval time, wherein a user can set time by himself in milliseconds; and passive, receiving the user notification, and actively synchronizing the service data by the socket or the Webservice through the program.
7. The multi-source heterogeneous structured data synchronization method according to claim 6, wherein the table requiring real-time sharing of data is selected passively when configured; the selection initiative with the requirement on the time interval is used for setting the interval time and timing to execute the task.
8. The multi-source heterogeneous structured data synchronization method according to claim 1, wherein the process of reading the configuration information and executing the query operation is as follows: and reading database connection configuration, source table configuration and field configuration, executing query reading operation through configuration information, and performing relevant processing on fields needing conversion or encryption/decryption.
9. The multi-source heterogeneous structured data synchronization method according to claim 1, wherein the data synchronization process is as follows: reading the name of a program method, and calling a related processing method;
if the synchronization mode is newly added after deletion, when writing in the target table, checking whether the table has the same record according to the primary key, if so, deleting the record and then inserting the record; if not, the insertion is direct. If the synchronization mode is updating, when writing in the target table, checking whether the table has the same record according to the primary key, if so, directly updating; if not, adding the new file after deletion;
if the synchronous mode is deleting, when writing in the target table, checking whether the table has the same record according to the primary key, if so, directly deleting.
10. The multi-source heterogeneous structured data synchronization method according to claim 1, wherein the synchronization progress display process is as follows: and controlling the running state of the program through the program running interface, and displaying the running condition of each synchronous scheme.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010276827.0A CN111506556A (en) | 2020-04-09 | 2020-04-09 | Multi-source heterogeneous structured data synchronization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010276827.0A CN111506556A (en) | 2020-04-09 | 2020-04-09 | Multi-source heterogeneous structured data synchronization method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111506556A true CN111506556A (en) | 2020-08-07 |
Family
ID=71872656
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010276827.0A Pending CN111506556A (en) | 2020-04-09 | 2020-04-09 | Multi-source heterogeneous structured data synchronization method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111506556A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112182104A (en) * | 2020-09-25 | 2021-01-05 | 中国建设银行股份有限公司 | Data synchronization method, device, equipment and storage medium |
| CN112181944A (en) * | 2020-09-24 | 2021-01-05 | 陕西天行健车联网信息技术有限公司 | Application of offline data synchronization tool of heterogeneous data source in Internet of vehicles industry |
| CN112434004A (en) * | 2020-10-28 | 2021-03-02 | 金蝶云科技有限公司 | Data migration method and device for heterogeneous system, computer equipment and storage medium |
| CN113051347A (en) * | 2021-03-25 | 2021-06-29 | 未鲲(上海)科技服务有限公司 | Method, system, equipment and storage medium for data synchronization between heterogeneous databases |
| CN113326316A (en) * | 2021-04-14 | 2021-08-31 | 深圳新贝奥数字科技有限公司 | Rapid data synchronization method for heterogeneous system |
| CN113821565A (en) * | 2021-09-10 | 2021-12-21 | 上海得帆信息技术有限公司 | Method for synchronizing data of multiple data sources |
| US20220075799A1 (en) * | 2020-09-08 | 2022-03-10 | Chinese University of Hong Kong in Shenzhen | Terminal, storage medium, and database synchronization method thereof |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030037029A1 (en) * | 2001-08-15 | 2003-02-20 | Iti, Inc. | Synchronization of plural databases in a database replication system |
| CN105224527A (en) * | 2014-05-27 | 2016-01-06 | 北京宸瑞科技有限公司 | Be applicable to the general ETL method of multiple object table update mode |
| CN107885804A (en) * | 2017-11-02 | 2018-04-06 | 平安科技（深圳）有限公司 | Database synchronization method, application server and computer-readable recording medium |
| CN108681590A (en) * | 2018-05-15 | 2018-10-19 | 普信恒业科技发展（北京）有限公司 | Incremental data processing method and processing device, computer equipment, computer storage media |
| CN108900497A (en) * | 2018-06-25 | 2018-11-27 | 江苏欧软信息科技有限公司 | Method of data synchronization and system between a kind of heterogeneous system |
| CN109284312A (en) * | 2018-08-27 | 2019-01-29 | 山东威尔数据股份有限公司 | A kind of heterogeneous database change real-time informing method |
| CN109669983A (en) * | 2018-12-27 | 2019-04-23 | 杭州火树科技有限公司 | Visualize multi-data source ETL tool |
| CN110297866A (en) * | 2019-05-20 | 2019-10-01 | 平安普惠企业管理有限公司 | Method of data synchronization and data synchronization unit based on log analysis |
-
2020
- 2020-04-09 CN CN202010276827.0A patent/CN111506556A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030037029A1 (en) * | 2001-08-15 | 2003-02-20 | Iti, Inc. | Synchronization of plural databases in a database replication system |
| CN105224527A (en) * | 2014-05-27 | 2016-01-06 | 北京宸瑞科技有限公司 | Be applicable to the general ETL method of multiple object table update mode |
| CN107885804A (en) * | 2017-11-02 | 2018-04-06 | 平安科技（深圳）有限公司 | Database synchronization method, application server and computer-readable recording medium |
| CN108681590A (en) * | 2018-05-15 | 2018-10-19 | 普信恒业科技发展（北京）有限公司 | Incremental data processing method and processing device, computer equipment, computer storage media |
| CN108900497A (en) * | 2018-06-25 | 2018-11-27 | 江苏欧软信息科技有限公司 | Method of data synchronization and system between a kind of heterogeneous system |
| CN109284312A (en) * | 2018-08-27 | 2019-01-29 | 山东威尔数据股份有限公司 | A kind of heterogeneous database change real-time informing method |
| CN109669983A (en) * | 2018-12-27 | 2019-04-23 | 杭州火树科技有限公司 | Visualize multi-data source ETL tool |
| CN110297866A (en) * | 2019-05-20 | 2019-10-01 | 平安普惠企业管理有限公司 | Method of data synchronization and data synchronization unit based on log analysis |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20220075799A1 (en) * | 2020-09-08 | 2022-03-10 | Chinese University of Hong Kong in Shenzhen | Terminal, storage medium, and database synchronization method thereof |
| US11604805B2 (en) * | 2020-09-08 | 2023-03-14 | Chinese University of Hong Kong in Shenzhen | Terminal, storage medium, and database synchronization method thereof |
| CN112181944A (en) * | 2020-09-24 | 2021-01-05 | 陕西天行健车联网信息技术有限公司 | Application of offline data synchronization tool of heterogeneous data source in Internet of vehicles industry |
| CN112182104A (en) * | 2020-09-25 | 2021-01-05 | 中国建设银行股份有限公司 | Data synchronization method, device, equipment and storage medium |
| CN112434004A (en) * | 2020-10-28 | 2021-03-02 | 金蝶云科技有限公司 | Data migration method and device for heterogeneous system, computer equipment and storage medium |
| CN113051347A (en) * | 2021-03-25 | 2021-06-29 | 未鲲(上海)科技服务有限公司 | Method, system, equipment and storage medium for data synchronization between heterogeneous databases |
| CN113051347B (en) * | 2021-03-25 | 2024-03-29 | 未鲲(上海)科技服务有限公司 | Method, system, equipment and storage medium for synchronizing data between heterogeneous databases |
| CN113326316A (en) * | 2021-04-14 | 2021-08-31 | 深圳新贝奥数字科技有限公司 | Rapid data synchronization method for heterogeneous system |
| CN113821565A (en) * | 2021-09-10 | 2021-12-21 | 上海得帆信息技术有限公司 | Method for synchronizing data of multiple data sources |
| CN113821565B (en) * | 2021-09-10 | 2024-03-15 | 上海得帆信息技术有限公司 | Method for synchronizing data by multiple data sources |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111506556A (en) | Multi-source heterogeneous structured data synchronization method | |
| EP3602341B1 (en) | Data replication system | |
| RU2599538C2 (en) | Methods and systems for loading data into temporal data warehouse | |
| CN111460023B (en) | Method, device, equipment and storage medium for processing service data based on elastic search | |
| US8938421B2 (en) | Method and a system for synchronizing data | |
| US20170004185A1 (en) | Method and system for implementing collection-wise processing in a log analytics system | |
| US20120254249A1 (en) | Database Management System | |
| CN111324610A (en) | Data synchronization method and device | |
| CN112997167A (en) | Task scheduling in a database system | |
| US11874875B2 (en) | Graph processing system | |
| US8595238B2 (en) | Smart index creation and reconciliation in an interconnected network of systems | |
| US8260761B2 (en) | Detecting performance degrading design and algorithm issues in database applications | |
| CN113407600B (en) | Enhanced real-time calculation method for dynamically synchronizing multi-source large table data in real time | |
| US20180032567A1 (en) | Method and device for processing data blocks in a distributed database | |
| US7020656B1 (en) | Partition exchange loading technique for fast addition of data to a data warehousing system | |
| US20220027365A1 (en) | Enterprise resource planning system, server and supervision method of sql queries in such a system or server | |
| CN111708895B (en) | Knowledge graph system construction method and device | |
| US11907262B2 (en) | System and method for data pruning via dynamic partition management | |
| CN107168822B (en) | Oracle streams exception recovery system and method | |
| CN115080666A (en) | Data synchronization method, system, electronic device and storage medium | |
| CN111680036B (en) | Configuration management database based on graph storage | |
| US11836125B1 (en) | Scalable database dependency monitoring and visualization system | |
| CN111259082A (en) | Method for realizing full data synchronization in big data environment | |
| US10558637B2 (en) | Modularized data distribution plan generation | |
| Lev-Ari et al. | Quick: a queuing system in cloudkit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |