CN112000681A - Synchronization method capable of adapting mode change - Google Patents
Synchronization method capable of adapting mode change Download PDFInfo
- Publication number
- CN112000681A CN112000681A CN202010861100.9A CN202010861100A CN112000681A CN 112000681 A CN112000681 A CN 112000681A CN 202010861100 A CN202010861100 A CN 202010861100A CN 112000681 A CN112000681 A CN 112000681A
- Authority
- CN
- China
- Prior art keywords
- data
- mode
- information
- end database
- record
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000008859 change Effects 0.000 title claims abstract description 20
- 230000001360 synchronised effect Effects 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 6
- 230000003044 adaptive effect Effects 0.000 claims description 3
- 238000012790 confirmation Methods 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a synchronization method capable of self-adapting mode change, belonging to the technical field of real-time data synchronization, and comprising the following steps: the method comprises the following steps: the data pusher will push out the record; step two: registering and registering by a mode registration center; step three: the pattern information detector parses the record. Based on data-based synchronization, mode change automatic synchronization is supported.
Description
Technical Field
The invention relates to a data real-time synchronization technology, in particular to a synchronization method capable of self-adapting mode change.
Background
The data synchronization is a pure synchronization which means that the same operation is performed at the same time, and the data synchronization positions an object as data, which is mostly understood as a backup operation between different storage devices or a terminal and a terminal, and a server. However, the complete data synchronization should be real-time, that is, the two current operating parties should be mirror images of each other, for example, the data synchronization is performed until the backup process is completed, but the operation sample should belong to the backup.
Real-time data synchronization is the process of establishing a real-time coordination between a data source and a target. A common real-time synchronization technique for data is a data-based synchronization technique. The data transmitted by the technology does not contain information for detailed operation of the database, so that the DML operation on the table of the source database cannot be synchronized to the end database.
Disclosure of Invention
In order to solve the above technical problems, the present invention provides a synchronization method capable of adapting to mode changes, which supports automatic synchronization of mode changes based on data synchronization.
The method realizes a mode version management method based on a registration mechanism by defining a uniform mode template, and mainly supports the function of automatic synchronization of a source database table mode change back-end database table.
The technical scheme of the invention is as follows:
a synchronization method capable of adaptive mode change, comprising:
step 1): the data pusher pushes out the record;
step 2): registering and registering by a mode registration center;
step 3): the pattern information detector parses the record.
Further, in the above-mentioned case,
the step 1) comprises the following steps:
mode information, source table information, and synchronization message patterns are bound in the record.
In a still further aspect of the present invention,
when a new record is generated, the data pusher pushes the new record to the data capturer; the data pusher can verify the integrity of the record, and if any information is missing in the record, the data pusher can throw back the end database and process the end database after requesting confirmation.
Further, in the above-mentioned case,
step 2) comprises the following steps:
2.1) the data change capturer generates source table mode information according to the source table field information and the synchronous message mode template;
2.2) sending to a mode registration center to complete initialization registration;
3.3) assembling the registration information into DML and sending the DML to an end database.
In a still further aspect of the present invention,
and after the registration information is sent to the end database, the end database completes the table creation operation.
Further, in the above-mentioned case,
via step 1), step 2) source database and end database schema are consistent, then synchronize the data normally.
Step 3) comprises the following steps:
3.1) the data change capturer sends data to the mode information detector for analysis, namely flattening and re-assembling;
3.2) the mode information detector pulls the registration information in the mode registry to compare with the new data pattern;
3.3) if the data are the same, synchronizing to the end database; if not, the data change capturer reinitializes the registration mode, assembles new mode information into a DML and sends the DML to the end database.
At this point, the schema remains consistent and the data can be synchronized normally to the end database.
The invention has the advantages that
The invention can solve the problem of mode change. The conventional idea is that if schema changes are required, the table fields of the end database need to be modified according to the table field changes in the source database. And when the table field of the source database is changed, the corresponding table field of the end database is automatically modified.
In conclusion, the scheme can avoid the complex operation of modifying the table structure.
Drawings
FIG. 1 is a schematic workflow diagram of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention, and based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts belong to the scope of the present invention.
The invention solves the problem that the traditional real-time synchronization technical method based on data does not support automatic mode change, namely, in the data synchronization process, the mode templates are required to be kept consistent in principle. When the table structure of the source data table changes, such as adding a new field, deleting a field or modifying the name of a field, the table structure of the target table is also changed correspondingly.
The invention mainly relates to a mode registration center for persistently storing mode information, a data change capturer and a mode information detector.
The method comprises the following steps: the data pusher will push out the record
Each time a new record is generated, the data pusher pushes it to the data capturer. Each record contains source table data, field information, and synchronization message patterns.
The data pusher can verify the integrity of the records, and if any information is missing in the records, the data pusher can throw back the end database and process the end database after requesting confirmation.
Step two: mode registry registration mode
And the data change capturer generates source table mode information according to the source table field information and the synchronous message mode, sends the source table mode information to the mode registration center to complete registration, assembles the registration information into DML (distributed management language) and sends the DML to the end database, and the end database completes table creation operation.
Step three: parsing records by a pattern information detector
In step two, the source database and the end database are in schema agreement, followed by normal synchronization of the data, via step one.
The data change capturer sends each piece of data to the pattern information detector, the patterns of the data are flattened and reassembled, the pattern information detector compares the newly acquired pattern information with the information registered in the pattern registration center before, and if the newly acquired pattern information is the same as the information registered in the pattern registration center, the pattern information detector delivers the pattern information to the end database for operation. And if the DML is different from the DML, sending new data information to the mode registration center, reinitializing registration, assembling into a new DML, and sending the new DML to the end database. At this point, the schema remains consistent and the data can be synchronized normally to the end database.
The key technology is realized:
data binding source table data, field information and synchronization message patterns. Is the basis for realizing the mode adaptive updating.
The modes are managed in the mode registration center in a centralized mode, comparison of data modes at each time is facilitated, correctness is verified, and modes of DML updating end databases can be assembled according to mode changes.
Noun interpretation
DML: a data Manipulation language dml (data Manipulation language) through which a user can perform basic operations on a database, including insertion, modification, and deletion operations.
Mode change: a table in the source database has its schema (e.g., column name or column type) changed during synchronization.
The above description is only a preferred embodiment of the present invention, and is only used to illustrate the technical solutions of the present invention, and not to limit the protection scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (8)
1. A synchronization method capable of adaptive mode change, comprising:
step 1): the data pusher pushes out the record;
step 2): registering and registering by a mode registration center;
step 3): the pattern information detector parses the record.
2. The method of claim 1,
the step 1) comprises the following steps:
mode information, source table information, and synchronization message patterns are bound in the record.
3. The method of claim 2,
when a new record is generated, the data pusher pushes the new record to the data capturer; the data pusher can verify the integrity of the record, and if any information is missing in the record, the data pusher can throw back the end database and process the end database after requesting confirmation.
4. The stepping method of claim 1,
step 2) comprises the following steps:
2.1) the data change capturer generates source table mode information according to the source table field information and the synchronous message mode template;
2.2) sending to a mode registration center to complete initialization registration;
3.3) assembling the registration information into DML and sending the DML to an end database.
5. The stepping method of claim 4,
and after the registration information is sent to the end database, the end database completes the table creation operation.
6. The method of claim 1,
via step 1), step 2) source database and end database schema are consistent, then synchronize the data normally.
7. The method of claim 6,
step 3) comprises the following steps:
3.1) the data change capturer sends data to the mode information detector for analysis, namely flattening and re-assembling;
3.2) the mode information detector pulls the registration information in the mode registry to compare with the new data pattern;
3.3) if the data are the same, synchronizing to the end database; if not, the data change capturer reinitializes the registration mode, assembles new mode information into a DML and sends the DML to the end database.
8. The method of claim 7,
at this point, the schema remains consistent and the data can be synchronized normally to the end database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010861100.9A CN112000681A (en) | 2020-08-25 | 2020-08-25 | Synchronization method capable of adapting mode change |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010861100.9A CN112000681A (en) | 2020-08-25 | 2020-08-25 | Synchronization method capable of adapting mode change |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112000681A true CN112000681A (en) | 2020-11-27 |
Family
ID=73471812
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010861100.9A Pending CN112000681A (en) | 2020-08-25 | 2020-08-25 | Synchronization method capable of adapting mode change |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112000681A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112783981A (en) * | 2021-02-05 | 2021-05-11 | 腾讯科技（深圳）有限公司 | Method and device for processing and monitoring data in reference station network |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101655812A (en) * | 2009-09-15 | 2010-02-24 | 中兴通讯股份有限公司 | Self-adaptive data backup system and method thereof |
| CN109033193A (en) * | 2018-06-16 | 2018-12-18 | 温州职业技术学院 | A kind of database structure and its method of data synchronization |
| CN109376160A (en) * | 2018-08-21 | 2019-02-22 | 平安科技（深圳）有限公司 | Method of data synchronization, device, computer equipment and storage medium |
| CN109997125A (en) * | 2016-09-15 | 2019-07-09 | 英国天然气控股有限公司 | System for importing data to data storage bank |
| CN110188118A (en) * | 2019-04-25 | 2019-08-30 | 广州至真信息科技有限公司 | A kind of method of data synchronization, device |
| CN110427426A (en) * | 2019-08-02 | 2019-11-08 | 中国工商银行股份有限公司 | A kind of data synchronizing processing method and device |
| CN111177173A (en) * | 2020-01-02 | 2020-05-19 | 普元信息技术股份有限公司 | System and method for realizing data synchronization optimization processing under big data environment |
| CN111274257A (en) * | 2020-01-20 | 2020-06-12 | 山东汇贸电子口岸有限公司 | Real-time synchronization method and system based on data |
-
2020
- 2020-08-25 CN CN202010861100.9A patent/CN112000681A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101655812A (en) * | 2009-09-15 | 2010-02-24 | 中兴通讯股份有限公司 | Self-adaptive data backup system and method thereof |
| CN109997125A (en) * | 2016-09-15 | 2019-07-09 | 英国天然气控股有限公司 | System for importing data to data storage bank |
| CN109033193A (en) * | 2018-06-16 | 2018-12-18 | 温州职业技术学院 | A kind of database structure and its method of data synchronization |
| CN109376160A (en) * | 2018-08-21 | 2019-02-22 | 平安科技（深圳）有限公司 | Method of data synchronization, device, computer equipment and storage medium |
| CN110188118A (en) * | 2019-04-25 | 2019-08-30 | 广州至真信息科技有限公司 | A kind of method of data synchronization, device |
| CN110427426A (en) * | 2019-08-02 | 2019-11-08 | 中国工商银行股份有限公司 | A kind of data synchronizing processing method and device |
| CN111177173A (en) * | 2020-01-02 | 2020-05-19 | 普元信息技术股份有限公司 | System and method for realizing data synchronization optimization processing under big data environment |
| CN111274257A (en) * | 2020-01-20 | 2020-06-12 | 山东汇贸电子口岸有限公司 | Real-time synchronization method and system based on data |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112783981A (en) * | 2021-02-05 | 2021-05-11 | 腾讯科技（深圳）有限公司 | Method and device for processing and monitoring data in reference station network |
| CN112783981B (en) * | 2021-02-05 | 2022-04-08 | 腾讯科技（深圳）有限公司 | Method and device for processing and monitoring data in reference station network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8171171B2 (en) | Data synchronization method and system between devices | |
| CN110347746B (en) | Heterogeneous database synchronous data consistency checking method and device | |
| CN105243067B (en) | A kind of method and device for realizing real-time incremental synchrodata | |
| US20070255763A1 (en) | Database replication method and system | |
| US7933296B2 (en) | Services for data sharing and synchronization | |
| US20100241764A1 (en) | Method and apparatus for synchronizing data | |
| CN110138588A (en) | Configuration file automatic management method and system, configuration management platform and client | |
| CN111209344A (en) | Data synchronization method and device | |
| US11194769B2 (en) | System and method for re-synchronizing a portion of or an entire source database and a target database | |
| CN113986873A (en) | Massive Internet of things data modeling processing, storing and sharing method | |
| US20110167041A1 (en) | Method and device for maintaining a changelog in data synchronization | |
| CN112364049B (en) | Data synchronization script generation method, system, terminal and storage medium | |
| CN112000681A (en) | Synchronization method capable of adapting mode change | |
| US20050235161A1 (en) | Method to reduce input parameter interface error and inconsistency for servlets | |
| CN102594874B (en) | Synchronization processing method and device | |
| US11941023B2 (en) | System and method for implementing incremental data comparison | |
| CN115687503A (en) | Method, device and equipment for synchronizing data among databases and storage medium | |
| CN113568921A (en) | Multi-person cooperative operation method for geographic information data production and update | |
| CN112559495A (en) | System for supporting multi-system label mapping to realize unified label management | |
| CN117076574A (en) | Method and device capable of arranging multiple data sources for synchronous aggregation of data | |
| CN115361399B (en) | Multi-terminal data synchronization method, device and system | |
| US8230444B2 (en) | Global attribute uniqueness (GAU) using an ordered message service (OMS) | |
| US20190065327A1 (en) | Efficient versioned object management | |
| CN111090648A (en) | Relational database data synchronization conflict resolution method | |
| CN113127521B (en) | Full cache scheme and system under distributed system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201127 |
|
| RJ01 | Rejection of invention patent application after publication |