CN111177254A - Method and device for data synchronization between heterogeneous relational databases - Google Patents
Method and device for data synchronization between heterogeneous relational databases Download PDFInfo
- Publication number
- CN111177254A CN111177254A CN201911233936.8A CN201911233936A CN111177254A CN 111177254 A CN111177254 A CN 111177254A CN 201911233936 A CN201911233936 A CN 201911233936A CN 111177254 A CN111177254 A CN 111177254A
- Authority
- CN
- China
- Prior art keywords
- database
- destination
- language content
- content
- source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Abstract
The invention relates to the technical field of data synchronization, and provides a method and a device for data synchronization between heterogeneous relational databases. The method comprises the steps that a synchronous server obtains a source end database log; the synchronous server converts the database log into database language content supported by a destination terminal according to the subscription information corresponding to the source terminal database, and stores the database language content in a corresponding sector; and the synchronous server sends a notification message of updating the log content to the subscriber so that the destination end can acquire the log content. The invention provides a subscription mechanism of a destination terminal through the arranged synchronous server, so that the corresponding heterogeneous database synchronization process can complete the conversion of subscribed data aiming at different database types in advance on the synchronous server, thereby ensuring the real-time property and reducing the occupation of processing resources under the condition of properly sacrificing the storage space.
Description
[ technical field ] A method for producing a semiconductor device
The invention relates to the technical field of data synchronization, in particular to a method and a device for data synchronization between heterogeneous relational databases.
[ background of the invention ]
In the current internet era, the data access demand of many enterprises is rapidly increased, and a single core database cannot support larger and larger concurrent operations, so that many enterprises establish a plurality of cross-regional target databases and transfer a large amount of business data from the core database to the corresponding target databases. In order to ensure the normal operation of enterprise business, the data synchronization between the core database and the target database of each region needs to be realized, and the integrity and consistency of the data are ensured. On the other hand, with the development of business, part of enterprises will upgrade the core business system, the upgrade process will involve the replacement of the database, during the period from the completion of the upgrade to the stable operation of the core system, the database in the original system will often be reserved as a backup database for a period of time, and during this period, data synchronization between the new database and the old database needs to be realized, so as to prevent the new system from being quickly switched back to the old system when a problem occurs, thereby ensuring that the business can continue to operate.
There are two conventional data synchronization schemes, the first: data synchronization mainly includes mutual data synchronization between a master server and a slave server in the same database cluster, however, when data synchronization is performed in different database clusters, normal operation of enterprise business is affected due to the fact that real-time data synchronization cannot be achieved; and the second method comprises the following steps: structured data synchronization methods based on database logs, such as oracle golden gate. The method can solve the problem that the real-time synchronization cannot be realized in the first method, but the method occupies the computing resources and disk IO resources of the source-end database due to the need of analyzing the database logs, so that certain influence is generated on the performance of the source-end database, especially when the deployed synchronization architecture is one-to-many, the influence on the source-end database is more obvious, and the overall synchronization performance is lower.
In view of the above, overcoming the drawbacks of the prior art is an urgent problem in the art.
[ summary of the invention ]
The technical problem to be solved by the invention is that an effective data synchronization method under a heterogeneous database scene is lacked in the prior art; the problem that computing resources and disk IO resources of a source-end database are occupied due to the fact that either real-time performance is lacked or database logs need to be analyzed temporarily in two synchronization modes in the prior art is solved.
The technical problem to be further solved by the present invention is how to further address the difference in the time when different destinations generate subscriptions, and how to provide corresponding subscription content for the new destination when the subscribed information stored in the synchronization server has been deleted in case that the subscription request may be completed after the previous subscription synchronization is completed.
The invention adopts the following technical scheme:
in a first aspect, the present invention provides a method for data synchronization between heterogeneous relational databases, including:
a synchronous server acquires a source end database log;
the synchronous server converts the database log into database language content supported by a destination terminal according to the subscription information corresponding to the source terminal database, and stores the database language content in a corresponding sector;
and the synchronous server sends a notification message of updating the log content to the subscriber so that the destination end can acquire the log content.
Preferably, the synchronous server stores subscription information, where the subscription information includes database types of one or more destination terminals subscribing to the source terminal database, and then the synchronous server converts a database log into database language content supported by a destination terminal according to the subscription information corresponding to the source terminal database, and stores the database language content in a corresponding sector, specifically including:
the synchronous server converts the acquired source database log into database language content supported by a destination end according to the database type of the destination end subscribing the source database;
when the data is stored in the corresponding sector, a counting item is added to the language content of the database supported by the corresponding converted destination according to the number of the subscribed destinations of the database of the same type; and the counting item is updated after the database language content acquisition of a destination end is finished each time, and when the state of the counting item is matched with the number of the destination ends, the corresponding type of database language content in the corresponding sector is deleted.
Preferably, the database log is converted into the database language content supported by the destination according to the subscription information corresponding to the source database, and is stored in the corresponding sector, specifically including:
and for one or more items of the importance level of the destination terminal, the total amount of the subscribed data and the number of subscribed destination terminal devices contained in the subscription information, storing the database language content converted into the language content supported by the destination terminal in hard disk partitions with different reading speeds.
Preferably, the hard disk partitions with different reading speeds specifically include:
dividing the hard disk into a plurality of subareas according to concentric circles with different diameters; wherein a larger diameter partition corresponds to a faster read speed than a smaller diameter partition.
Preferably, the subscription information of each source database and the destination database is periodically scanned, and then the method further includes:
if the number of the databases of the subscriber destinations identifying the same source-end database is reduced and exceeds a preset threshold value, the corresponding database language content is converted into the storage position of the database language content supported by the destination end, and the hard disk partition with the initial speed is migrated to the hard disk with the reading speed reduced by one level;
if the number of the databases of the subscribers who identify the same source-end database is increased to exceed the preset threshold value, the corresponding database language content is converted into the storage position of the database language content supported by the destination end, and the hard disk partition with the initial speed is migrated to the hard disk with the reading speed increased by one level.
Preferably, if the first database language content stored by the synchronization server and converted into the content supported by the destination is deleted, the synchronization server receives a new subscription request of the destination, and the subscribed content is the same as the deleted first database language content, the method further includes:
the synchronous server sends reverse subscription requests to one or more destination terminals which also subscribe the first database language content before deleting the first database language content;
wherein, the reverse subscription request comprises the related information of the language content of the first database;
and after receiving the reverse subscription request, if the one or more destination terminals confirm that the locally stored corresponding language content of the first database is modified by the language of the related database, returning a reverse subscription response to the synchronization server so that the synchronization server can select the destination terminal from the language content of the first database to obtain the language content of the first database.
Preferably, the information related to the language content of the first database includes:
one or more of a database ID, a table name, one or more database operation languages, a start time, and an end time.
Preferably, the database type of the destination includes one or more of a relational database Oracle, DB2, Microsoft SQLServer, and MySQL.
In a second aspect, the present invention further provides an apparatus for synchronizing data between heterogeneous relational databases, where the apparatus is configured to implement the method for synchronizing data between heterogeneous relational databases in the first aspect, and the apparatus includes:
at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor and programmed to perform the method for data synchronization between heterogeneous relational databases of the first aspect.
In a third aspect, the present invention also provides a non-transitory computer storage medium storing computer-executable instructions for execution by one or more processors for performing the method for data synchronization between heterogeneous relational databases according to the first aspect.
The method for data synchronization between heterogeneous relational databases provided by the invention provides a subscription mechanism of a destination end through the arranged synchronization server, so that the corresponding heterogeneous database synchronization process can be completed on the synchronization server in advance aiming at the conversion of subscribed data of different database types, and the real-time property is ensured and the occupation of processing resources is reduced under the condition of properly sacrificing the storage space. Moreover, in consideration of the real scene, the types of databases actually involved are very limited, so that the conversion of subscribed data for different database types is completed in advance, and the waste of too much storage resources is not caused.
Furthermore, the invention sets technical items by subscribing the type attribution of the destination terminal, thereby greatly compressing the use of the storage space of the synchronous server in the application scene that the destination terminal is mostly database of the same type.
Furthermore, the invention also provides a technical solution of reverse subscription, so that after a certain subscription destination end misses the synchronization content generated in the synchronization server, the corresponding synchronization content can be strived for based on the synchronization server, and the robustness of the whole scheme architecture is improved.
[ description of the drawings ]
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings described below are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
FIG. 1 is a block diagram illustrating an architecture for data synchronization between heterogeneous relational databases according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for data synchronization between heterogeneous relational databases according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of partitioned storage for data synchronization between heterogeneous relational databases according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a many-to-one scenario for data synchronization between heterogeneous relational databases according to an embodiment of the present invention;
fig. 5 is a schematic view of a scenario of data cross synchronization between heterogeneous relational databases according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an apparatus for data synchronization between heterogeneous relational databases according to an embodiment of the present invention.
[ detailed description ] embodiments
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In the description of the present invention, the terms "inner", "outer", "longitudinal", "lateral", "upper", "lower", "top", "bottom", and the like indicate orientations or positional relationships based on those shown in the drawings, and are for convenience only to describe the present invention without requiring the present invention to be necessarily constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.
In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
Example 1:
in step 201, the synchronization server obtains a source database log.
The source database log records all database operation actions of the source, so that for the embodiment of the invention, if the corresponding synchronization process is performed from the beginning of the operation of the source database, the source database log represents that the source data content is comprehensively backed up and synchronized.
In step 202, the synchronization server converts the database log into the database language content supported by the destination according to the subscription information corresponding to the source database, and stores the database language content in the corresponding sector.
The database type of the destination end comprises one or more of a relational database Oracle, DB2, Microsoft SQLServer and MySQL.
Still taking fig. 1 as an example, the subscriber and the destination database have a one-to-one correspondence, that is: the oracle has an oracle log receiving module and an oracle log parsing module, and the mysql has a mysql log receiving module and a mysql log parsing module, etc. (the reason why the one-to-one correspondence is needed here is that since some differences exist in sql syntax among databases, if the same log parsing module is used, the parsed sql can run on the mysql but cannot run on the oracle). The log receiving module is responsible for reading database log data sent by the source end from the log capturing module, then the log analyzing module is responsible for analyzing the log data, converting the log data into corresponding sql statements, and then executing the sql to write the data into a corresponding destination end database.
In step 203, the synchronization server sends a notification message of log content update to the subscriber, so that the destination can obtain the log content.
The method for data synchronization between heterogeneous relational databases provided by the invention provides a subscription mechanism of a destination end through the arranged synchronization server, so that the corresponding heterogeneous database synchronization process can be completed on the synchronization server in advance aiming at the conversion of subscribed data of different database types, and the real-time property is ensured and the occupation of processing resources is reduced under the condition of properly sacrificing the storage space. Moreover, in consideration of the real scene, the types of databases actually involved are very limited, so that the conversion of subscribed data for different database types is completed in advance, and the waste of too much storage resources is not caused.
In this embodiment of the present invention, the synchronization Server stores subscription information, where the subscription information includes a database type of one or more destination terminals subscribing to the source terminal database (for example, the database type of the destination terminal includes one or more of a relational database Oracle, DB2, Microsoft SQL Server, and MySQL), and then the synchronization Server converts a database log into a database language content supported by the destination terminal according to the subscription information corresponding to the source terminal database, and stores the database language content in a corresponding sector, specifically including:
the synchronous server converts the acquired source database log into database language content supported by a destination end according to the database type of the destination end subscribing the source database;
when the data is stored in the corresponding sector, a counting item is added to the language content of the database supported by the corresponding converted destination according to the number of the subscribed destinations of the database of the same type; and the counting item is updated after the database language content acquisition of a destination end is finished each time, and when the state of the counting item is matched with the number of the destination ends, the corresponding type of database language content in the corresponding sector is deleted.
In the preferred scheme of the embodiment of the invention, the technical items are set by subscribing the type attribution of the destination terminal, so that the use of the storage space of the synchronous server in the application scene that the destination terminal is mostly the database of the same type is greatly compressed.
In the implementation process of the present invention, for the subscription request of the destination, corresponding policy characteristics may also be provided, that is, the destination may specify the subscription content of the data content modified at the source, may also specify the subscription content of the source data operation in a specific time period (herein, usually, a future time period), may also specify the subscription content of the data table that is modified in relation, and so on. It usually appears to the synchronization server that a storage area is newly opened up to provide storage of the data to be synchronized for the destination of the customization request. Further, in the scenario of differentiated storage of different read sectors, which is provided later in conjunction with the embodiment of the present invention, the destination of the customized request provides the default data to be synchronized to be stored in the sector with low read speed, because it is usually a specific requirement of the destination of the small population.
In this embodiment of the present invention, in order to optimally improve the synchronization efficiency of the subscription content, the database log is converted into the database language content supported by the destination according to the subscription information corresponding to the source database, and the database language content is stored in the corresponding sector, specifically including:
and for one or more items of the importance level of the destination terminal, the total amount of the subscribed data and the number of subscribed destination terminal devices contained in the subscription information, storing the database language content converted into the language content supported by the destination terminal in hard disk partitions with different reading speeds. The hard disk partitions with different reading speeds specifically include: dividing the hard disk into a plurality of subareas according to concentric circles with different diameters; wherein a larger diameter partition corresponds to a faster read speed than a smaller diameter partition. As shown in fig. 3, it presents an effect schematic diagram of the three partitions storing the subscription content, where the read-write speeds of the partition 1, the partition 2, and the partition 3 decrease sequentially.
As shown in FIG. 4, when many-to-one synchronization is used, each topic contains multiple partitions. Assume that the source has 3 databases: database 1, database 2 and database 3, each having 3 tables: table1, table2, table 3. In a many-to-one scenario, data on the same table on each source library is generally required to be summarized to a destination, so that the subject here is still a table name, and the number of partitions under each subject is consistent with the number of source databases, so that the key here is database ID + table name. For example, an operation on table1 of database 1 may be written to partition 1 of topic 1, an operation on table1 of database 2 may be written to partition 2 of topic 1, and an operation on table1 of database n may be written to partition n of topic 1, and the operations of the other tables are the same. Unlike fig. 3, in fig. 3, the subscription content (e.g., subscription content 1.1) presented in each partition directly represents a subscription content object of one subscription task. While in fig. 4, the granularity of representation is smaller, it can be understood that messages (e.g., message 1, message 2) are specific database operation statements.
In the specific operation process, when the importance level of the destination is higher (for example, the VIP level of the destination), the total amount of subscribed data is small, and/or the number of subscribed destination devices is large,
in consideration of the fact that the content of the source-end database is unified, the subscription relationship may change, and especially in a scenario of performing one round of database upgrade or server replacement, the subscription relationship may be short-time; in addition, it is also considered that some subscription relationships may be time-efficient, that is, only for a certain period of subscription, the subscription relationship is abandoned after expiration; therefore, the different reading speed partition modes in which the subscription contents are stored are set in the similar number reference mode as described above. Preferably, a further improvement may be made, specifically, the subscription information of each source database and the destination database is periodically scanned, and then the method further includes:
if the number of the databases of the subscriber destinations identifying the same source-end database is reduced and exceeds a preset threshold value, the corresponding database language content is converted into the storage position of the database language content supported by the destination end, and the hard disk partition with the initial speed is migrated to the hard disk with the reading speed reduced by one level;
if the number of the databases of the subscribers who identify the same source-end database is increased to exceed the preset threshold value, the corresponding database language content is converted into the storage position of the database language content supported by the destination end, and the hard disk partition with the initial speed is migrated to the hard disk with the reading speed increased by one level. The preset threshold is an empirical value, and in the actual operation process, the considered dimension may be far more than the above number by one dimension, and the preset threshold here may be set according to a compromise parameter of the maximum subscription number and the minimum subscription number of the historical experience.
A normal subscription relationship is that, naturally, after the subscriber acquires the subscription content, the corresponding synchronization server deletes the corresponding subscription content stored in the synchronization server, thereby freeing up space for serving other subscription tasks. However, in the actual implementation process, after a round of subscription task is completed, a new destination initiates a subscription request that is the same as the completed subscription task, and at this time, the existing synchronization server cannot effectively solve the corresponding problem. If a subscription request is sent to the same source again, the corresponding synchronized data may have been deleted and the source-synchronized data content cannot be provided to the new destination. Based on such a problem scenario, an embodiment of the present invention further provides a preferred extension scheme, and specifically, if the first database language content stored by the synchronization server and converted to be supported by the destination is deleted, the synchronization server receives a new subscription request of the destination, and the subscribed content is the same as the deleted first database language content, where the method further includes:
the synchronous server sends reverse subscription requests to one or more destination terminals which also subscribe the first database language content before deleting the first database language content;
wherein, the reverse subscription request comprises the related information of the language content of the first database; the information related to the first database language content comprises: one or more of a database ID, a table name, one or more database operation languages, a start time, and an end time.
And after receiving the reverse subscription request, if the one or more destination terminals confirm that the locally stored corresponding language content of the first database is modified by the language of the related database, returning a reverse subscription response to the synchronization server so that the synchronization server can select the destination terminal from the language content of the first database to obtain the language content of the first database.
In the above, the embodiment of the present invention provides a technical solution for reverse subscription, so that after a certain subscription destination misses a synchronization content generated in a synchronization server, a corresponding synchronization content can be obtained based on the synchronization server, and robustness of the whole scheme architecture is improved.
Example 2:
the embodiment of the present invention takes fig. 5 as an example, which lists a cross-type subscription relationship, and compared with embodiment 1, the applicable scenario is directly presented by way of an example, and the specific subscription relationship establishment and subscription synchronization and storage content management may use the content described in embodiment 1 for reference, and will not be repeatedly described in this embodiment.
In the embodiment of the present invention, there are 2 tables, table1 and table2 (which may be from one source end or from different source ends), and each table has 1 partition. The destination has 2 subscriber groups (which can be understood as being composed of a plurality of subscribers), wherein the subscriber group 1 is composed of 2 oracle subscribers, and the subscriber group 2 is composed of 2 mysql subscribers. the messages in the partitions on the table1 are respectively subscribed by an oracle subscriber 1 and a mysql subscriber 2, each subscriber analyzes the data and then writes the data into the corresponding database, and similarly, the table2 is consumed by the oracle subscriber 2 and the mysql subscriber 1 and then writes the data into the databases, so that the data of the table1 and the table2 are written into the oracle and the mysql simultaneously. The same applies when there are multiple partitioned scenes.
Example 3:
fig. 6 is a schematic structural diagram of a content recommendation device based on human body status according to an embodiment of the present invention. The human body state-based content recommendation apparatus of the present embodiment includes one or more processors 21 and a memory 22. In fig. 6, one processor 21 is taken as an example.
The processor 21 and the memory 22 may be connected by a bus or other means, such as the bus connection in fig. 6.
The memory 22, which is a non-volatile computer-readable storage medium, can be used for storing a non-volatile software program and a non-volatile computer-executable program, such as the method for data synchronization between heterogeneous relational databases in embodiment 1. The processor 21 performs a method of data synchronization between heterogeneous relational databases by executing non-volatile software programs and instructions stored in the memory 22.
The memory 22 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 22 may optionally include memory located remotely from the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The program instructions/modules are stored in the memory 22 and when executed by the one or more processors 21, perform the method for data synchronization between heterogeneous relational databases in the above embodiment 1, for example, perform the steps shown in fig. 2 and described above.
It should be noted that, for the information interaction, execution process and other contents between the modules and units in the apparatus and system, the specific contents may refer to the description in the embodiment of the method of the present invention because the same concept is used as the embodiment of the processing method of the present invention, and are not described herein again.
Those of ordinary skill in the art will appreciate that all or part of the steps of the various methods of the embodiments may be implemented by associated hardware as instructed by a program, which may be stored on a computer-readable storage medium, which may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic or optical disk, or the like.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (9)
1. A method for synchronizing data between heterogeneous relational databases, comprising:
a synchronous server acquires a source end database log;
the synchronous server converts the database log into database language content supported by a destination terminal according to the subscription information corresponding to the source terminal database, and stores the database language content in a corresponding sector;
and the synchronous server sends a notification message of updating the log content to the subscriber so that the destination end can acquire the log content.
2. The method of claim 1, wherein the synchronization server stores subscription information, the subscription information including database types of one or more destinations subscribing to the source database, and the synchronization server converts a database log into database language content supported by a destination according to the subscription information corresponding to the source database and stores the database language content in a corresponding sector, specifically comprising:
the synchronous server converts the acquired source database log into database language content supported by a destination end according to the database type of the destination end subscribing the source database;
when the data is stored in the corresponding sector, a counting item is added to the language content of the database supported by the corresponding converted destination according to the number of the subscribed destinations of the database of the same type; and the counting item is updated after the database language content acquisition of a destination end is finished each time, and when the state of the counting item is matched with the number of the destination ends, the corresponding type of database language content in the corresponding sector is deleted.
3. The method for data synchronization between heterogeneous relational databases according to claim 1, wherein the database logs are converted into database language contents supported by a destination according to subscription information corresponding to the source database, and are stored in corresponding sectors, specifically including:
and for one or more items of the importance level of the destination terminal, the total amount of the subscribed data and the number of subscribed destination terminal devices contained in the subscription information, storing the database language content converted into the language content supported by the destination terminal in hard disk partitions with different reading speeds.
4. The method for data synchronization between heterogeneous relational databases according to claim 3, wherein the hard disk partitions with different reading speeds specifically comprise:
dividing the hard disk into a plurality of subareas according to concentric circles with different diameters; wherein a larger diameter partition corresponds to a faster read speed than a smaller diameter partition.
5. The method of claim 3, wherein subscription information of each source database and destination database is periodically scanned, and the method further comprises:
if the number of the databases of the subscriber destinations identifying the same source-end database is reduced and exceeds a preset threshold value, the corresponding database language content is converted into the storage position of the database language content supported by the destination end, and the hard disk partition with the initial speed is migrated to the hard disk with the reading speed reduced by one level;
if the number of the databases of the subscribers who identify the same source-end database is increased to exceed the preset threshold value, the corresponding database language content is converted into the storage position of the database language content supported by the destination end, and the hard disk partition with the initial speed is migrated to the hard disk with the reading speed increased by one level.
6. The method for data synchronization between heterogeneous relational databases according to claim 2, wherein if the first database language content stored by the synchronization server and converted into the content supported by the destination is deleted, the synchronization server receives a new subscription request from the destination, and the subscribed content is the same as the deleted first database language content, the method further comprising:
the synchronous server sends reverse subscription requests to one or more destination terminals which also subscribe the first database language content before deleting the first database language content;
wherein, the reverse subscription request comprises the related information of the language content of the first database;
and after receiving the reverse subscription request, if the one or more destination terminals confirm that the locally stored corresponding language content of the first database is modified by the language of the related database, returning a reverse subscription response to the synchronization server so that the synchronization server can select the destination terminal from the language content of the first database to obtain the language content of the first database.
7. The method for data synchronization between heterogeneous relational databases according to claim 6, wherein the information related to the language content of the first database comprises:
one or more of a database ID, a table name, one or more database operation languages, a start time, and an end time.
8. The method of data synchronization between heterogeneous relational databases according to claim 1, wherein the database type of the destination comprises one or more of relational database Oracle, DB2, Microsoft SQL Server and MySQL.
9. An apparatus for data synchronization between heterogeneous relational databases, the apparatus comprising:
at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor and programmed to perform the method for data synchronization between heterogeneous relational databases of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911233936.8A CN111177254B (en) | 2019-12-05 | 2019-12-05 | Method and device for data synchronization between heterogeneous relational databases |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911233936.8A CN111177254B (en) | 2019-12-05 | 2019-12-05 | Method and device for data synchronization between heterogeneous relational databases |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111177254A true CN111177254A (en) | 2020-05-19 |
| CN111177254B CN111177254B (en) | 2021-08-17 |
Family
ID=70624125
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911233936.8A Active CN111177254B (en) | 2019-12-05 | 2019-12-05 | Method and device for data synchronization between heterogeneous relational databases |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111177254B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112000669A (en) * | 2020-08-14 | 2020-11-27 | 中科三清科技有限公司 | Environment monitoring data processing method and device, storage medium and terminal |
| CN113709250A (en) * | 2021-08-31 | 2021-11-26 | 中国电子科技集团公司第二十八研究所 | Cross-domain user data synchronization method based on subscription sending mode |
| CN113779138A (en) * | 2021-02-04 | 2021-12-10 | 北京京东振世信息技术有限公司 | Order management method and device |
| CN115017223A (en) * | 2022-08-04 | 2022-09-06 | 成都运荔枝科技有限公司 | System supporting large data volume import and export |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040240480A1 (en) * | 2003-05-30 | 2004-12-02 | Hiben Bradley M. | Method for selecting an operating mode based on a detected synchronization pattern |
| US20080208923A1 (en) * | 2007-01-10 | 2008-08-28 | Satoru Watanabe | Method for verifying data consistency of backup system, program and storage medium |
| CN102291416A (en) * | 2011-09-14 | 2011-12-21 | 成都软智科技有限公司 | Two-way synchronizing method and system of client-side and server-side |
| CN102741829A (en) * | 2009-12-30 | 2012-10-17 | 赛贝斯股份有限公司 | Message based synchronization for mobile business objects |
| US20140156595A1 (en) * | 2012-11-30 | 2014-06-05 | Metaswitch Networks Ltd. | Synchronisation system and method |
| CN105824865A (en) * | 2015-12-09 | 2016-08-03 | 广东亿迅科技有限公司 | Distributed type database data synchronization system |
| CN107180116A (en) * | 2017-06-28 | 2017-09-19 | 努比亚技术有限公司 | A kind of data synchronizing processing method, mobile terminal and computer-readable recording medium |
| CN108200219A (en) * | 2018-03-13 | 2018-06-22 | 广东欧珀移动通信有限公司 | Method of data synchronization, device, server and storage medium |
| CN108228621A (en) * | 2016-12-15 | 2018-06-29 | 上海祈贝健康管理咨询有限公司 | A kind of method of strange land real-time synchronization SQL data |
| US20180308139A1 (en) * | 2015-10-20 | 2018-10-25 | Websoft Limited | Systems And Methods For Auction Customer Engagement Platform |
| CN108763234A (en) * | 2018-02-01 | 2018-11-06 | 宝付网络科技（上海）有限公司 | A kind of real time data synchronization method and system |
| CN109522361A (en) * | 2018-09-27 | 2019-03-26 | 中国经济信息社有限公司 | A kind of method and system carrying out database real-time synchronization using messaging bus |
| CN109714392A (en) * | 2018-11-26 | 2019-05-03 | 聚好看科技股份有限公司 | Across the computer room method of data synchronization of one kind and device |
| CN109960710A (en) * | 2019-01-16 | 2019-07-02 | 平安科技（深圳）有限公司 | Method of data synchronization and system between database |
| CN110083660A (en) * | 2019-04-29 | 2019-08-02 | 重庆天蓬网络有限公司 | A kind of method, apparatus of synchrodata, medium and electronic equipment |
| CN110532247A (en) * | 2019-08-28 | 2019-12-03 | 北京皮尔布莱尼软件有限公司 | Data migration method and data mover system |
-
2019
- 2019-12-05 CN CN201911233936.8A patent/CN111177254B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040240480A1 (en) * | 2003-05-30 | 2004-12-02 | Hiben Bradley M. | Method for selecting an operating mode based on a detected synchronization pattern |
| US20080208923A1 (en) * | 2007-01-10 | 2008-08-28 | Satoru Watanabe | Method for verifying data consistency of backup system, program and storage medium |
| CN102741829A (en) * | 2009-12-30 | 2012-10-17 | 赛贝斯股份有限公司 | Message based synchronization for mobile business objects |
| CN102291416A (en) * | 2011-09-14 | 2011-12-21 | 成都软智科技有限公司 | Two-way synchronizing method and system of client-side and server-side |
| US20140156595A1 (en) * | 2012-11-30 | 2014-06-05 | Metaswitch Networks Ltd. | Synchronisation system and method |
| US20180308139A1 (en) * | 2015-10-20 | 2018-10-25 | Websoft Limited | Systems And Methods For Auction Customer Engagement Platform |
| CN105824865A (en) * | 2015-12-09 | 2016-08-03 | 广东亿迅科技有限公司 | Distributed type database data synchronization system |
| CN108228621A (en) * | 2016-12-15 | 2018-06-29 | 上海祈贝健康管理咨询有限公司 | A kind of method of strange land real-time synchronization SQL data |
| CN107180116A (en) * | 2017-06-28 | 2017-09-19 | 努比亚技术有限公司 | A kind of data synchronizing processing method, mobile terminal and computer-readable recording medium |
| CN108763234A (en) * | 2018-02-01 | 2018-11-06 | 宝付网络科技（上海）有限公司 | A kind of real time data synchronization method and system |
| CN108200219A (en) * | 2018-03-13 | 2018-06-22 | 广东欧珀移动通信有限公司 | Method of data synchronization, device, server and storage medium |
| CN109522361A (en) * | 2018-09-27 | 2019-03-26 | 中国经济信息社有限公司 | A kind of method and system carrying out database real-time synchronization using messaging bus |
| CN109714392A (en) * | 2018-11-26 | 2019-05-03 | 聚好看科技股份有限公司 | Across the computer room method of data synchronization of one kind and device |
| CN109960710A (en) * | 2019-01-16 | 2019-07-02 | 平安科技（深圳）有限公司 | Method of data synchronization and system between database |
| CN110083660A (en) * | 2019-04-29 | 2019-08-02 | 重庆天蓬网络有限公司 | A kind of method, apparatus of synchrodata, medium and electronic equipment |
| CN110532247A (en) * | 2019-08-28 | 2019-12-03 | 北京皮尔布莱尼软件有限公司 | Data migration method and data mover system |
Non-Patent Citations (1)
| Title |
|---|
| 杨跃武 等: "分布式数据库同步更新的实现方法", 《佛山科学技术学院学报(自然科学版)》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112000669A (en) * | 2020-08-14 | 2020-11-27 | 中科三清科技有限公司 | Environment monitoring data processing method and device, storage medium and terminal |
| CN113779138A (en) * | 2021-02-04 | 2021-12-10 | 北京京东振世信息技术有限公司 | Order management method and device |
| CN113709250A (en) * | 2021-08-31 | 2021-11-26 | 中国电子科技集团公司第二十八研究所 | Cross-domain user data synchronization method based on subscription sending mode |
| CN113709250B (en) * | 2021-08-31 | 2023-09-15 | 中国电子科技集团公司第二十八研究所 | Cross-domain user data synchronization method based on subscription transmission mode |
| CN115017223A (en) * | 2022-08-04 | 2022-09-06 | 成都运荔枝科技有限公司 | System supporting large data volume import and export |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111177254B (en) | 2021-08-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111177254B (en) | Method and device for data synchronization between heterogeneous relational databases | |
| CN110147411A (en) | Method of data synchronization, device, computer equipment and storage medium | |
| CN111367984B (en) | Method and system for loading high-timeliness data into data lake | |
| CN107402722B (en) | Data migration method and storage device | |
| CN109271435B (en) | Data extraction method and system supporting breakpoint continuous transmission | |
| US9836516B2 (en) | Parallel scanners for log based replication | |
| CN112286941B (en) | Big data synchronization method and device based on Binlog + HBase + Hive | |
| US20190384775A1 (en) | Active Transaction List Synchronization Method and Apparatus | |
| JP6225262B2 (en) | System and method for supporting partition level journaling to synchronize data in a distributed data grid | |
| CN104113597A (en) | Multi- data-centre hadoop distributed file system (HDFS) data read-write system and method | |
| CN105472002A (en) | Session synchronization method based on instant copying among cluster nodes | |
| CN111177159B (en) | Data processing system and method and data updating equipment | |
| CN103870570A (en) | HBase (Hadoop database) data usability and durability method based on remote log backup | |
| CN110990432A (en) | Device and method for synchronizing distributed cache clusters across machine rooms | |
| CN111177161A (en) | Data processing method and device, computing equipment and storage medium | |
| CN105138679A (en) | Data processing system and method based on distributed caching | |
| CN111694863B (en) | Database cache refreshing method, system and device | |
| CN115146002A (en) | Cross-data-center data synchronization method and device | |
| CN108090056B (en) | Data query method, device and system | |
| CN109657000B (en) | Real-time data synchronization method and device of rail transit comprehensive monitoring system | |
| CN105407044A (en) | Method for implementing cloud storage gateway system based on network file system (NFS) | |
| CN113438275B (en) | Data migration method and device, storage medium and data migration equipment | |
| CN116304390B (en) | Time sequence data processing method and device, storage medium and electronic equipment | |
| CN116991858A (en) | Acquisition method, acquisition device and data acquisition system for changed data | |
| US10860580B2 (en) | Information processing device, method, and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province Applicant after: Wuhan dream database Co.,Ltd. Address before: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province Applicant before: WUHAN DAMENG DATABASE Co.,Ltd. |
|
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Feng Yuan Inventor after: Yu Yuanlan Inventor after: Song Tianyang Inventor before: Fu Quan Inventor before: Feng Yuan Inventor before: Yu Yuanlan Inventor before: Song Tianyang |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220916 Address after: 100000 901, floor 9, building 5, courtyard 8, KEGU 1st Street, Beijing Economic and Technological Development Zone, Daxing District, Beijing Patentee after: Beijing Dameng Database Technology Co.,Ltd. Address before: 430000 16-19 / F, building C3, future technology building, 999 Gaoxin Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province Patentee before: Wuhan dream database Co.,Ltd. |