CN108664659A - A kind of method of data synchronization and device of Distributed Heterogeneous Database - Google Patents
A kind of method of data synchronization and device of Distributed Heterogeneous Database Download PDFInfo
- Publication number
- CN108664659A CN108664659A CN201810487831.4A CN201810487831A CN108664659A CN 108664659 A CN108664659 A CN 108664659A CN 201810487831 A CN201810487831 A CN 201810487831A CN 108664659 A CN108664659 A CN 108664659A
- Authority
- CN
- China
- Prior art keywords
- database
- source
- data
- change sequence
- object table
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 98
- 230000008859 change Effects 0.000 claims abstract description 94
- 230000009467 reduction Effects 0.000 claims abstract description 37
- 238000013506 data mapping Methods 0.000 claims abstract description 31
- 238000012795 verification Methods 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 13
- 238000013507 mapping Methods 0.000 claims description 11
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 5
- 238000003780 insertion Methods 0.000 description 8
- 230000037431 insertion Effects 0.000 description 8
- 230000001360 synchronised effect Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000033772 system development Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of method of data synchronization of Distributed Heterogeneous Database and device, this method includes：Establish the data mapping relations in source table and target database between object table in source database；If capturing change sequence from the table of source, reduction treatment is carried out to change sequence according to data mapping relations, obtains the SQL structured query language sentences of object table；Change sequence is distributed to object table from source table, and operation is updated to object table using SQL statement and change sequence.The above a kind of method of data synchronization and device of Distributed Heterogeneous Database provided by the invention, the data that this method can be completed between distributed data base based on SQL reduction methods synchronize, and since SQL statement meets SQL92 standards, data can be made to synchronize and have more validity.
Description
Technical field
The present invention relates to computer database technology fields, more specifically to a kind of Distributed Heterogeneous Database
Method of data synchronization and device.
Background technology
With the rapid development of information age, the use of database is more and more extensive, but because of system development environment and
The difference of type of database produces many relatively independent Distributed Heterogeneous Databases.
Since many data are associated between Distributed Heterogeneous Database, therefore, in order to ensure that each number of nodes in database
According to consistent, data synchronization problems when urgent need to resolve is distributed between heterogeneous database.
Invention content
To solve the above problems, the application provides a kind of method of data synchronization and device of Distributed Heterogeneous Database, skill
Art scheme is as follows：
A kind of method of data synchronization of distributed data base, including：
Establish the data mapping relations in source table and target database between object table in source database；
If capturing change sequence from the source table, according to the data mapping relations to the change sequence into
Row reduction treatment obtains the SQL structured query language sentences of the object table；
The change sequence is distributed to the object table from the source table, and utilizes the SQL statement and the variation
Object table described in sequence pair is updated operation.
Preferably, the data mapping relations established in source database in source table and target database between object table,
Including：
The first log-on message of source database and the second log-on message of target database are obtained respectively；
It is driven using presetting database according to first log-on message and type and version is carried out to the source database
Verification, and according to second log-on message using the presetting database drive to the target database carry out type and
The verification of version；
The source database and the target database by verification in the case of, the source table in the source database
And the mapping relations between data type and/or reserved word and/or operator are established in the target database between object table.
Preferably, the process that change sequence is captured from the source table, including：
The change sequence of the source table is captured using trigger hair or Log File Analysis.
Preferably, described that reduction treatment is carried out to the change sequence according to the data mapping relations, including：
Judge the field with the presence or absence of multiple media types in the change sequence；
If it is not, being based on SQL reduction methods and the data mapping relations, reduction treatment is carried out to the change sequence.
Preferably, further include：
If so, the DML grammers based on the target database and the data mapping relations, carry out the change sequence
Reduction treatment.
Preferably, described that operation is updated to the object table using the SQL statement and the change sequence, it wraps
It includes：
The change sequence is handled using the SQL statement；
Using treated, the change sequence is updated operation to the object table.
A kind of data synchronization unit of distributed data base, including：
Relationship establishes module, for establishing the data mapping in source database in source table and target database between object table
Relationship；
If data restoring module is mapped according to the data and is closed for capturing change sequence from the source table
System carries out reduction treatment to the change sequence, obtains the SQL structured query language sentences of the object table；
Update module for the change sequence to be distributed to the object table from the source table, and utilizes the SQL languages
Sentence and the change sequence are updated operation to the object table.
Preferably, the relationship establishes module, is specifically used for：
The first log-on message of source database and the second log-on message of target database are obtained respectively；According to described
One log-on message drives the verification that type and version are carried out to the source database using presetting database, and according to described the
Two log-on messages drive the verification that type and version are carried out to the target database using the presetting database；In the source
Database and the target database by verification in the case of, source table and the target database in the source database
The mapping relations between data type and/or reserved word and/or operator are established between middle object table.
Preferably, it is used to carry out the data of reduction treatment also to the change sequence according to the data mapping relations
Former module, is specifically used for：
Judge the field with the presence or absence of multiple media types in the change sequence；If it is not, being based on SQL reduction methods and the number
According to mapping relations, reduction treatment is carried out to the change sequence.
Preferably, the institute for being updated operation to the object table using the SQL statement and the change sequence
Update module is stated, is specifically used for：
The change sequence is handled using the SQL statement；Using treated, the change sequence is to described
Object table is updated operation.
Compared to the prior art, what the present invention realized has the beneficial effect that：
The above a kind of method of data synchronization and device of Distributed Heterogeneous Database provided by the invention, this method can be with base
The data between distributed data base are completed in SQL reduction methods to synchronize, and since SQL statement meets SQL92 standards, it can
So that data, which synchronize, has more validity.
Description of the drawings
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is the method flow diagram of the method for data synchronization of distributed data base provided in an embodiment of the present invention；
Fig. 2 is the Part Methods flow chart of the method for data synchronization of distributed data base provided in an embodiment of the present invention；
Fig. 3 is another part method flow of the method for data synchronization of distributed data base provided in an embodiment of the present invention
Figure；
Fig. 4 is another Part Methods flow of the method for data synchronization of distributed data base provided in an embodiment of the present invention
Figure；
Fig. 5 is the structural schematic diagram of the data synchronization unit of distributed data base provided in an embodiment of the present invention.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Current most of databases all support SQL (Structured Query Language) 92 standards, " also based on SQL
Former method " simultaneous techniques variation capture the result is that according to SQL92 standards SQL statement form occur, any support can be given
The database of SQL92 standards synchronizes.If without the SQL statement of standard, corresponding database D ML operations must be just write, this
Sample is once must be known by being which kind of database, because the DML statement of disparate databases is not quite similar." being based on SQL reduction methods "
The thing done is exactly that the SQL statement of change operation is obtained from source database, and executing the SQL statement in target database implements more
Newly.
The embodiment of the present application provides a kind of method of data synchronization of distributed data base, and the method flow diagram of this method is as schemed
Shown in 1, include the following steps：
S10 establishes the data mapping relations in source table and target database between object table in source database；
In the embodiment of the present application, data mapping relations include any one in data type, reserved word and operator or more
The mapping relations of a combination.By taking data type is mapping relations as an example, the decimal types of Mysql correspond to Oracle's
Number is assured that how data are converted after the correspondence for establishing the two when synchronous.
During specific implementation, step S10 " is established in source database in source table and target database between object table
Following steps may be used in data mapping relations ", and method flow diagram is as shown in Figure 2：
S101 obtains the first log-on message of source database and the second log-on message of target database respectively；
In the present embodiment, the first log-on message includes server address, example, account relevant information of source database etc.；
Second log-on message includes the server address of target database, by the example used, by account relevant information used etc..
S102 is driven using presetting database according to the first log-on message and is tested source database progress type and version
Card, and the verification that type and version are carried out to target database is driven using presetting database according to the second log-on message；
System includes that frequently-used data library driver or non-common database-driven download network address.By reading registration table
It can determine that local data base driving whether there is, such as be not present, then will download network address from the driving configured is downloaded, so
Silent installation is carried out afterwards, to be registered to database-driven.
In the present embodiment, connecting test can be passed through in conjunction with the log-on message and the first log-on message of database-driven
Determine whether database-driven matches with source database, if connecting time-out database of descriptions driving with source database not
Match, conversely, then matching；If matching the version information for inquiring source database again；
In the present embodiment, connecting test can be passed through in conjunction with the log-on message and the second log-on message of database-driven
Determine database-driven whether with target data storehouse matching, if connect time-out if database of descriptions driving with target database not
Matching, conversely, then matching；If matching the version information for inquiring target database again.
S103, source database and target database by verification in the case of, source table and target in source database
The mapping relations between data type and/or reserved word and/or operator are established in database between object table.
S20 carries out also original place according to data mapping relations if capturing change sequence from the table of source to change sequence
Reason, obtains the SQL structured query language sentences of object table；
In the present embodiment, the change sequence of source table is captured using variation capture technique.Below to variation capture technique into
Row is simple to be introduced：
Variation capture technique is the process of the change sequence of capture source table.Due to most databases journal format not
It is open, it directly reads logic log and has any problem, so will be to the variation of source table in source database into line trace, capture.Carefully divide
Snapshot, trigger method, diary method, API methods, shadow table method and control table change, applicant are true in analysis variation capture technique
Fixed most method (other synchronous method for removing shadow table method) is also to rely on trigger or daily record after all, therefore
Trigger method and diary method are the methods of most basic variation capture.And currently a popular commercial data base all provides trigger machine
System or log analysis tool, therefore the change sequence of trigger hair or Log File Analysis capture source table may be used in the application.
Snapshot, trigger method, diary method, API (Application Programming are introduced individually below
Interface, application programming interface) method, shadow table method and control table change：
Snapshot, snapshot (Snapshot) is the instant image of storage object at a time in database, by being same
It walks one snapshot of object definition or uses similar approach, it can be using its current image as the content of more latest copy.
Trigger method is to need each synchronous source table that insertion (INSERT), deletion is respectively created in source database
(DELETE) and (UPDATA) three triggers are changed, trigger is opened when source table occurs insertion, operation is deleted or modified
It is dynamic, the interim table of row variation is influenced to obtain the change information of source table using record in trigger, and the operation of generation is reduced to
SQL statement, and record log file, journal file should include at least the necessary informations such as operating time, the SQL statement being reduced.
Diary method, important tool of the database journal as maintaining data integrity and database recovery, wherein having wrapped
The operation note information that whole successes are submitted is contained, has been exactly to be captured together by the information of analytical database daily record based on diary method
Walk the change sequence of object.
The database of API methods, some toy data bases and non-relational does not have trigger and log mechanism, can apply
A kind of middleware is introduced between program and database, is provided a series of API (including ODBC driver) by it, is come on API
While application program is completed to database update, the change sequence of synchronization object is recorded.
Shadow table method, in many cases source database without it is understood that synchronization object operation each time, it is only it is to be understood that last
Have occurred that variation is sufficient in total.Therefore, a shadow can be established for synchronization object table T in initialization
(shadow) table S, that is, make a copy at that time, it machine can be obtained in due course by the content of T before hair and S later
Take net change information.
Control table change exactly creates the major key word that control table a C, C only include T for each source table T to be synchronized
Section Pk and some control information fields, when some record changes in T, the record in C with major key Pk is also changed immediately.
This process can usually be realized by trigger, and the change information of T then only need to be just can know that according to C.
In addition, in the present embodiment, after the change sequence of source table, so that it may to combine the data of source table and object table to map
Relationship restores the SQL statement of object table.Since almost all of popular databases all support SQL92 standards, " also based on SQL
Former method " unified will carry out SQL statement reduction according to SQL92 standards, without being directed to a certain specific data DML.
By taking data type is mapping relations as an example, " being based on SQL reduction methods " is by the way of interface come same conversion, interface
The data type and SQL92 for inside presetting the popular databases such as Oralce, SQLServer, DB2, Sybase, a MySql are marked
The mapping table of accurate data type.
It includes following three kinds of situations to restore SQL statement：If source table is insertion operation, it is reduced into object table
INSERT sentences；If source table is delete operation, it is reduced into the DELETE sentences of object table；If source table is modification behaviour
Make, two sentences of DELETE and INSERT of object table are then reduced into, i.e., if being sent out using trigger there are two types of situation
UPDATE statement is split as first deleting is inserted into two sub-operations afterwards, because the interim table of two of slave flipflop can be reduced directly
Go out two sentences of DELETE and INSERT；If using Log File Analysis, then the UPADTE sentences of object table are reduced directly to,
Because the result of log analysis is exactly UPDATE statement, it is not necessary that split into two sentences.
During specific implementation, " carrying out reduction treatment to change sequence according to data mapping relations " in step S20 can
To use following steps, method flow diagram as shown in Figure 3：
S201 judges the field that whether there is multiple media types in change sequence；If it is not, thening follow the steps S202；
S202 is based on SQL reduction methods and data mapping relations, to updating the data carry out reduction treatment.
Due to not being defined to medium type in SQL92 standards, so if not containing multimedia in change sequence
The field of type can be directly based upon SQL reduction methods and data mapping relations, and reduction treatment is carried out to change sequence.
And if the field containing multiple media types in change sequence, can only according to the DML grammers of target database come
The field of reduction treatment multiple media types.
Change sequence is distributed to object table from source table, and is carried out to object table using SQL statement and change sequence by S30
Update operation；
In the present embodiment, change sequence is distributed to object table by availability data distribution technology from source table, that is, by source
Table as source node, using object table as destination node.Data dissemination techniques are simply introduced below：
Data dissemination techniques otherwise referred to as propagate (Propagate), the data transmission and reception between responsible node.
Its connection type has been generally acknowledged that based on message (Message-based) formula and dialogue-based two kinds of (Session-based) formula：
(1) message type connection is based on to be also known as message queue (MessageQueuing) formula or preservation and forward (Store-
And-Forward it) connects.Source node is stored in the data to be distributed in the daily record of strategic point in advance, then according still further to sending and connect
The mode of message package is received to transmit data, transmits in/out (Outbound/ of the message package by source node and destination node
Inbound) message queue is completed.It is typical to have through Ftp, Email or come using message-oriented middleware as such as MQ real
It is existing.
(2) dialogue-based formula connects also referred to as point-to-point (Peer-to-Peer) connection.Which need not prepare point in advance
Data are sent out, the just capture variation when issuing is allowed.Data dissemination between node is carried out with conversational mode, and each node can be same
When send and receive data.Dialogue-based formula is connected to efficiency, scalar etc. and is better than being connected based on message type, but can
By being but not so good as to connect based on message type in terms of property.
In addition, its distribution model, which is generally divided into, is divided into three class models such as pushing-type (Push), pull-type (Pull), push-and-pull combination：
(1) log push ship：If undertaking distributed tasks by source node, propagating contents show as initiatively being improved by source node
To destination node, referred to as " pushing-type (Push) " model.The advantage of log push ship is that efficiency is higher；But height is poor, it
Information is propagated all to set in advance.
(2) pull-type model：If undertaking the task of issuing by destination node, propagating contents show as being saved from destination node to source
Point application, referred to as " pull-type (Pull) " model, the advantage of pull-type model are to be easy highly, and source node can reduce propagation
Information, as the face of increasing for participating in number of nodes shows to become apparent；But its efficiency is poor, and can not achieve in real time
Information is propagated.
(3) binding model is pushed and pulled：If distributed tasks undertake referred to as " push-and-pull combines " model by third node.When node increases
Mostly and there are a large amount of synchronous tasks, when in particular for working asynchronously independent, push-and-pull binding pattern is the most suitable.And work as
When interstitial content is fewer or synchronous task is lighter, its performance and cost is all uneconomical.
In addition, can be updated operation to object table using data update technology in the present embodiment, data update refers to
Object table is changed according to the change sequence of source table, to ensure the consistency of data.
During specific implementation, " operation is updated to object table using SQL statement and change sequence " in step S30
Following steps may be used, method flow diagram is as shown in Figure 4：
S301 is handled change sequence using SQL statement；
In the present embodiment, the processing for change sequence includes that complete copy, sequence change or net change again.According to difference
Operation processing when, the requirement for major key is also different, is simply introduced major key below：
It is one or more of table field that major key, which is also major key (primary key), its value is for uniquely
Identify a certain item record in table.In the relationship of two tables, major key is used for the reference in a table and comes from another
Specific record in table.Major key is a kind of unique key, the part that table defines.The major key of one table can be by multiple
Keyword collectively constitutes, and the row of major key cannot include null value.
(1) CREATE is operated：A new table is created with updating the data, is generally used for initializing.Whether source table and object table
There are major key and major key all inessential condition whether of equal value.
(2) INSERT is operated：It will update the data and be inserted into already present table as new record.There are two types of situation, defined masters
When key, the major key field data of object table are centainly than or equal to source table；When without defining major key, object table includes to repeat to remember
The phenomenon that record, is with regard to inevitable.
(3) DELETE operation：Delete the relative recording of existing table.Source table and object table must have major key and major key domain
It must be of equal value.
(4) UPDATE operation：Change the relative recording of existing table.Only the case where it is complete copy to update the data
Under, it can not just define major key；Otherwise source table and object table must all have major key and major key domain must be of equal value.
S302, using treated, change sequence is updated operation to object table.
Further, since transmission trunking is overtime, hardware fault and violates original constraints etc., update operation may lose
It loses, wherein there is part can be by the recovery machine of DBMS (database management system, data base management system)
System is resolved, and irreclaimable update is unsuccessfully called " exception (Exception) " here.The generation of exception can extend data
The potential phase (Latency) of inconsistency, database can be caused to hang up when serious, or even cause system crash.
How to handle exception and depends primarily on the shadow that exception bears the extent of the destruction of data consistency and user program
It rings, general alternative processing mode has：
(1) Redo modes：It is required that distribution updates the data again；
(2) Ignore modes：Ignore this exception, is solved when remaining down subsynchronous；
(3) Execute modes：One group of pre-defined instruction is executed, this is a kind of processing mode the most flexible.
Above step S101~step S103 is only that step S10 disclosed by the embodiments of the present invention " is established in source database
A kind of preferred realization method of data mapping relations in source table and target database between object table " process, this related mistake
The specific implementation of journey can be arbitrarily arranged according to the demand of oneself, not limit herein.
Above step S201~step S202 is only " to be mapped according to data in step S20 disclosed by the embodiments of the present invention
Relationship to change sequence carry out reduction treatment " process a kind of preferred realization method, the specific implementation in relation to this process
It can be arbitrarily arranged according to the demand of oneself, not limited herein.
Above step S301~step S302 be only in step S30 disclosed by the embodiments of the present invention " using SQL statement and
Change sequence is updated operation to object table " a kind of preferred realization method of process, the specific implementation side in relation to this process
Formula can be arbitrarily arranged according to the demand of oneself, not limit herein.
It is complete can be based on SQL reduction methods for the method for data synchronization of Distributed Heterogeneous Database provided in an embodiment of the present invention
It is synchronized at the data between distributed data base, and since SQL statement meets SQL92 standards, data can be made to synchronize
With more validity.
Method of data synchronization based on the Distributed Heterogeneous Database that above-described embodiment provides, the embodiment of the present invention then correspond to
The device for the method for data synchronization for executing above-mentioned Distributed Heterogeneous Database is provided, the structural schematic diagram of the device as shown in figure 5,
Including：
Relationship establishes module 10, is reflected for establishing the data in source database in source table and target database between object table
Penetrate relationship；
Data restoring module 20, if for capturing change sequence from the table of source, according to data mapping relations to becoming
Change sequence and carry out reduction treatment, obtains the SQL structured query language sentences of object table；
Update module 30 for change sequence to be distributed to object table from source table, and utilizes SQL statement and change sequence pair
Object table is updated operation.
In some other embodiment, relationship establishes module 10, is specifically used for：
The first log-on message of source database and the second log-on message of target database are obtained respectively；According to the first note
Volume information drives the verification that type and version are carried out to source database using presetting database, and is made according to the second log-on message
The verification of type and version is carried out to target database with presetting database driving；Pass through in source database and target database
In the case of verification, data type and/or reserved word are established between object table in source database in source table and target database
And/or the mapping relations between operator.
In some other embodiment, data restoring module 20 captures the process of change sequence from the table of source, including：
The change sequence of source table is captured using trigger hair or Log File Analysis.
In some other embodiment, for carrying out the data of reduction treatment also to change sequence according to data mapping relations
Former module 20, is specifically used for：
Judge the field with the presence or absence of multiple media types in change sequence；If it is not, being closed based on SQL reduction methods and data mapping
System carries out reduction treatment to change sequence.
In some other embodiment, for carrying out the data of reduction treatment also to change sequence according to data mapping relations
Former module 20, also particularly useful for：
If so, the DML grammers based on target database and data mapping relations, reduction treatment is carried out to change sequence.
In some other embodiment, for being updated operation more to object table using SQL statement and change sequence
New module 30, is specifically used for：
Change sequence is handled using SQL statement；Using treated, change sequence is updated behaviour to object table
Make.
It is complete can be based on SQL reduction methods for the data synchronization unit of Distributed Heterogeneous Database provided in an embodiment of the present invention
It is synchronized at the data between distributed data base, and since SQL statement meets SQL92 standards, data can be made to synchronize
With more validity.
The method of data synchronization and device of a kind of Distributed Heterogeneous Database provided by the present invention have been carried out in detail above
Thin to introduce, principle and implementation of the present invention are described for specific case used herein, and above example is said
The bright method and its core concept for being merely used to help understand the present invention；Meanwhile for those of ordinary skill in the art, foundation
The thought of the present invention, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification is not
It is interpreted as limitation of the present invention.
It should be noted that each embodiment in this specification is described in a progressive manner, each embodiment weight
Point explanation is all difference from other examples, and the same or similar parts between the embodiments can be referred to each other.
For the device disclosed in the embodiment, since it is corresponded to the methods disclosed in the examples, so fairly simple, the phase of description
Place is closed referring to method part illustration.
It should also be noted that, herein, relational terms such as first and second and the like are used merely to one
Entity or operation are distinguished with another entity or operation, without necessarily requiring or implying between these entities or operation
There are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant are intended to contain
Lid non-exclusive inclusion, so that the element that the process, method, article or equipment including a series of elements is intrinsic,
Further include either the element intrinsic for these process, method, article or equipments.In the absence of more restrictions,
The element limited by sentence "including a ...", it is not excluded that in the process, method, article or equipment for including the element
In there is also other identical elements.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest range caused.
Claims (10)
1. a kind of method of data synchronization of distributed data base, which is characterized in that including：
Establish the data mapping relations in source table and target database between object table in source database；
If capturing change sequence from the source table, the change sequence is gone back according to the data mapping relations
Original place is managed, and the SQL structured query language sentences of the object table are obtained；
The change sequence is distributed to the object table from the source table, and utilizes the SQL statement and the change sequence
Operation is updated to the object table.
2. according to the method described in claim 1, it is characterized in that, described establish in source database in source table and target database
Data mapping relations between object table, including：
The first log-on message of source database and the second log-on message of target database are obtained respectively；
The verification that type and version are carried out to the source database is driven using presetting database according to first log-on message,
And it is driven using the presetting database according to second log-on message and type and version is carried out to the target database
Verification；
The source database and the target database by verification in the case of, source table and institute in the source database
State the mapping relations established between object table in target database between data type and/or reserved word and/or operator.
3. according to the method described in claim 1, it is characterized in that, it is described from the source table capture change sequence process,
Including：
The change sequence of the source table is captured using trigger hair or Log File Analysis.
4. according to the method described in claim 1, it is characterized in that, it is described according to the data mapping relations to the variation sequence
Row carry out reduction treatment, including：
Judge the field with the presence or absence of multiple media types in the change sequence；
If it is not, being based on SQL reduction methods and the data mapping relations, reduction treatment is carried out to the change sequence.
5. method according to claim 4, which is characterized in that further include：
If so, the DML grammers based on the target database and the data mapping relations, restore the change sequence
Processing.
6. according to the method described in claim 1, it is characterized in that, described utilize the SQL statement and the change sequence pair
The object table is updated operation, including：
The change sequence is handled using the SQL statement；
Using treated, the change sequence is updated operation to the object table.
7. a kind of data synchronization unit of distributed data base, which is characterized in that including：
Relationship establishes module, maps for establishing the data in source database in source table and target database between object table and closes
System；
Data restoring module, if for capturing change sequence from the source table, according to the data mapping relations pair
The change sequence carries out reduction treatment, obtains the SQL structured query language sentences of the object table；
Update module, for the change sequence to be distributed to the object table from the source table, and using the SQL statement and
The change sequence is updated operation to the object table.
8. device according to claim 7, which is characterized in that the relationship establishes module, is specifically used for：
The first log-on message of source database and the second log-on message of target database are obtained respectively；According to first note
Volume information drives the verification that type and version are carried out to the source database using presetting database, and according to second note
Volume information drives the verification that type and version are carried out to the target database using the presetting database；In the source data
Library and the target database by verification in the case of, the mesh in source table and the target database in the source database
The mapping relations between data type and/or reserved word and/or operator are established between mark table.
9. device according to claim 7, which is characterized in that be used for according to the data mapping relations to the variation sequence
Row carry out the data restoring module of reduction treatment, are specifically used for：
Judge the field with the presence or absence of multiple media types in the change sequence；If it is not, being reflected based on SQL reduction methods and the data
Relationship is penetrated, reduction treatment is carried out to the change sequence.
10. device according to claim 7, which is characterized in that for utilizing the SQL statement and the change sequence pair
The object table is updated the update module of operation, is specifically used for：
The change sequence is handled using the SQL statement；Using treated, the change sequence is to the target
Table is updated operation.
Priority Applications (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810487831.4A CN108664659A (en) | 2018-05-21 | 2018-05-21 | A kind of method of data synchronization and device of Distributed Heterogeneous Database |
| CN201811148493.8A CN109829011B (en) | 2018-05-21 | 2018-09-29 | Data synchronization method and device for distributed heterogeneous database |
| PCT/CN2018/111879 WO2019223228A1 (en) | 2018-05-21 | 2018-10-25 | Data synchronization method and apparatus for distributed heterogeneous database |
| ZA2019/05486A ZA201905486B (en) | 2018-05-21 | 2019-08-20 | Data synchronization method and device for distributed heterogeneous databases |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810487831.4A CN108664659A (en) | 2018-05-21 | 2018-05-21 | A kind of method of data synchronization and device of Distributed Heterogeneous Database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108664659A true CN108664659A (en) | 2018-10-16 |
Family
ID=63776994
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810487831.4A Pending CN108664659A (en) | 2018-05-21 | 2018-05-21 | A kind of method of data synchronization and device of Distributed Heterogeneous Database |
| CN201811148493.8A Active CN109829011B (en) | 2018-05-21 | 2018-09-29 | Data synchronization method and device for distributed heterogeneous database |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811148493.8A Active CN109829011B (en) | 2018-05-21 | 2018-09-29 | Data synchronization method and device for distributed heterogeneous database |
Country Status (3)
| Country | Link |
|---|---|
| CN (2) | CN108664659A (en) |
| WO (1) | WO2019223228A1 (en) |
| ZA (1) | ZA201905486B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109558452A (en) * | 2018-11-19 | 2019-04-02 | 武汉达梦数据库有限公司 | It is a kind of to inquire the synchronous method for building table handling |
| WO2019223228A1 (en) * | 2018-05-21 | 2019-11-28 | 四川中电启明星信息技术有限公司 | Data synchronization method and apparatus for distributed heterogeneous database |
| CN110765144A (en) * | 2019-10-11 | 2020-02-07 | 口碑(上海)信息技术有限公司 | Distributed heterogeneous database data processing method and device |
| CN111061740A (en) * | 2019-12-17 | 2020-04-24 | 北京软通智慧城市科技有限公司 | Data synchronization method, equipment and storage medium |
| CN112632183A (en) * | 2020-12-11 | 2021-04-09 | 广东电力通信科技有限公司 | Grid network partition data table based quasi-real-time synchronization method and system |
| CN115391459A (en) * | 2022-08-24 | 2022-11-25 | 南京领行科技股份有限公司 | Data synchronization method and device, electronic equipment and computer readable storage medium |
| CN115470294A (en) * | 2022-08-30 | 2022-12-13 | 昆仑数智科技有限责任公司 | Database synchronization method and device, electronic equipment and storage medium |
| CN117971984A (en) * | 2024-03-29 | 2024-05-03 | 天津南大通用数据技术股份有限公司 | Method and system for synchronizing table data among distributed database clusters |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110209741A (en) * | 2019-06-14 | 2019-09-06 | 上海中通吉网络技术有限公司 | Method of data synchronization, device and equipment between heterogeneous data source |
| CN112000732A (en) * | 2020-07-16 | 2020-11-27 | 北京计算机技术及应用研究所 | Trigger-based mutual synchronization method between heterogeneous databases |
| CN112306994A (en) * | 2020-11-10 | 2021-02-02 | 北京沃东天骏信息技术有限公司 | Database data migration method and device and storage medium |
| CN113220711A (en) * | 2021-05-19 | 2021-08-06 | 李坚白 | Method and system for synchronizing heterogeneous robot controller databases |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102542007B (en) * | 2011-12-13 | 2014-06-25 | 中国电子科技集团公司第十五研究所 | Method and system for synchronization of relational databases |
| CN104346377B (en) * | 2013-07-31 | 2017-08-08 | 克拉玛依红有软件有限责任公司 | A kind of data integration and transfer method based on unique mark |
| CN104572672B (en) * | 2013-10-15 | 2018-10-02 | 北大方正集团有限公司 | The synchronous method and application system of heterogeneous database |
| CN103761318B (en) * | 2014-01-27 | 2017-08-18 | 中国工商银行股份有限公司 | A kind of method and system of relationship type synchronization of data in heterogeneous database |
| CN104376062B (en) * | 2014-11-11 | 2018-01-26 | 中国有色金属长沙勘察设计研究院有限公司 | A kind of synchronous method of heterogeneous database platform data |
| GB201516727D0 (en) * | 2015-09-22 | 2015-11-04 | Ibm | Distributed merging of data sets |
| CN107122187A (en) * | 2017-04-26 | 2017-09-01 | 四川中电启明星信息技术有限公司 | A kind of operation system OS accesses design method |
| CN107330003A (en) * | 2017-06-12 | 2017-11-07 | 上海藤榕网络科技有限公司 | Method of data synchronization, system, memory and data syn-chronization equipment |
| CN108664659A (en) * | 2018-05-21 | 2018-10-16 | 四川中电启明星信息技术有限公司 | A kind of method of data synchronization and device of Distributed Heterogeneous Database |
-
2018
- 2018-05-21 CN CN201810487831.4A patent/CN108664659A/en active Pending
- 2018-09-29 CN CN201811148493.8A patent/CN109829011B/en active Active
- 2018-10-25 WO PCT/CN2018/111879 patent/WO2019223228A1/en active Application Filing
-
2019
- 2019-08-20 ZA ZA2019/05486A patent/ZA201905486B/en unknown
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019223228A1 (en) * | 2018-05-21 | 2019-11-28 | 四川中电启明星信息技术有限公司 | Data synchronization method and apparatus for distributed heterogeneous database |
| CN109558452A (en) * | 2018-11-19 | 2019-04-02 | 武汉达梦数据库有限公司 | It is a kind of to inquire the synchronous method for building table handling |
| CN109558452B (en) * | 2018-11-19 | 2020-12-08 | 武汉达梦数据库有限公司 | Synchronization method for query table building operation |
| CN110765144A (en) * | 2019-10-11 | 2020-02-07 | 口碑(上海)信息技术有限公司 | Distributed heterogeneous database data processing method and device |
| CN110765144B (en) * | 2019-10-11 | 2022-02-18 | 口碑(上海)信息技术有限公司 | Distributed heterogeneous database data processing method and device |
| CN111061740A (en) * | 2019-12-17 | 2020-04-24 | 北京软通智慧城市科技有限公司 | Data synchronization method, equipment and storage medium |
| CN111061740B (en) * | 2019-12-17 | 2023-08-04 | 北京软通智慧科技有限公司 | Data synchronization method, device and storage medium |
| CN112632183A (en) * | 2020-12-11 | 2021-04-09 | 广东电力通信科技有限公司 | Grid network partition data table based quasi-real-time synchronization method and system |
| CN112632183B (en) * | 2020-12-11 | 2023-07-11 | 广东电力通信科技有限公司 | Quasi-real-time synchronization method and system based on power grid network partition data table |
| CN115391459A (en) * | 2022-08-24 | 2022-11-25 | 南京领行科技股份有限公司 | Data synchronization method and device, electronic equipment and computer readable storage medium |
| CN115470294A (en) * | 2022-08-30 | 2022-12-13 | 昆仑数智科技有限责任公司 | Database synchronization method and device, electronic equipment and storage medium |
| CN117971984A (en) * | 2024-03-29 | 2024-05-03 | 天津南大通用数据技术股份有限公司 | Method and system for synchronizing table data among distributed database clusters |
Also Published As
| Publication number | Publication date |
|---|---|
| ZA201905486B (en) | 2020-07-29 |
| WO2019223228A1 (en) | 2019-11-28 |
| CN109829011A (en) | 2019-05-31 |
| CN109829011B (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108664659A (en) | A kind of method of data synchronization and device of Distributed Heterogeneous Database | |
| CN107402963B (en) | Search data construction method, incremental data pushing device and equipment | |
| US7523141B2 (en) | Synchronization operations involving entity identifiers | |
| Preguiça et al. | A commutative replicated data type for cooperative editing | |
| Almeida et al. | Efficient state-based crdts by delta-mutation | |
| US7577691B2 (en) | Extending hierarchical synchronization scopes to non-hierarchical scenarios | |
| CN100501731C (en) | System and method for delayed fetching of designated members of a user defined type | |
| CN105574187B (en) | A kind of Heterogeneous Database Replication transaction consistency support method and system | |
| CN107835983A (en) | Backup-and-restore is carried out in distributed data base using consistent database snapshot | |
| KR20120064044A (en) | Computer method and system for combining oltp database and olap database environments | |
| CN102640151A (en) | High throughput, reliable replication of transformed data in information systems | |
| CN110532241A (en) | Data migration method is not shut down using MySQL Binlog | |
| CN107644030A (en) | Data synchronization method for distributed database, relevant apparatus and system | |
| CN109144785A (en) | Method and apparatus for Backup Data | |
| CN113051347B (en) | Method, system, equipment and storage medium for synchronizing data between heterogeneous databases | |
| Brunette et al. | ODK tables: building easily customizable information applications on Android devices | |
| US20100293143A1 (en) | Initialization of database for synchronization | |
| WO2013157099A1 (en) | Database management method, database system and program | |
| Conway et al. | Edelweiss: Automatic storage reclamation for distributed programming | |
| CN109033193A (en) | A kind of database structure and its method of data synchronization | |
| CN108629026A (en) | A kind of update method of full-text index, device, equipment and storage medium | |
| CN112416944A (en) | Method and equipment for synchronizing service data | |
| CN103107898A (en) | Method, device and system for sending or receiving multiple managed objects simultaneously | |
| CN104753659B (en) | A kind of method, apparatus and system of more network element data synchronization loads | |
| CN106980617A (en) | A kind of method and system based on JSON sentence operating databases |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181016 |