CN104346479A - Database synchronization method and database synchronization device - Google Patents
Database synchronization method and database synchronization device Download PDFInfo
- Publication number
- CN104346479A CN104346479A CN201410696885.3A CN201410696885A CN104346479A CN 104346479 A CN104346479 A CN 104346479A CN 201410696885 A CN201410696885 A CN 201410696885A CN 104346479 A CN104346479 A CN 104346479A
- Authority
- CN
- China
- Prior art keywords
- database
- journal
- operation note
- database journal
- mark
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/273—Asynchronous replication or reconciliation
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a database synchronization method and a database synchronization device. The method and the device are applied to a data system which at least comprises a first database and a second database. The method comprises the steps of after the first database is switched into a slave database from a main database, extracting the first database log of the first database and the second database log of the second database; updating the second database log according to the first database log and the second database log; carrying out synchronous operation on data in the second database according to all operation records contained in the updated second database log. After the method is adopted, the log which is not synchronized yet is obtained by the second database, and the operation recorded in the log is carried out on the self data, so that the second database is not lack of data which is not synchronized yet by the first database, and the data state in the second database is correct.
Description
Technical field
The present invention relates to field of computer technology, particularly relate to a kind of database synchronization method and device.
Background technology
In a distributed system, in order to strengthen the reliability of system, master slave mode is often used to dispose the database used in systems in which.Database used in a distributed system can comprise multiple copy at present, usually has a master data base with at least one from database, when master data base normally works, the data syn-chronization of self can be given each from database.
In the prior art, when master data base (hereinafter referred to as the first database) cannot normally work, when associated process as the first database collapses, or the main frame at the first database place delay machine time, then have priority the highest switch to new master data base from database (hereinafter referred to as the second database), and the first database is after the normal work of recovery, can switch to one new for database, after this give the first database by the second database synchronization data and other is respectively from database.
But, because the first database is before cannot normally working, the nearest data of part may be had not yet to be synchronized to each from database (comprising the second database), and after this, be by the second database synchronization data give the first database and other respectively from database, therefore, in the second database, lacked this part data that the first database is not yet synchronously come, thus, the data mode in the second database may be caused incorrect.
Summary of the invention
In view of the above problems, propose the present invention to provide a kind of overcoming the problems referred to above or the database synchronization method solved the problem at least in part and device, be switching to after database in original master data base, guarantee that in new master data base, data mode is correct.
According to a first aspect of the invention, a kind of database synchronization method is proposed, described method is applied to the data system at least comprising the first database and the second database, described first database switches to from database by master data base, described second database switches to master data base by from database, and described method comprises:
Switch to after database at described first database, extract the first database journal of described first database and the second database journal of described second database;
According to described first database journal and described second database journal, upgrade described second database journal;
According to each operation note comprised in the second database journal after renewal, synchronous operation is carried out to the data in described second database.
Alternatively, described daily record at least comprises binary log binlog.
Alternatively, according to described first database journal and described second database journal, upgrade described second database journal, comprising:
Search and be included in described first database journal, and be not included in the operation note in described second database journal;
The each operation note found is incorporated in described second database journal, and using the second database journal after being incorporated to as the second database journal after upgrading.
Alternatively, search and be included in described first database journal, and be not included in the operation note in described second database journal, comprising:
In described first database journal, determine that the rise time apart from current time recently and the operation note be simultaneously included in described first database journal and described second database journal;
Using rise time of operation note of determining as boundary timestamp;
In described first database journal, search the operation note that the rise time is later than described boundary timestamp.
Alternatively, each operation note found is incorporated in described second database journal, comprises:
For the every bar operation note found, determine the mark that this operation note is corresponding, as mark undetermined;
In described second database journal, determine that the rise time is later than mark corresponding to each operation note of the rise time of this operation note, identify as a comparison;
When there is not described mark undetermined in each contrast mark, according to the rise time of this operation note, this operation note is inserted in described second database journal.
Alternatively, when there is described mark undetermined in each contrast mark, the operation note that described in described first database journal, mark undetermined is corresponding is ignored.
Alternatively, according to each operation note comprised in the second database journal after renewal, synchronous operation is carried out to the data in described second database, comprising:
For the every bar operation note be incorporated in described second database journal, extract the operational order comprised in this operation note, and perform this operational order on described second database.
According to a second aspect of the invention, a kind of database synchronization device is proposed, described application of installation is in the data system at least comprising the first database and the second database, described first database switches to from database by master data base, described second database switches to master data base by from database, and described device comprises:
Extraction module, for switching to after database at described first database, extracts the first database journal of described first database and the second database journal of described second database;
Update module, for according to described first database journal and described second database journal, upgrades described second database journal;
Synchronization module, for according to each operation note comprised in the second database journal after renewal, carries out synchronous operation to the data in described second database.
Alternatively, described daily record at least comprises binary log binlog.
Alternatively, described update module is further used for, search and be included in described first database journal, and the operation note be not included in described second database journal, the each operation note found is incorporated in described second database journal, and using the second database journal after being incorporated to as the second database journal after upgrading.
Alternatively, described update module is further used for, in described first database journal, determine that the rise time apart from current time recently and the operation note be simultaneously included in described first database journal and described second database journal, using rise time of operation note of determining as boundary timestamp, in described first database journal, search the operation note that the rise time is later than described boundary timestamp.
Alternatively, described update module is further used for, for the every bar operation note found, determine the mark that this operation note is corresponding, as mark undetermined, in described second database journal, determine that the rise time is later than mark corresponding to each operation note of the rise time of this operation note, identify as a comparison, when there is not described mark undetermined in each contrast mark, according to the rise time of this operation note, this operation note is inserted in described second database journal.
Optionally, described update module is further used for, and when there is described mark undetermined in each contrast mark, ignores the operation note that described in described first database journal, mark undetermined is corresponding.
Alternatively, described synchronization module is further used for, and for the every bar operation note be incorporated in described second database journal, extracts the operational order comprised in this operation note, and perform this operational order on described second database.
According to database synchronization method provided by the invention, owing to switching to after database at the first database, second database is from the daily record of not yet synchronously coming before the first database obtains, and according to this daily record, synchronous operation has been carried out to the data of self, therefore, the data of not yet synchronously coming before after this no longer lacking the first database in the second database, and the data mode in the second database is also correct.
Accompanying drawing explanation
By reading hereafter detailed description of the preferred embodiment, various other advantage and benefit will become cheer and bright for those of ordinary skill in the art.Accompanying drawing only for illustrating the object of preferred implementation, and does not think limitation of the present invention.And in whole accompanying drawing, represent identical parts by identical reference symbol.In the accompanying drawings:
Fig. 1 is the database synchronization process according to the embodiment of the present invention;
Fig. 2 is the detailed process of the database synchronization according to the embodiment of the present invention;
Fig. 3 is the database synchronization apparatus structure schematic diagram according to the embodiment of the present invention.
Embodiment
Below with reference to accompanying drawings exemplary embodiment disclosed by the invention is described in more detail.Although show exemplary embodiment disclosed by the invention in accompanying drawing, however should be appreciated that can realize the present invention in a variety of manners and not should limit by the embodiment set forth here.On the contrary, provide these embodiments to be in order to more thoroughly the present invention can be understood, and complete for scope disclosed by the invention can be conveyed to those skilled in the art.
Below in conjunction with accompanying drawing, describe the technical scheme that various embodiments of the present invention provide in detail.
Fig. 1 is the database synchronization process according to the embodiment of the present invention, and this process can occur in the data system at least comprising the first database and the second database.Wherein, time initial, the first database is master data base, and the second database is from database, and then, the first database switches to from database by master data base, and the second database switches to master data base by from database.Then database synchronization process can comprise step:
S101: server switches to after database at the first database, extracts the first database journal of the first database and the second database journal of the second database.
S102: according to the first database journal and the second database journal, upgrade the second database journal.
S103: according to each operation note comprised in the second database journal after renewal, synchronous operation is carried out to the data in the second database.
In an embodiment of the present invention, described server can be the management server of the data syn-chronization of master-slave database being carried out to unified management, and a master data base in server admin and at least one is from database.Wherein, described master data base and described can lay respectively at different main frames from database, also can be positioned on same main frame.Under general scene, based on the management of server, master data base synchrodata is given each from database.But, when master data base cannot normally work (for convenience of description, after this this master data base is called the first database), such as, when the associated process of the first database collapses, or the main frame at the first database place delay machine time, then have pre-set priority the highest switch to new master data base (after this this new master data base being called the second database) from database.
From the foregoing, in this embodiment, after the first database recovery normally works, can switch to one new for database, because the data that now the first database may still have part nearest are not yet synchronized to each from database (comprising the second database), also namely, there is part operation to perform on the first database, but not yet respectively performing from database.In order to also correspondingly perform this part operation on the second database, correct to ensure the data mode in the second database, then server now can extract the first database journal of the first database and the second database journal of the second database.Wherein, the operation that the first database journal performed for recording the first database, the operation that the second database journal performed for recording the second database.Then, server can according to the first database journal and the second database journal, upgrade the second database journal, also be, upgrade in the second database journal the first database executed and the second database still unenforced this part operation corresponding to operation note, and then server according to each operation note in the second database journal after renewal, can carry out synchronous operation to the data in the second database.So far, the data of not yet synchronously coming before no longer lacking the first database in the second database, and the data mode in the second database is also correct.
According to one embodiment of the present of invention, daily record described in the present invention can be binary log binlog, also can be the daily record of the other types for the data manipulation performed in database of record, described database can be MYSQL, also can be the database of other types, embodiments of the invention are to the type of described daily record, and the type of described database does not limit.Wherein, each database can be respectively arranged with self daily record, and according to time order and function order, all data operating records performed by this database are in this daily record.
Because user is to send Structured Query Language (SQL) (Structured Query Language to database, SQL) mode of statement, data modification operation is carried out to this database, therefore, every bar operation note that database records in the daily record of self all can include a corresponding SQL statement, these SQL statement can be DROP, CREATE, ALTER, GRANT, REVOKE, TRUNCATE, UPDATE, INSERT, the statements such as DELETE, in addition, SELECT statement can't be recorded in daily record and data are not carried out to the UPDATE statement of actual renewal.Except SQL statement, the every bar operation note in daily record also can comprise the information such as rise time, the execution spended time of this SQL statement, the host identification (Server ID) generating this operation note, the client's thread identification sending this SQL statement of this operation note.
At this, also the mode of carrying out master-slave database synchronous based on binlog is briefly described.At the daily record switch open of master data base, and when normal recordings binlog, respectively can obtain the binlog of master data base by increment termly from database, as the binlog of oneself, and according to each operation note in this binlog, the operation (also namely, this binlog of playback) that each operation note records is performed to the data of self, synchronous to realize master-slave database.
According to one embodiment of the present of invention, in above-mentioned steps S102, after server extracts the first database journal and the second database journal, according to the first database journal and the second database journal, the second database journal can be upgraded.Concrete, server can be searched and be included in the first database journal, and is not included in the operation note in the second database journal, then, the each operation note found is incorporated in the second database journal, and using the second database journal after being incorporated to as the second database journal after upgrading.Obviously, the operation corresponding to this part operation note that whois lookup arrives, is the first database executed, and the still unenforced operation of the second database.
Further, according to one embodiment of the present of invention, search and be included in the first database journal, and the method not being included in the operation note in the second database journal is specifically as follows: server can in the first database journal, determine that the rise time is nearest apart from current time, and the operation note be simultaneously included in the first database journal and the second database journal, and using rise time of operation note of determining as boundary timestamp, then, in the first database journal, search the operation note that the rise time is later than boundary timestamp, be included in the first database journal as what find, and the operation note be not included in the second database journal, also namely the first database is not yet synchronized to the operation note of the second database.
It should be noted that, due in database journal, be preserve each operation note according to the order of rise time, then the first database be not yet synchronized to each operation note of the second database be positioned at the first database journal afterbody and for continuous print.Therefore, rise time of determining of server apart from current time recently and the operation note be simultaneously included in the first database journal and the second database journal, is: the current the last item operation note being synchronized to the second database of the first database.Thus each operation note be positioned in the first database after this operation note is: the first database is not yet synchronized to the operation note of the second database.
In embodiments of the present invention, to in the first database journal, determine the rise time apart from current time recently and the method being simultaneously included in the operation note in the first database journal and the second database journal do not limit, at this, only a kind of wherein feasible method is described.
Server can according to the rise time from rear to the order of elder generation, for each operation note in the first database journal, judge in the second database journal, whether include the operation note that LogID is identical with the LogID of this operation note, if, then this operation note is defined as in the first database journal, rise time apart from current time recently and the operation note be simultaneously included in the first database journal and the second database journal, otherwise, above-mentioned judgement is re-started to next operation note, until judged result for till being.Wherein, LogID is that in these database journals, different operation notes can distinguish the different LogID of unique correspondence one at master data base daily record and a global variable respectively using from database journal, to be distributed and safeguard by server to LogID.Because each LogID can unique identification operation note, therefore, the operation note that LogID is identical is identical operation note.
According to one embodiment of the present of invention, server is included in the first database journal finding, and after not being included in each operation note in the second database journal, then each operation note found can be incorporated in the second database journal.For in actual applications, the following scene that may exist is made a concrete analysis of.
The first scene, after the second database switches to master data base, if not yet performed data modification operation in the second database, then each operation note found can directly all be inserted in the second database journal by server.Concrete, each operation note that server can will comprise in all operations record found and the second database journal, by the restructuring in internal memory, according to the order of the rise time of operation note, is kept in the second database journal again.
The second scene, after the second database switches to master data base, if the second database has performed some data modification operation, such as, assignment operation has been carried out to certain data item in the second database, in such a scenario, owing to being not yet synchronized in each operation note of the second database at the first database, the operation corresponding to operation note may be had also to be carry out assignment operation to this data item, therefore, if follow-up, the second database is inserted in this not yet synchronous operation note, and then, the words of the second database journal after playback insertion, the value state of this data item incorrect (when whole second database journal of playback) can be made, or this data item is carried out to the assignment operation (when each operation note of only playback insertion) of redundancy.
Therefore, under above-mentioned the second scene, the each operation note found can not directly all be inserted in the second database journal by server, but for the every bar operation note found, determine the mark that this operation note is corresponding, as mark undetermined, and in the second database journal, determine that the rise time is later than mark corresponding to each operation note of the rise time of this operation note, identify as a comparison, when there is not this mark undetermined in each contrast mark, according to the rise time of this operation note, this operation note is inserted in the second database journal, when there is this mark undetermined in each contrast mark, then from each operation note found, reject this operation note, follow-uply also this operation note is not inserted in the second database journal, namely, ignore the operation note that described in described first database journal, mark undetermined is corresponding.
At this, described mark is described, described mark is used for identifying the data item (hereinafter referred to as data item to be modified) that the SQL statement comprised in the operation note of its correspondence will be revised, during owing to wanting Update Table, first need be positioned to corresponding data item to be modified, treat Update Table item again and carry out assignment operation, therefore, described mark can be locating information.Again owing to being store data with the form of two-dimentional form in a database, then to locate data item to be modified, its locating information at least comprises two attributes of this data item to be modified, and the value of one of them attribute, thus, this data item to be modified can be navigated to and assignment operation is carried out to another attribute.
According to above-mentioned explanation, described mark can be this locating information, and also, described mark can comprise two attributes of data item to be modified, and the value of one of them attribute.The presentation format of the embodiment of the present invention to described mark does not limit, and only lists wherein a kind of feasible presentation format at this:
Described mark can be expressed as two tuples { (key1, value1), (key2) }.Wherein key1 and key2 is two attributes of data item to be modified, and value1 is the value of key1.Citing is described in detail, assuming that include tables of data Student in the first database, before execution data modification operation, tables of data Student is as shown in table 1 below:
| Sno | Sname | Ssex | Sage | Sdept |
| 20140001 | Zhang San | Man | 18 | CS |
| 20140002 | Li Si | Female | 22 | CS |
| 20140003 | King five | Man | 20 | Math |
Table 1
In table 1 above, each data item in tables of data Student includes 5 attribute Sno (student number), Sname (name), Ssex (sex), Sage (age), Sdept (institute), and the value of each attribute.Certain user have sent following SQL statement to the first database:
“UPDATE Student
SET Sage＝22
WHERE Sno＝‘20140001’”。
Mean: by tables of data Student, student number is the age of the student of ' 20140001 ' be revised as 22.
Then the first database is when performing this statement, can navigate to data item to be modified, then carry out assignment operation to it, and write corresponding operation note in the first database journal according to the locating information comprised in this SQL statement.According to above-mentioned explanation, the mark of this operation note can be expressed as { (Sno, ' 20140001 '), (Sage) }.That is, key1 is Sno, value1 be ' 20140001 ', key2 is Sage.
Then after performing this SQL statement, tables of data Student is as shown in table 2 below:
| Sno | Sname | Ssex | Sage | Sdept |
| 20140001 | Zhang San | Man | 22 | CS |
| 20140002 | Li Si | Female | 22 | CS |
| 20140003 | King five | Man | 20 | Math |
Table 2
Can see, in table 2, student number is the age of the student of ' 20140001 ' be revised as 22.
Continue based on upper example below, the process that is incorporated to of above-mentioned the second scene is described.
Assuming that the first database is switching to after database, the operation note one being not yet synchronized to the first database is having two, operation note a and operation note b.
The SQL statement comprised in operation note a is:
“UPDATE Student
SET Sage＝22
WHERE Sno＝‘20140001’”。
Mean: by tables of data Student, student number is the age of the student of ' 20140001 ' be revised as ' 22 '.
The SQL statement comprised in operation note b is:
“UPDATE Student
SET Sdept＝‘Math’
WHERE Sno＝‘20140002’”。
Mean: by tables of data Student, student number is that the institute of the student of ' 20140002 ' is revised as ' Math '.
Assuming that the second database is after switching to master data base, when not yet from the first database synchronization operation note a and operation note b, perform again a data modification operation, and be recorded in the second database journal and (be designated as operation note c).
The SQL statement comprised in operation note c is:
“UPDATE Student
SET Sage＝24
WHERE Sno＝‘20140001’”。
Mean: by tables of data Student, student number is the age of the student of ' 20140001 ' be revised as ' 24 '.
Then first server finds operation note a and operation note b in the first database journal, and determines the mark of their correspondences respectively, as mark undetermined.Corresponding undetermined of operation note a is designated { (Sno, ' 20140001 '), (Sage) }, and corresponding undetermined of operation note b is designated { (Sno, ' 20140002 '), (Sdept) }.
Then, for operation note a, server, in the second database journal, is determined that the rise time is later than the mark of each operation note (being operation note c) correspondence of the rise time of operation note a, is identified as a comparison.Contrast marker is the mark { (Sno, ' 20140001 '), (Sage) } of operation note c.
Next, because the mark of operation note c is identical with the mark of operation note a, this mark undetermined is there is in then contrast mark, also be, if operation note a is synchronized in the second database journal, then when subsequent playback operation note a, the data item that the operation institute assignment that meeting retouching operation record c is corresponding is crossed, but, rise time due to operation note c is complete in the rise time of operation note a, therefore, the current state of the data item crossed of the operation institute assignment that operation note c is corresponding is only correct status, thus, operation note a can be rejected, follow-uply also operation note a is not inserted in the second database journal, namely mark { (Sno undetermined in the first database journal is ignored, ' 20140001 '), (Sage) } corresponding operation note a.
Similar, for operation note b, server, in the second database journal, is determined that the rise time is later than the mark of each operation note (being operation note c) correspondence of the rise time of operation note b, is identified as a comparison.Then contrast marker is the mark { (Sno of operation note c, ' 20140001 '), (Sage) }, because the mark of operation note c is not identical with the mark of operation note b, then there is not this mark undetermined in contrast mark, therefore, the state of the data item that the operation institute assignment that operation note b is corresponding is crossed is only correct status, thus, operation note b need be inserted in the second database journal, follow-up playback be carried out to operation note b.
According to one embodiment of the present of invention, for above-mentioned steps S103, concrete, after server is incorporated to not yet synchronous operation note in the second database journal, can for the every bar operation note being incorporated to the second database journal, extract the operational order comprised in this operation note, and perform this operational order on described second database.Wherein said operational order is SQL statement.So far, no longer lack not yet synchronous data in the second database, and data mode is also correct.
In addition, second database is after synchronous complete data, second database journal can be synchronized to each from database (comprising the first database), then each database journal that can adopt the second database journal replacement self from database, and after playback second database switches to master data base, each operation note newly-increased in the second database journal, makes respectively to keep data consistent and database journal is also consistent from database and the second database.
The detailed process of the database synchronization that Fig. 2 provides for the embodiment of the present invention, specifically comprises the following steps:
S201: server switches to after database at the first database, extracts the first database journal of the first database and the second database journal of the second database.
S202: in the first database journal, determines that the rise time apart from current time recently and the operation note be simultaneously included in the first database journal and the second database journal.
S203: using rise time of operation note of determining as boundary timestamp.
S204: in the first database journal, searches the operation note that the rise time is later than boundary timestamp.
S205: for the every bar operation note found, determine the mark that this operation note is corresponding, as mark undetermined.
S206: in the second database journal, determines that the rise time is later than mark corresponding to each operation note of the rise time of this operation note, identifies as a comparison.
S207: when there is not mark undetermined in each contrast mark, according to the rise time of this operation note, this operation note is inserted in the second database journal.
S208: for the every bar operation note in insertion second database journal, extract the operational order comprised in this operation note, and perform this operational order on the second database.
In another embodiment, when there is mark undetermined in each contrast mark, then ignore the operation note that in the first database journal, mark undetermined is corresponding, and this operation note is not inserted in the second database journal.
Be a kind of database synchronization method that the embodiment of the present invention provides above, based on same invention thinking, the embodiment of the present invention also provides a kind of database synchronization device, as shown in Figure 3.
Fig. 3 is the database switching device shifter structural representation according to one embodiment of the invention, specifically comprises:
Extraction module 301, for switching to after database at described first database, extracts the first database journal of described first database and the second database journal of described second database;
Update module 302, for according to described first database journal and described second database journal, upgrades described second database journal;
Synchronization module 303, for according to each operation note comprised in the second database journal after renewal, carries out synchronous operation to the data in described second database.
Described daily record at least comprises binary log binlog.
Described update module 302 is further used for, search and be included in described first database journal, and the operation note be not included in described second database journal, the each operation note found is incorporated in described second database journal, and using the second database journal after being incorporated to as the second database journal after upgrading.
Described update module 302 is further used for, in described first database journal, determine that the rise time apart from current time recently and the operation note be simultaneously included in described first database journal and described second database journal, using rise time of operation note of determining as boundary timestamp, in described first database journal, search the operation note that the rise time is later than described boundary timestamp.
Described update module 302 is further used for, for the every bar operation note found, determine the mark that this operation note is corresponding, as mark undetermined, in described second database journal, determine that the rise time is later than mark corresponding to each operation note of the rise time of this operation note, identify as a comparison, when there is not described mark undetermined in each contrast mark, according to the rise time of this operation note, this operation note is inserted in described second database journal.
Described update module 302 is further used for, and when there is described mark undetermined in each contrast mark, ignores the operation note that described in described first database journal, mark undetermined is corresponding.
Described synchronization module 303 is further used for, and for every bar operation note of inserting in described second database journal, extracts the operational order comprised in this operation note, and perform this operational order on described second database.
Concrete above-mentioned database synchronization device as shown in Figure 3 can be arranged in server.
The embodiment of the present invention provides a kind of database synchronization method and device, is applied to the data system at least comprising the first database and the second database, and the first database switches to from database by master data base, and the second database switches to master data base by from database.The method switches to after database at the first database, extract the first database journal of the first database and the second database journal of the second database, according to the first database journal and the second database journal, upgrade the second database journal, according to each operation note comprised in the second database journal after renewal, synchronous operation is carried out to the data in the second database.Owing to switching to after database at the first database, second database is from the daily record of not yet synchronously coming before the first database obtains, and according to this daily record, synchronous operation has been carried out to the data of self, therefore, after this no longer lack the data that the first database is not yet synchronously come in the second database, and the data mode in the second database is also correct.
Intrinsic not relevant to any certain computer, virtual system or miscellaneous equipment with display at this algorithm provided.Various general-purpose system also can with use based on together with this teaching.According to description above, the structure constructed required by this type systematic is apparent.In addition, the present invention is not also for any certain programmed language.It should be understood that and various programming language can be utilized to realize content of the present invention described here, and the description done language-specific is above to disclose preferred forms of the present invention.
In instructions provided herein, describe a large amount of detail.But can understand, embodiments of the invention can be put into practice when not having these details.In some instances, be not shown specifically known method, structure and technology, so that not fuzzy understanding of this description.
Similarly, be to be understood that, in order to simplify the disclosure and to help to understand in each inventive aspect one or more, in the description above to exemplary embodiment of the present invention, each feature of the present invention is grouped together in single embodiment, figure or the description to it sometimes.But, the method for the disclosure should be construed to the following intention of reflection: namely the present invention for required protection requires feature more more than the feature clearly recorded in each claim.Or rather, as claims below reflect, all features of disclosed single embodiment before inventive aspect is to be less than.Therefore, the claims following embodiment are incorporated to this embodiment thus clearly, and wherein each claim itself is as independent embodiment of the present invention.
Those skilled in the art are appreciated that and adaptively can change the module in the equipment in embodiment and they are arranged in one or more equipment different from this embodiment.Module in embodiment or unit or assembly can be combined into a module or unit or assembly, and multiple submodule or subelement or sub-component can be put them in addition.Except at least some in such feature and/or process or unit be mutually repel except, any combination can be adopted to combine all processes of all features disclosed in this instructions (comprising adjoint claim, summary and accompanying drawing) and so disclosed any method or equipment or unit.Unless expressly stated otherwise, each feature disclosed in this instructions (comprising adjoint claim, summary and accompanying drawing) can by providing identical, alternative features that is equivalent or similar object replaces.
In addition, those skilled in the art can understand, although embodiments more described herein to comprise in other embodiment some included feature instead of further feature, the combination of the feature of different embodiment means and to be within scope of the present invention and to form different embodiments.Such as, in the following claims, the one of any of embodiment required for protection can use with arbitrary array mode.
All parts embodiment of the present invention with hardware implementing, or can realize with the software module run on one or more processor, or realizes with their combination.It will be understood by those of skill in the art that the some or all functions that microprocessor or digital signal processor (DSP) can be used in practice to realize according to the some or all parts in the equipment of the implantation information in the interception application program of the embodiment of the present invention.The present invention can also be embodied as part or all equipment for performing method as described herein or device program (such as, computer program and computer program).Realizing program of the present invention and can store on a computer-readable medium like this, or the form of one or more signal can be had.Such signal can be downloaded from internet website and obtain, or provides on carrier signal, or provides with any other form.
The present invention will be described instead of limit the invention to it should be noted above-described embodiment, and those skilled in the art can design alternative embodiment when not departing from the scope of claims.In the claims, any reference symbol between bracket should be configured to limitations on claims.Word " comprises " or " comprising " is not got rid of existence and do not arrange element in the claims or step.Word "a" or "an" before being positioned at element is not got rid of and be there is multiple such element.The present invention can by means of including the hardware of some different elements and realizing by means of the computing machine of suitably programming.In the unit claim listing some devices, several in these devices can be carry out imbody by same hardware branch.Word first, second and third-class use do not represent any order.Can be title by these word explanations.
The application can be applied to computer system/server, and it can operate with other universal or special computing system environment numerous or together with configuring.The example of the well-known computing system being suitable for using together with computer system/server, environment and/or configuration includes but not limited to: personal computer system, server computer system, thin client, thick client computer, hand-held or laptop devices, the system based on microprocessor, Set Top Box, programmable consumer electronics, NetPC Network PC, little type Ji calculate machine Xi Tong ﹑ large computer system and comprise the distributed cloud computing technology environment of above-mentioned any system, etc.
Computer system/server can describe under the general linguistic context of the computer system executable instruction (such as program module) performed by computer system.Usually, program module can comprise routine, program, target program, assembly, logic, data structure etc., and they perform specific task or realize specific abstract data type.Computer system/server can be implemented in distributed cloud computing environment, and in distributed cloud computing environment, task is performed by the remote processing devices by communication network links.In distributed cloud computing environment, program module can be positioned at and comprise on the Local or Remote computing system storage medium of memory device.
The present invention includes A8, a kind of database synchronization device, it is characterized in that, described application of installation is in the data system at least comprising the first database and the second database, described first database switches to from database by master data base, described second database switches to master data base by from database, and described device comprises:
Extraction module, for switching to after database at described first database, extracts the first database journal of described first database and the second database journal of described second database;
Update module, for according to described first database journal and described second database journal, upgrades described second database journal;
Synchronization module, for according to each operation note comprised in the second database journal after renewal, carries out synchronous operation to the data in described second database.
A10, device as described in claim A8, it is characterized in that, described update module is further used for, search and be included in described first database journal, and the operation note be not included in described second database journal, the each operation note found is incorporated in described second database journal, and using the second database journal after being incorporated to as the second database journal after upgrading.
A11, device as described in claim A10, it is characterized in that, described update module is further used for, in described first database journal, determine that the rise time apart from current time recently and the operation note be simultaneously included in described first database journal and described second database journal, using rise time of operation note of determining as boundary timestamp, in described first database journal, search the operation note that the rise time is later than described boundary timestamp.
A12, device as described in claim A10, it is characterized in that, described update module is further used for, for the every bar operation note found, determine the mark that this operation note is corresponding, as mark undetermined, in described second database journal, determine that the rise time is later than mark corresponding to each operation note of the rise time of this operation note, identify as a comparison, when there is not described mark undetermined in each contrast mark, according to the rise time of this operation note, this operation note is inserted in described second database journal.
A13, device as described in claim A12, it is characterized in that, described update module is further used for, and when there is described mark undetermined in each contrast mark, ignores the operation note of mark correspondence undetermined described in described first database journal.
A14, device as described in claim A10, it is characterized in that, described synchronization module is further used for, for the every bar operation note be incorporated in described second database journal, extract the operational order comprised in this operation note, and perform this operational order on described second database.
Claims (10)
1. a database synchronization method, it is characterized in that, described method is applied to the data system at least comprising the first database and the second database, and described first database switches to from database by master data base, described second database switches to master data base by from database, and described method comprises:
Switch to after database at described first database, extract the first database journal of described first database and the second database journal of described second database;
According to described first database journal and described second database journal, upgrade described second database journal;
According to each operation note comprised in the second database journal after renewal, synchronous operation is carried out to the data in described second database.
2. the method for claim 1, is characterized in that, described daily record at least comprises binary log binlog.
3. the method for claim 1, is characterized in that, according to described first database journal and described second database journal, upgrades described second database journal, comprising:
Search and be included in described first database journal, and be not included in the operation note in described second database journal;
The each operation note found is incorporated in described second database journal, and using the second database journal after being incorporated to as the second database journal after upgrading.
4. method as claimed in claim 3, is characterized in that, search and be included in described first database journal, and be not included in the operation note in described second database journal, comprising:
In described first database journal, determine that the rise time apart from current time recently and the operation note be simultaneously included in described first database journal and described second database journal;
Using rise time of operation note of determining as boundary timestamp;
In described first database journal, search the operation note that the rise time is later than described boundary timestamp.
5. method as claimed in claim 3, is characterized in that, each operation note found be incorporated in described second database journal, comprise:
For the every bar operation note found, determine the mark that this operation note is corresponding, as mark undetermined;
In described second database journal, determine that the rise time is later than mark corresponding to each operation note of the rise time of this operation note, identify as a comparison;
When there is not described mark undetermined in each contrast mark, according to the rise time of this operation note, this operation note is inserted in described second database journal.
6. method as claimed in claim 5, is characterized in that, when there is described mark undetermined in each contrast mark, ignores the operation note that described in described first database journal, mark undetermined is corresponding.
7. method as claimed in claim 3, is characterized in that, according to each operation note comprised in the second database journal after renewal, carries out synchronous operation, comprising the data in described second database:
For the every bar operation note be incorporated in described second database journal, extract the operational order comprised in this operation note, and perform this operational order on described second database.
8. a database synchronization device, it is characterized in that, described application of installation is in the data system at least comprising the first database and the second database, and described first database switches to from database by master data base, described second database switches to master data base by from database, and described device comprises:
Extraction module, for switching to after database at described first database, extracts the first database journal of described first database and the second database journal of described second database;
Update module, for according to described first database journal and described second database journal, upgrades described second database journal;
Synchronization module, for according to each operation note comprised in the second database journal after renewal, carries out synchronous operation to the data in described second database.
9. device as claimed in claim 8, it is characterized in that, described daily record at least comprises binary log binlog.
10. device as claimed in claim 8, it is characterized in that, described update module is further used for, search and be included in described first database journal, and the operation note be not included in described second database journal, the each operation note found is incorporated in described second database journal, and using the second database journal after being incorporated to as the second database journal after upgrading.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410696885.3A CN104346479A (en) | 2014-11-26 | 2014-11-26 | Database synchronization method and database synchronization device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410696885.3A CN104346479A (en) | 2014-11-26 | 2014-11-26 | Database synchronization method and database synchronization device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104346479A true CN104346479A (en) | 2015-02-11 |
Family
ID=52502070
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410696885.3A Pending CN104346479A (en) | 2014-11-26 | 2014-11-26 | Database synchronization method and database synchronization device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104346479A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104699541A (en) * | 2015-03-30 | 2015-06-10 | 北京奇虎科技有限公司 | Method, device, data transmission assembly and system for synchronizing data |
| CN104731956A (en) * | 2015-04-02 | 2015-06-24 | 北京奇虎科技有限公司 | Method and system for synchronizing data and related database |
| CN106021019A (en) * | 2016-05-12 | 2016-10-12 | 广西尊达电子商务有限公司 | Automatic recovery method for databases |
| CN106997352A (en) * | 2016-01-22 | 2017-08-01 | 晖保智能科技（上海）有限公司 | A kind of master-slave synchronisation management method |
| CN107395770A (en) * | 2017-09-06 | 2017-11-24 | 武汉斗鱼网络科技有限公司 | Method of data synchronization and device |
| CN107632906A (en) * | 2017-09-30 | 2018-01-26 | 北京九桥同步软件有限公司 | Database data backup method and device |
| CN108108431A (en) * | 2017-12-19 | 2018-06-01 | 中国民航信息网络股份有限公司 | Method of data synchronization and system |
| CN108363791A (en) * | 2018-02-13 | 2018-08-03 | 沈阳东软医疗系统有限公司 | A kind of method of data synchronization and device of database |
| CN108763358A (en) * | 2018-05-16 | 2018-11-06 | 阿里巴巴集团控股有限公司 | A kind of data verification method and device |
| CN108984337A (en) * | 2018-05-29 | 2018-12-11 | 杭州网易再顾科技有限公司 | A kind of restorative procedure of data synchronous abnormality, prosthetic device, medium and calculate equipment |
| CN110413685A (en) * | 2019-04-12 | 2019-11-05 | 财付通支付科技有限公司 | Database service switching method, device, readable storage medium storing program for executing and computer equipment |
| CN110955723A (en) * | 2019-11-15 | 2020-04-03 | 北京浪潮数据技术有限公司 | Master-slave database synchronization state monitoring method and device and electronic equipment |
| CN110958287A (en) * | 2018-09-27 | 2020-04-03 | 阿里巴巴集团控股有限公司 | Operation object data synchronization method, device and system |
| CN111414416A (en) * | 2020-02-28 | 2020-07-14 | 平安科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
| CN111639132A (en) * | 2020-05-29 | 2020-09-08 | 中国联合网络通信集团有限公司 | Log synchronization method and device |
| CN114020850A (en) * | 2022-01-05 | 2022-02-08 | 深圳市明源云科技有限公司 | Database data synchronization method, device, equipment and readable storage medium |
| WO2024098756A1 (en) * | 2022-11-11 | 2024-05-16 | 华为云计算技术有限公司 | Database management method and related device |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102089760A (en) * | 2008-03-04 | 2011-06-08 | 苹果公司 | Synchronization server process |
| CN102456052A (en) * | 2010-11-02 | 2012-05-16 | 江苏大学 | Method for data synchronization between embedded device and database |
| CN102984278A (en) * | 2012-12-17 | 2013-03-20 | 北京奇虎科技有限公司 | System and method for realizing browser data synchronization |
| CN103488668A (en) * | 2012-06-07 | 2014-01-01 | 国际商业机器公司 | Reducing data transfers while eliminating data loss for asynchronous replication of databases |
| CN103500229A (en) * | 2013-10-24 | 2014-01-08 | 北京奇虎科技有限公司 | Database synchronization method and database system |
| CN103577546A (en) * | 2013-10-12 | 2014-02-12 | 北京奇虎科技有限公司 | Method and equipment for data backup, and distributed cluster file system |
| CN103678494A (en) * | 2013-11-15 | 2014-03-26 | 北京奇虎科技有限公司 | Method and device for client side and server side data synchronization |
| CN103761318A (en) * | 2014-01-27 | 2014-04-30 | 中国工商银行股份有限公司 | Method and system for data synchronization of relational heterogeneous databases |
| CN103795754A (en) * | 2012-10-31 | 2014-05-14 | 中国电信股份有限公司 | Method and system for data synchronization among multiple systems |
| CN103916482A (en) * | 2014-04-21 | 2014-07-09 | 合肥盈云信息科技有限公司 | Data synchronous transmission method based on sqlite |
| CN103970833A (en) * | 2014-04-02 | 2014-08-06 | 浙江大学 | Method for achieving two-way synchronous data circulation in heterogeneous database synchronizing system based on logs |
| CN104023085A (en) * | 2014-06-25 | 2014-09-03 | 武汉大学 | Security cloud storage system based on increment synchronization |
-
2014
- 2014-11-26 CN CN201410696885.3A patent/CN104346479A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102089760A (en) * | 2008-03-04 | 2011-06-08 | 苹果公司 | Synchronization server process |
| CN102456052A (en) * | 2010-11-02 | 2012-05-16 | 江苏大学 | Method for data synchronization between embedded device and database |
| CN103488668A (en) * | 2012-06-07 | 2014-01-01 | 国际商业机器公司 | Reducing data transfers while eliminating data loss for asynchronous replication of databases |
| CN103795754A (en) * | 2012-10-31 | 2014-05-14 | 中国电信股份有限公司 | Method and system for data synchronization among multiple systems |
| CN102984278A (en) * | 2012-12-17 | 2013-03-20 | 北京奇虎科技有限公司 | System and method for realizing browser data synchronization |
| CN103577546A (en) * | 2013-10-12 | 2014-02-12 | 北京奇虎科技有限公司 | Method and equipment for data backup, and distributed cluster file system |
| CN103500229A (en) * | 2013-10-24 | 2014-01-08 | 北京奇虎科技有限公司 | Database synchronization method and database system |
| CN103678494A (en) * | 2013-11-15 | 2014-03-26 | 北京奇虎科技有限公司 | Method and device for client side and server side data synchronization |
| CN103761318A (en) * | 2014-01-27 | 2014-04-30 | 中国工商银行股份有限公司 | Method and system for data synchronization of relational heterogeneous databases |
| CN103970833A (en) * | 2014-04-02 | 2014-08-06 | 浙江大学 | Method for achieving two-way synchronous data circulation in heterogeneous database synchronizing system based on logs |
| CN103916482A (en) * | 2014-04-21 | 2014-07-09 | 合肥盈云信息科技有限公司 | Data synchronous transmission method based on sqlite |
| CN104023085A (en) * | 2014-06-25 | 2014-09-03 | 武汉大学 | Security cloud storage system based on increment synchronization |
Non-Patent Citations (2)
| Title |
|---|
| 屈志毅等: "DB2大型数据库容灾备份实时复制系统", 《计算机系统应用》 * |
| 钱志新著: "《数据大金矿》", 31 August 2013, 南京大学出版社 * |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104699541A (en) * | 2015-03-30 | 2015-06-10 | 北京奇虎科技有限公司 | Method, device, data transmission assembly and system for synchronizing data |
| CN104699541B (en) * | 2015-03-30 | 2018-07-10 | 北京奇虎科技有限公司 | Method, apparatus, data transfer components and the system of synchrodata |
| CN104731956B (en) * | 2015-04-02 | 2019-07-23 | 北京奇虎科技有限公司 | Method, system and the Relational database of synchrodata |
| CN104731956A (en) * | 2015-04-02 | 2015-06-24 | 北京奇虎科技有限公司 | Method and system for synchronizing data and related database |
| CN106997352A (en) * | 2016-01-22 | 2017-08-01 | 晖保智能科技（上海）有限公司 | A kind of master-slave synchronisation management method |
| CN106021019A (en) * | 2016-05-12 | 2016-10-12 | 广西尊达电子商务有限公司 | Automatic recovery method for databases |
| CN107395770A (en) * | 2017-09-06 | 2017-11-24 | 武汉斗鱼网络科技有限公司 | Method of data synchronization and device |
| CN107632906A (en) * | 2017-09-30 | 2018-01-26 | 北京九桥同步软件有限公司 | Database data backup method and device |
| CN108108431B (en) * | 2017-12-19 | 2021-07-23 | 中国民航信息网络股份有限公司 | Data synchronization method and system |
| CN108108431A (en) * | 2017-12-19 | 2018-06-01 | 中国民航信息网络股份有限公司 | Method of data synchronization and system |
| CN108363791A (en) * | 2018-02-13 | 2018-08-03 | 沈阳东软医疗系统有限公司 | A kind of method of data synchronization and device of database |
| CN108763358A (en) * | 2018-05-16 | 2018-11-06 | 阿里巴巴集团控股有限公司 | A kind of data verification method and device |
| CN108984337B (en) * | 2018-05-29 | 2021-04-16 | 杭州网易再顾科技有限公司 | Data synchronization abnormity repairing method, device, medium and computing equipment |
| CN108984337A (en) * | 2018-05-29 | 2018-12-11 | 杭州网易再顾科技有限公司 | A kind of restorative procedure of data synchronous abnormality, prosthetic device, medium and calculate equipment |
| CN110958287B (en) * | 2018-09-27 | 2022-06-24 | 阿里云计算有限公司 | Operation object data synchronization method, device and system |
| CN110958287A (en) * | 2018-09-27 | 2020-04-03 | 阿里巴巴集团控股有限公司 | Operation object data synchronization method, device and system |
| CN110413685A (en) * | 2019-04-12 | 2019-11-05 | 财付通支付科技有限公司 | Database service switching method, device, readable storage medium storing program for executing and computer equipment |
| CN110413685B (en) * | 2019-04-12 | 2024-03-12 | 财付通支付科技有限公司 | Database service switching method, device, readable storage medium and computer equipment |
| CN110955723A (en) * | 2019-11-15 | 2020-04-03 | 北京浪潮数据技术有限公司 | Master-slave database synchronization state monitoring method and device and electronic equipment |
| CN111414416A (en) * | 2020-02-28 | 2020-07-14 | 平安科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
| WO2021169268A1 (en) * | 2020-02-28 | 2021-09-02 | 平安科技（深圳）有限公司 | Data processing method, apparatus and device, and storage medium |
| CN111414416B (en) * | 2020-02-28 | 2023-05-23 | 平安科技（深圳）有限公司 | Data processing method, device, equipment and storage medium |
| CN111639132A (en) * | 2020-05-29 | 2020-09-08 | 中国联合网络通信集团有限公司 | Log synchronization method and device |
| CN111639132B (en) * | 2020-05-29 | 2023-06-06 | 中国联合网络通信集团有限公司 | Log synchronization method and equipment |
| CN114020850A (en) * | 2022-01-05 | 2022-02-08 | 深圳市明源云科技有限公司 | Database data synchronization method, device, equipment and readable storage medium |
| WO2024098756A1 (en) * | 2022-11-11 | 2024-05-16 | 华为云计算技术有限公司 | Database management method and related device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104346479A (en) | Database synchronization method and database synchronization device | |
| EP2784665B1 (en) | Program and version control method | |
| US20160306612A1 (en) | Determining errors and warnings corresponding to a source code revision | |
| CN106874281B (en) | Method and device for realizing database read-write separation | |
| US10353874B2 (en) | Method and apparatus for associating information | |
| CN103226485A (en) | Code publishing method, machine and system | |
| US11334348B2 (en) | Utilizing machine learning to identify and correct differences in application programming interface (API) specifications | |
| US10048978B2 (en) | Apparatus and method for identifying a virtual machine having changeable settings | |
| CN106445643B (en) | It clones, the method and apparatus of upgrading virtual machine | |
| KR20160124744A (en) | Systems and methods for hosting an in-memory database | |
| CN105740303A (en) | Improved object storage method and apparatus | |
| CN106605217B (en) | For the method and system for being moved to another website from a website will to be applied | |
| CN106709066B (en) | Data synchronization method and device | |
| CN105095392B (en) | A kind of method and device of Data Mart data sharing | |
| CN105637489A (en) | Asynchronous garbage collection in a distributed database system | |
| US20140108367A1 (en) | Client apparatus and database server for resumable transaction and method thereof | |
| CN104462342B (en) | database snapshot synchronization processing method and device | |
| US9031905B2 (en) | Data synchronization | |
| US20140136568A1 (en) | System and method for automatic wrapper induction using target strings | |
| US11086899B2 (en) | On demand synchronization of information | |
| US20150248440A1 (en) | Method for reconfiguration of database, recording medium, and reconfiguration device | |
| US9703848B2 (en) | Caching linked queries for optimized compliance management | |
| US20180032510A1 (en) | Automated translation of source code | |
| US10387527B2 (en) | Global filtering across heterogeneous datasets | |
| CN107357691B (en) | Method and device for processing mirror image file |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination |