CN106354830A - Device and method for data synchronization of database cluster nodes - Google Patents
Device and method for data synchronization of database cluster nodes Download PDFInfo
- Publication number
- CN106354830A CN106354830A CN201610777767.4A CN201610777767A CN106354830A CN 106354830 A CN106354830 A CN 106354830A CN 201610777767 A CN201610777767 A CN 201610777767A CN 106354830 A CN106354830 A CN 106354830A
- Authority
- CN
- China
- Prior art keywords
- lock
- queue
- information
- data
- deletion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Abstract
The invention provides a device and a method for data synchronization of database cluster nodes. The method includes: acquiring information of to-be-synchronized lists from a metadata server; checking a lock queue to confirm whether a list deletion lock is included or not; if list deletion information is not included in the lock queue, performing data synchronization. Lists can be deleted before modification, and accordingly data exchange synchronization is reduced, waste of time and system resources is avoided, and database synchronization can be completed efficiently.
Description
Technical field
The invention belongs to data-base cluster field, especially relate to a kind of synchronous method of data-base cluster internodal data
And device.
Background technology
During data-base cluster uses, user needs often to carry out building table, calculates, deletes the such operation of table.Before deleting table, such as
Table on fruit data-base cluster node there occurs damage, needs the table on this database node is synchronized.
Can adopt at present and synchronize with the following method: lock on by synchronous table, wait after synchronously completing, delete table behaviour
Make just can carry out.If user has done the calculating that is over when synchronous, need to delete this table, then user will be when synchronous complete quilt
This table could be deleted after the table deleted, thus can waste plenty of time and system resource.
Content of the invention
Embodiments provide a kind of synchronous method and device of data-base cluster internodal data, for solving
State the technical problem of presence.
On the one hand, embodiments provide a kind of synchronous method of data-base cluster internodal data, comprising:
Obtaining from meta data server needs the information of synchronous table；
Check lock queue, be confirmed whether including the lock deleting described table；
If not including the deletion information of described table in the queue of described lock, carry out data syn-chronization.
Further, methods described also includes:
If the queue of described lock does not include the deletion information of described table, abandon carrying out data syn-chronization.
Further, after carrying out data syn-chronization, also include:
Confirm whether data syn-chronization is successful, if it fails, then by the title of described table and failure corresponding node write
Meta data server, generates damage information；
Add deletion to be locked in queue；
After deleting described telogenesis work(, delete described deletion and lock；
Further, after carrying out data syn-chronization, also include:
Delete damage information in meta data server for the described table.
On the other hand, the embodiment of the present invention additionally provides a kind of synchronous device of data-base cluster internodal data, comprising:
Information acquisition unit, needs the information of synchronous table for obtaining from meta data server；
Lock inspection unit, for checking lock queue, is confirmed whether including the lock deleting described table；
Lock unit, if for the deletion information not including described table in the queue of described lock, carry out data syn-chronization.
Further, described device also includes:
Abandoning unit, if including the deletion information of described table for the queue of described lock, abandoning carrying out data syn-chronization.
Further, described device also includes:
Writing unit, for confirming data syn-chronization whether success, if it fails, then by the title of described table and unsuccessfully right
The node write meta data server answered, generates damage information；
Adding device, is locked in queue for adding deletion；
Unit deleted by lock, for after deleting described telogenesis work(, deleting described deletion lock.
Further, described device also includes:
Damage information deletion unit, for deleting damage information in meta data server for the described table.
The embodiment of the present invention is passed through to obtain from meta data server needs the information of synchronous table；Look into lock queue, confirmation is
The no lock including the described table of deletion；If not including the deletion information of described table in the queue of described lock, carry out data syn-chronization.Can
To delete table before table is modified, reduce data exchange synchronous, it is to avoid lose time and system resource.Efficiently complete
Become database synchronization.
Brief description
In order to be illustrated more clearly that the technical scheme of the embodiment of the present invention, below will be in embodiment or description of the prior art
The accompanying drawing of required use be briefly described it should be apparent that, drawings in the following description be only the present invention some are real
Apply example, for those of ordinary skill in the art, without having to pay creative labor, can also be attached according to these
Figure obtains other accompanying drawings.
Fig. 1 is the schematic flow sheet of the method for data-base cluster internodal data synchronization that the embodiment of the present invention one provides；
Fig. 2 is the schematic flow sheet of the method for data-base cluster internodal data synchronization that the embodiment of the present invention two provides；
Fig. 3 is the schematic flow sheet of the method for data-base cluster internodal data synchronization that the embodiment of the present invention three provides；
Fig. 4 is the structural representation of data synchronization unit between the divided data storehouse clustered node that the embodiment of the present invention four provides.
Specific embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation description is it is clear that described embodiment a part of embodiment that is the present invention, rather than whole embodiments.Based on this
Embodiment in bright, the every other enforcement that those of ordinary skill in the art are obtained under the premise of not making creative work
Example, broadly falls into the scope of protection of the invention.
Embodiment one
Fig. 1 is the schematic flow sheet of the method for data-base cluster internodal data synchronization that the embodiment of the present invention one provides.
The present embodiment is applicable to the synchronous situation of data-base cluster internodal data, the method can by data base set group node between number
To execute according to synchronizer, this device can be realized by software/hardware mode, and can be integrated in data base set group node.
Referring to Fig. 1, described distributed data base data backup burst dispositions method, comprising:
S110, obtaining from meta data server needs the information of synchronous table.
Data in any file system is divided into data and metadata.Data refers to the real data in ordinary file, and
Metadata refers to the system data of the feature for describing a file, such as access rights, file owner and file data
Distributed intelligence of block etc..A node is had specially to take charge of metadata management in system, all metadata are stored in this node
Storage device on.Before all clients are to the request of file, first metadata will be asked to this meta data manager.From first number
The information of the table needing to synchronize can be obtained according to server.Exemplary, including the deposit position of synchronous meter.
S120, checks lock queue, is confirmed whether including the lock deleting described table.
Include locking queue management device in data-base cluster, lock queue management device is used for the locking of the sql in processing data storehouse
Request, request queue is stored, and provides query interface to synchronization means.When data base's execution data changes operation, if there are
Failure on partial database node, and on failure node, the copy of corresponding data runs succeeded on other nodes, will fail
Information (table name, database node name) recorded meta data server.Table handling is deleted in data base's execution, adds deleting lock simultaneously
Enter to lock queue management device.By inquiring about it may be determined that whether including deleting the lock of described table to lock queue management device.
S130, if not including the deletion information of described table in the queue of described lock, carries out data syn-chronization.
If not including the deletion information of described table in the queue of described lock, illustrating that this table does not damage, then can adopt
Synchronization means carries out data syn-chronization to described table.
The embodiment of the present invention is passed through to obtain from meta data server needs the information of synchronous table；Look into lock queue, confirmation is
The no lock including the described table of deletion；If not including the deletion information of described table in the queue of described lock, carry out data syn-chronization.Can
To delete table before table is modified, reduce data exchange synchronous, it is to avoid lose time and system resource.Efficiently complete
Become database synchronization.
In a preferred implementation of the present embodiment, methods described also includes: if the queue of described lock include described
The deletion information of table, then abandon carrying out data syn-chronization.If the queue of described lock includes the deletion information of described table, this is described
Table has damaged, and need not carry out data syn-chronization using synchronization means to described table.Or terminate synchronous in synchronous batch of data, keep away
Exempt to lose time and system resource.
Embodiment two
Fig. 2 is the schematic flow sheet of the method for data-base cluster internodal data synchronization that the embodiment of the present invention two provides,
The embodiment of the present invention, based on above-described embodiment, further, increases following steps: confirm number after carrying out data syn-chronization
According to synchronous whether successful, if it fails, then the title of described table and the corresponding node of failure are write meta data server, raw
Become damage information；Add deletion to be locked in queue；After deleting described telogenesis work(, delete described deletion and lock.
Referring to Fig. 2, the synchronous method of described data-base cluster internodal data, comprising:
S210, obtaining from meta data server needs the information of synchronous table.
S220, checks lock queue, is confirmed whether including the lock deleting described table.
S230, if not including the deletion information of described table in the queue of described lock, carries out data syn-chronization.
S240, confirms whether data syn-chronization is successful, if it fails, then by the title of described table and the corresponding node of failure
Write meta data server, generates damage information.
Data base is used for accepting sql request, if the table on certain node damages in request implementation procedure, data base will damage
Bad table name, database node name recorded in meta data server.
S250, adds deletion and is locked in queue.
Because table has damaged, then table can be deleted, table handling is deleted in data base's execution, be added to deleting lock simultaneously
Lock queue management device.
S260, after deleting described telogenesis work(, deletes described deletion and locks.
If successfully deleting table, notifying meta data server, deleting corresponding table in metadata and damaging information, and notifying to lock
Queue server, deletes the deletion lock of this table in lock queue.
The present embodiment passes through to increase following steps after carrying out data syn-chronization: confirm whether data syn-chronization is successful, if
Unsuccessful, then the title of described table and the corresponding node of failure are write meta data server, generate damage information；Add and delete
It is locked in queue；After deleting described telogenesis work(, delete described deletion and lock.Synchronous unsuccessful table can be deleted,
To avoid the later stage to synchronize again, lose time and system resource.
Embodiment three
Fig. 3 is the schematic flow sheet of the method for data-base cluster internodal data synchronization that the embodiment of the present invention three provides,
The embodiment of the present invention, based on above-described embodiment, further, increases following steps: delete institute after carrying out data syn-chronization
State damage information in meta data server for the table.
Referring to Fig. 3, the synchronous method of described data-base cluster internodal data, comprising:
S310, obtaining from meta data server needs the information of synchronous table.
S320, checks lock queue, is confirmed whether including the lock deleting described table.
S330, if not including the deletion information of described table in the queue of described lock, carries out data syn-chronization.
S340, deletes damage information in meta data server for the described table.
During simultaneously operating, after synchronous batch of data, judge synchronous whether completing: if synchronously completed, delete metadata clothes
Table on business device damages information.Due to synchronously completing, the data of table is completely correct, so, described table can be deleted in unit
Damage information in data server.
The present embodiment passes through to increase following steps after carrying out data syn-chronization: deletes described table in meta data server
Damage information.Can be after table synchronization success, the corresponding metadata of time update table, it is to avoid by mistake delete synchronous successfully table.
Example IV
Fig. 4 is the structural representation of the device of data-base cluster internodal data synchronization that the embodiment of the present invention four provides,
As shown in figure 4, described device includes:
Information acquisition unit 410, needs the information of synchronous table for obtaining from meta data server；
Lock inspection unit 420, for checking lock queue, is confirmed whether including the lock deleting described table；
Lock unit 430, if for the deletion information not including described table in the queue of described lock, carry out data same
Step.
The embodiment of the present invention is passed through to obtain from meta data server needs the information of synchronous table；Look into lock queue, confirmation is
The no lock including the described table of deletion；If not including the deletion information of described table in the queue of described lock, carry out data syn-chronization.Can
To delete table before table is modified, reduce data exchange synchronous, it is to avoid lose time and system resource.Efficiently complete
Become database synchronization.
Further, described device also includes:
Abandoning unit, if including the deletion information of described table for the queue of described lock, abandoning carrying out data syn-chronization.
Further, described device also includes:
Writing unit, for confirming data syn-chronization whether success, if it fails, then by the title of described table and unsuccessfully right
The node write meta data server answered, generates damage information；
Adding device, is locked in queue for adding deletion；
Unit deleted by lock, for after deleting described telogenesis work(, deleting described deletion lock.
Further, described device also includes:
Damage information deletion unit, for deleting damage information in meta data server for the described table.
One of ordinary skill in the art will appreciate that: all or part of step realizing above-mentioned each method embodiment can be led to
Cross the related hardware of programmed instruction to complete.Aforesaid program can be stored in a computer read/write memory medium.This journey
Sequence upon execution, executes the step including above-mentioned each method embodiment；And aforesaid storage medium includes: rom, ram, magnetic disc or
Person's CD etc. is various can be with the medium of store program codes.
Finally it is noted that various embodiments above, only in order to technical scheme to be described, is not intended to limit；To the greatest extent
Pipe has been described in detail to the present invention with reference to foregoing embodiments, it will be understood by those within the art that: its according to
So the technical scheme described in foregoing embodiments can be modified, or wherein some or all of technical characteristic is entered
Row equivalent；And these modifications or replacement, do not make the essence of appropriate technical solution depart from various embodiments of the present invention technology
The scope of scheme.
Claims (8)
1. a kind of synchronous method of data-base cluster internodal data is it is characterised in that include:
Obtaining from meta data server needs the information of synchronous table；
Check lock queue, be confirmed whether including the lock deleting described table；
If not including the deletion information of described table in the queue of described lock, carry out data syn-chronization.
2. method according to claim 1 is it is characterised in that methods described also includes:
If the queue of described lock includes the deletion information of described table, abandon carrying out data syn-chronization.
3. method according to claim 1 is it is characterised in that after carrying out data syn-chronization, also include:
Confirm whether data syn-chronization is successful, if it fails, then by the title of described table and the first number of failure corresponding node write
According to server, generate damage information；
Add deletion to be locked in queue；
After deleting described telogenesis work(, delete described deletion and lock.
4. method according to claim 1 is it is characterised in that after carrying out data syn-chronization, also include:
Delete damage information in meta data server for the described table.
5. a kind of synchronous device of data-base cluster internodal data is it is characterised in that include:
Information acquisition unit, needs the information of synchronous table for obtaining from meta data server；
Lock inspection unit, for checking lock queue, is confirmed whether including the lock deleting described table；
Lock unit, if for the deletion information not including described table in the queue of described lock, carry out data syn-chronization.
6. device according to claim 5 is it is characterised in that described device also includes:
Abandoning unit, if including the deletion information of described table for the queue of described lock, abandoning carrying out data syn-chronization.
7. device according to claim 5 is it is characterised in that described device also includes:
Writing unit, whether successful for confirming data syn-chronization, if it fails, then will be corresponding with failure for the title of described table
Node writes meta data server, generates damage information；
Adding device, is locked in queue for adding deletion；
Unit deleted by lock, for after deleting described telogenesis work(, deleting described deletion lock.
8. device according to claim 5 is it is characterised in that described device also includes:
Damage information deletion unit, for deleting damage information in meta data server for the described table.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610777767.4A CN106354830B (en) | 2016-08-31 | 2016-08-31 | Method and device for data synchronization between database cluster nodes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610777767.4A CN106354830B (en) | 2016-08-31 | 2016-08-31 | Method and device for data synchronization between database cluster nodes |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106354830A true CN106354830A (en) | 2017-01-25 |
| CN106354830B CN106354830B (en) | 2020-12-04 |
Family
ID=57856596
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610777767.4A Active CN106354830B (en) | 2016-08-31 | 2016-08-31 | Method and device for data synchronization between database cluster nodes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106354830B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106991606A (en) * | 2017-03-29 | 2017-07-28 | 恒生电子股份有限公司 | transaction data processing method and device |
| CN108650320A (en) * | 2018-05-15 | 2018-10-12 | 北京中创腾锐技术有限公司 | Isomorphism cascade device configuration file synchronous method under a kind of cluster environment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020048327A1 (en) * | 2000-10-25 | 2002-04-25 | Yoshiju Watanabe | Data synchronization detection device, information recording device, information reproduction device, information recording format, signal processing device, information recording and reproducing device, and information recording medium |
| CN101610276A (en) * | 2008-06-16 | 2009-12-23 | 华为技术有限公司 | Data soft delete, recovery and method for synchronous and terminal and system |
| CN102023979A (en) * | 2009-09-09 | 2011-04-20 | 中国工商银行股份有限公司 | Meta-data management method and system |
| CN102685257A (en) * | 2012-06-11 | 2012-09-19 | 深圳中兴网信科技有限公司 | Data synchronous processing system, data synchronous processing method and server system |
| CN104967660A (en) * | 2015-05-11 | 2015-10-07 | 华中科技大学 | Network performance improving method oriented to multi-cloud architecture |
| CN105045938A (en) * | 2015-09-17 | 2015-11-11 | 浪潮(北京)电子信息产业有限公司 | Metadata concurrent access method and system |
| CN105718484A (en) * | 2014-12-04 | 2016-06-29 | 中兴通讯股份有限公司 | File writing method, file reading method, file deletion method, file query method and client |
-
2016
- 2016-08-31 CN CN201610777767.4A patent/CN106354830B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020048327A1 (en) * | 2000-10-25 | 2002-04-25 | Yoshiju Watanabe | Data synchronization detection device, information recording device, information reproduction device, information recording format, signal processing device, information recording and reproducing device, and information recording medium |
| CN101610276A (en) * | 2008-06-16 | 2009-12-23 | 华为技术有限公司 | Data soft delete, recovery and method for synchronous and terminal and system |
| CN102023979A (en) * | 2009-09-09 | 2011-04-20 | 中国工商银行股份有限公司 | Meta-data management method and system |
| CN102685257A (en) * | 2012-06-11 | 2012-09-19 | 深圳中兴网信科技有限公司 | Data synchronous processing system, data synchronous processing method and server system |
| CN105718484A (en) * | 2014-12-04 | 2016-06-29 | 中兴通讯股份有限公司 | File writing method, file reading method, file deletion method, file query method and client |
| CN104967660A (en) * | 2015-05-11 | 2015-10-07 | 华中科技大学 | Network performance improving method oriented to multi-cloud architecture |
| CN105045938A (en) * | 2015-09-17 | 2015-11-11 | 浪潮(北京)电子信息产业有限公司 | Metadata concurrent access method and system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106991606A (en) * | 2017-03-29 | 2017-07-28 | 恒生电子股份有限公司 | transaction data processing method and device |
| CN106991606B (en) * | 2017-03-29 | 2020-09-18 | 恒生电子股份有限公司 | Transaction data processing method and device |
| CN108650320A (en) * | 2018-05-15 | 2018-10-12 | 北京中创腾锐技术有限公司 | Isomorphism cascade device configuration file synchronous method under a kind of cluster environment |
| CN108650320B (en) * | 2018-05-15 | 2021-05-11 | 北京中创腾锐技术有限公司 | Method for synchronizing configuration files of isomorphic cascading equipment in cluster environment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106354830B (en) | 2020-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110532247B (en) | Data migration method and data migration system | |
| US6662196B2 (en) | Collision avoidance in bidirectional database replication | |
| CN103875229B (en) | asynchronous replication method, device and system | |
| CN101334797B (en) | Distributed file systems and its data block consistency managing method | |
| US7702660B2 (en) | I/O free recovery set determination | |
| RU2425415C2 (en) | Resource update and replication | |
| JP3779263B2 (en) | Conflict resolution for collaborative work systems | |
| US6873995B2 (en) | Method, system, and program product for transaction management in a distributed content management application | |
| US6772155B1 (en) | Looking data in a database system | |
| CN102103642B (en) | Data-erasure method based on OLTP, system and graphic data base server | |
| US20120054546A1 (en) | Methods for detecting split brain in a distributed system | |
| US6782399B2 (en) | Ultra-high speed database replication with multiple audit logs | |
| CN102360410B (en) | User operation discovery method of file system and synchronous system utilizing the same | |
| CN104657158B (en) | The method and apparatus of business processing in a kind of operation system | |
| CN111444027B (en) | Transaction processing method and device, computer equipment and storage medium | |
| WO2022242372A1 (en) | Object processing method and apparatus, computer device, and storage medium | |
| CN104793981B (en) | A kind of online snapshot management method and device of cluster virtual machine | |
| CN106446300A (en) | Transaction processing method and system based on shared storage pool | |
| CN106354830A (en) | Device and method for data synchronization of database cluster nodes | |
| CN110096237A (en) | Replica processes method and node, storage system, server, readable medium | |
| US7051051B1 (en) | Recovering from failed operations in a database system | |
| CN109947592A (en) | A kind of method of data synchronization, device and relevant device | |
| CN116089359A (en) | Database snapshot generation method and device, electronic equipment and medium | |
| CN104220982B (en) | A kind of transaction methods and device | |
| CN111917826A (en) | PBFT consensus algorithm based on block chain intellectual property protection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |