CN108536775A - Method for synchronizing or initializing multiple sub-databases and central database - Google Patents
Method for synchronizing or initializing multiple sub-databases and central database Download PDFInfo
- Publication number
- CN108536775A CN108536775A CN201810258265.XA CN201810258265A CN108536775A CN 108536775 A CN108536775 A CN 108536775A CN 201810258265 A CN201810258265 A CN 201810258265A CN 108536775 A CN108536775 A CN 108536775A
- Authority
- CN
- China
- Prior art keywords
- subdata
- subdata base
- data
- consolidated storage
- synchronous
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012217 deletion Methods 0.000 claims abstract description 25
- 230000037430 deletion Effects 0.000 claims abstract description 25
- 230000001360 synchronised effect Effects 0.000 claims abstract description 24
- 238000007792 addition Methods 0.000 claims description 25
- 238000011423 initialization method Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 abstract description 13
- 238000013461 design Methods 0.000 abstract description 6
- 238000003672 processing method Methods 0.000 abstract description 2
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000012550 audit Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for synchronizing or initializing a plurality of sub databases and a central database, which comprises the central database and a plurality of sub databases; the sub-databases needing synchronization or initialization are sequentially added into a queue; at least two sub databases at the head of the queue respectively and correspondingly enter different threads in the thread pool; judging whether the sub-databases need initialization processing or synchronous processing in each thread; if the initialization processing is needed, comparing the data in the sub-database with the data in the central database, correcting the data which is not in accordance with the central database in the sub-database, and adding an additional and a deletion trigger and a corresponding self-defined log table for the sub-database; if synchronous processing is needed, updating the added, deleted and modified data recorded in the custom log table in the sub-database into the central database, and deleting the record corresponding to the added, deleted and modified in the custom log table; the design processing method is flexible and efficient, and has the advantages of network abnormity, small influence and strong adaptability.
Description
Technical field
The present invention relates to the synchronization of database field, especially database and init processing methods.
Background technology
The multiple database synchronizations of tradition are all adopted to consolidated storage or database root according to the method that consolidated storage content initializes
With ETL tools, the one-to-one processing of this mode, underaction, fault freedom are poor, i.e., enabled to realize that many-one synchronizes, but to realize
It is not easy in real time.And this synchronize be need it is lasting, in the case where there is network and the abnormal conditions such as interrupting once in a while by target
Data on library are restored to normal it is difficult to accomplish.
Wherein, initialization process is that the data in the data and consolidated storage in subdata base are carried out batch comparison, is repaiied
The data not just being inconsistent so that the two data are consistent, and synchronization process is generally referred to as increment synchronization, by the increasing in subdata base
The data that amount modification (increase, delete, changing) is crossed are synchronized to consolidated storage, and traditional synchronization or initialisation process have serious stagnant
Property afterwards cannot accomplish to handle in real time.
Invention content
In order to solve the above technical problems, the object of the present invention is to provide clever between a kind of multiple subdata bases and consolidated storage
Living, high efficiency synchronous or initialization process method.
The technical solution adopted by the present invention is：
A kind of synchronous or initialization method between multiple subdata bases and consolidated storage, including consolidated storage and several energy
It is enough that the subdata base that additions and deletions change operation is carried out to internal data；
Subdata base that is synchronous or initializing will be needed to be added in queue successively；Positioned at least two sons of queue head
Database respectively correspondingly enters the different threads in thread pool；Judge that subdata base is needed at initialization in per thread
Reason or synchronization process；If desired initialization process corrects son then by the data comparison of data and consolidated storage in subdata base
The data not being inconsistent with consolidated storage in database, and add the trigger that changes of additions and deletions and corresponding self-defined for the subdata base
Log sheet；If desired synchronization process, then the data update additions and deletions of self-defined log sheet record in subdata base corrected one's mistakes is in
In heart library, and deletes and correspond to the record that the additions and deletions change in self-defined log sheet；
The subdata base that initialization process or synchronization process is completed is returned into queue tail.
After the data progress additions and deletions in subdata library change operation, major key is added by trigger and additions and deletions change type to certainly
It defines in log sheet.
When subdata base enters the thread in thread pool, judge whether subdata base is available, if it is not, then terminating to execute
The thread of the subdata base judges if so, entering in the step of subdata base needs initialization process or synchronization process.
By synchronization means detect subdata base with the presence or absence of it is artificial be modified as down state or can not connection status,
If in the presence of subdata base is set as unavailable.
Newly-increased subdata base is detected whether by synchronization means, and newly-increased subdata base is added to queue
Tail portion.
Beneficial effects of the present invention：
Synchronous or initialization process method between the multiple subdata bases of the present invention and consolidated storage, it is synchronous or initial to needing
The subdata base of change is added in queue successively, and then subdata base is put into thread pool in order, and per thread corresponds to one
A sub- database processing first judges that subdata base is to need synchronous or initialize, then will be in subdata base if initialization
The data comparison of data and consolidated storage corrects the data not being inconsistent with consolidated storage in subdata base, if synchronous, then by subdata base
In self-defined log sheet record the data update to consolidated storage corrected one's mistakes of additions and deletions in, while delete it is corresponding in self-defined log sheet should
The record information that additions and deletions change prevents retriggered subdata base from changing the synchronization process of operation about this additions and deletions with consolidated storage.
The design realizes multiple subdata bases in by the parallel processing of multiple threads in queue order and thread pool
Synchronization between heart library and initialization process so that processing mode is more flexible efficiently, while the synchronizing process of the design is to increase
Amount synchronize, even if there are the abnormal conditions such as network interruption, the self-defined log sheet in subdata base do not have it is impacted, restore just
Chang Hou, consolidated storage corresponding data will soon restore to normal condition, not need manual intervention.
Description of the drawings
The specific implementation mode of the present invention is described further below in conjunction with the accompanying drawings.
Fig. 1 is the basic flow chart of synchronization of the present invention or initial method.
Fig. 2 is the flow chart of one synchronization of the present invention or initial method.
Specific implementation mode
As shown in Figure 1 and Figure 2, synchronous or initialization method, packet between a kind of multiple subdata bases of the present invention and consolidated storage
The subdata base that additions and deletions change operation can be carried out to internal data by including consolidated storage and several, can in website configures the page
It needs to increase in subdata base and subdata base according to user and needs synchronous table name, thus be also required to increase in consolidated storage
The table name synchronous with being needed in subdata base corresponding synchronous meter can carry out initialization or same between subdata base and consolidated storage
Step operation；
Wherein, it would be desirable to which synchronous or initialization subdata base is added in queue successively；At least positioned at queue head
Two subdata bases respectively correspondingly enter the different threads in thread pool；Judge that subdata base is to need just in per thread
Beginningization processing or synchronization process；If desired initialization process, then by the data comparison of data and consolidated storage in subdata base,
It corrects the data that are not inconsistent with consolidated storage in subdata base, and trigger that additions and deletions change and corresponding is added for the subdata base
Self-defined log sheet；If desired synchronization process, then the data additions and deletions of self-defined log sheet record in subdata base corrected one's mistakes are more
Newly in consolidated storage, and deletes and correspond to the record information that the additions and deletions change operation in self-defined log sheet；It will be completed at initialization
The subdata base of reason or synchronization process returns to queue tail.
Before synchronizing operation, subdata base can all first pass through initialization operation, synchronization in need table initial
After the trigger changed for additions and deletions can be added by changing the when of running, while the data progress additions and deletions in subdata library change operation, lead to
It crosses trigger addition major key and additions and deletions changes in type to self-defined log sheet.
To needing subdata base that is synchronous or initializing to be added in queue successively, then subdata base is put into order
In thread pool, per thread corresponds to a sub- database processing, first judges that subdata base is needs synchronization or initializes, if
Initialization corrects the number not being inconsistent with consolidated storage in subdata base then by the data comparison of data and consolidated storage in subdata base
According to if synchronous, then in the data update to the consolidated storage additions and deletions of self-defined log sheet record in subdata base corrected one's mistakes, simultaneously
It deletes and corresponds to the record information that the additions and deletions change in self-defined log sheet, prevent retriggered subdata base with consolidated storage about this increasing
Revise the synchronization process of operation.
The design realizes multiple subdata bases in by the parallel processing of multiple threads in queue order and thread pool
Synchronization between heart library and initialization process so that processing mode is more flexible efficiently, while the synchronizing process of the design is to increase
Amount synchronize, even if there are the abnormal conditions such as network interruption, the self-defined log sheet in subdata base do not have it is impacted, restore just
Chang Hou, consolidated storage corresponding data will soon restore to normal condition, not need manual intervention.
Further, as shown in Fig. 2, user in operation, may increase subdata base newly, the design is in the process of running
Newly-increased subdata base can be also detected whether by synchronization means, and newly-increased subdata base is added to the tail of queue
Portion, to add trigger and the self-defined log sheet of trigger generation by initialization process, synchronization means is a isomery
The tool that data synchronize between database, the database supported at present have:access、sqlserver、oracle、db2、
Sybase ase 11-15, Mysql etc., may be otherwise and realized by audit functions such as CDC, history tables.
Meanwhile in actual operation, user can subdata library modify, subdata base is revised as unavailable shape
Whether situations such as state, or in inter-process, subdata base can not connect deposits here by synchronization means detection subdata base
It is modified as down state or can not connection status (i.e. failure state) artificial, and if it exists, then set subdata base to
It is unavailable, it can be realized by way of changing self-defined log sheet herein.
When subdata base enters the thread in thread pool, judge whether subdata base is available, if it is not, then terminating to execute
The thread of the subdata base judges the step of subdata base needs initialization process or synchronization process if so, can just enter
In.
This mode is configured after not available situation occurs in existing subdata base, while synchronizing processing
Before, terminate the thread of the subdata base in time, ensures the efficient of the speed of service.
The foregoing is merely the preferred embodiments of the present invention, and the present invention is not limited to the above embodiments, as long as with
Essentially identical means realize that the technical solution of the object of the invention belongs within protection scope of the present invention.
Claims (5)
1. a kind of synchronous between multiple subdata bases and consolidated storage or initialization method, it is characterised in that：Including consolidated storage with
And several can carry out the subdata base that additions and deletions change operation to internal data；
Subdata base that is synchronous or initializing will be needed to be added in queue successively；Positioned at least two subdatas of queue head
Library respectively correspondingly enters the different threads in thread pool；Judge that subdata base is to need initialization process also in per thread
It is synchronization process；If desired initialization process corrects subdata then by the data comparison of data and consolidated storage in subdata base
The data not being inconsistent with consolidated storage in library, and add the trigger and corresponding self-defined daily record that additions and deletions change for the subdata base
Table；If desired synchronization process, the then data update additions and deletions of self-defined log sheet record in subdata base corrected one's mistakes to consolidated storage
In, and delete and correspond to the record information that the additions and deletions change operation in self-defined log sheet；
The subdata base that initialization process or synchronization process is completed is returned into queue tail.
2. synchronous between a kind of multiple subdata bases according to claim 1 and consolidated storage or initialization method, special
Sign is：After the data progress additions and deletions in subdata library change operation, major key is added by trigger and additions and deletions change type to certainly
It defines in log sheet.
3. synchronous between a kind of multiple subdata bases according to claim 1 and consolidated storage or initialization method, special
Sign is：When subdata base enters the thread in thread pool, judge whether subdata base is available, if it is not, then terminating to execute
The thread of the subdata base judges if so, entering in the step of subdata base needs initialization process or synchronization process.
4. synchronous between a kind of multiple subdata bases according to claim 3 and consolidated storage or initialization method, special
Sign is：By synchronization means detect subdata base with the presence or absence of it is artificial be modified as down state or can not connection status,
If in the presence of subdata base is set as unavailable.
5. synchronous between a kind of multiple subdata bases according to claim 4 and consolidated storage or initialization method, special
Sign is：Newly-increased subdata base is detected whether by synchronization means, and newly-increased subdata base is added to queue
Tail portion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810258265.XA CN108536775A (en) | 2018-03-27 | 2018-03-27 | Method for synchronizing or initializing multiple sub-databases and central database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810258265.XA CN108536775A (en) | 2018-03-27 | 2018-03-27 | Method for synchronizing or initializing multiple sub-databases and central database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108536775A true CN108536775A (en) | 2018-09-14 |
Family
ID=63485186
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810258265.XA Withdrawn CN108536775A (en) | 2018-03-27 | 2018-03-27 | Method for synchronizing or initializing multiple sub-databases and central database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108536775A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021174696A1 (en) * | 2020-03-06 | 2021-09-10 | 平安科技（深圳）有限公司 | Data updating method and apparatus, computer device, and storage medium |
-
2018
- 2018-03-27 CN CN201810258265.XA patent/CN108536775A/en not_active Withdrawn
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021174696A1 (en) * | 2020-03-06 | 2021-09-10 | 平安科技（深圳）有限公司 | Data updating method and apparatus, computer device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106570086B (en) | Data migration system and data migration method | |
| US11550667B2 (en) | Pluggable database archive | |
| US10789131B2 (en) | Transportable backups for pluggable database relocation | |
| US9589041B2 (en) | Client and server integration for replicating data | |
| CN105550229B (en) | The method and apparatus of distributed memory system data reparation | |
| US8126848B2 (en) | Automated method for identifying and repairing logical data discrepancies between database replicas in a database cluster | |
| US7769714B2 (en) | Automatic error correction for replication and instantaneous instantiation | |
| CN103455557B (en) | A kind of structural data synchronous method based on daily record | |
| CN107609188B (en) | Data synchronization verification method based on golden gate | |
| US9672244B2 (en) | Efficient undo-processing during data redistribution | |
| CN105843707B (en) | Database quick recovery method and equipment | |
| US20150032694A1 (en) | Scalable Coordination Aware Static Partitioning For Database Replication | |
| CN110209728A (en) | A kind of Distributed Heterogeneous Database synchronous method, electronic equipment and storage medium | |
| US20070214168A1 (en) | Method and System for Removing Rows from Directory Tables | |
| CN108369588B (en) | Database level automatic storage management | |
| CN106815326A (en) | A kind of system and method for detecting dereliction key data table uniformity | |
| CN101739313A (en) | Method for protecting and restoring continuous data | |
| CN109376196A (en) | Method and device for batch synchronization of redo logs | |
| CN107291593A (en) | The replacing options and device of failed disk in a kind of RAID system | |
| KR20190063835A (en) | System for processing real-time data modification of in-memory database | |
| CN108536775A (en) | Method for synchronizing or initializing multiple sub-databases and central database | |
| CN110222121A (en) | A kind of SQL Server database increment synchronization realization method and system based on CDC mode | |
| CN112948359A (en) | Non-stop client data migration method based on database sub-base sub-table | |
| CN112835918A (en) | MySQL database increment synchronization implementation method | |
| CN110377602A (en) | A kind of method of data synchronization, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20180914 |
|
| WW01 | Invention patent application withdrawn after publication |