CN107798039A - A kind of method of data synchronization and device - Google Patents
A kind of method of data synchronization and device Download PDFInfo
- Publication number
- CN107798039A CN107798039A CN201710357020.8A CN201710357020A CN107798039A CN 107798039 A CN107798039 A CN 107798039A CN 201710357020 A CN201710357020 A CN 201710357020A CN 107798039 A CN107798039 A CN 107798039A
- Authority
- CN
- China
- Prior art keywords
- data
- duration
- precondition
- synchronization
- circulation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/273—Asynchronous replication or reconciliation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A kind of method of data synchronization includes：The precondition for needing circular wait set in synchronization means is obtained, and obtains the circular wait duration of the precondition；The corresponding relation of duration is performed with circulation according to the precondition prestored, judges whether the circular wait duration of the precondition is more than or equal to default circulation and performs duration；When the circular wait duration of the precondition, which is more than or equal to the circulation, performs duration, END instruction is sent to the synchronization means.This method causes the circular wait of synchronization means to operate and can timely and effectively close, and is advantageous to improve the convenience of staff's operation.
Description
Technical field
The invention belongs to data processing field, more particularly to a kind of method of data synchronization and device.
Background technology
In order to ensure the security needs of data, or in order to realize the data sharing requirement of the database of diverse location, lead to
It can often use and arrive database synchronization technology, the data in a database are sent to other databases.For example it will be distributed over each
Data in the database of individual terminal, database in backup server is synchronized to, the data of each terminal can be backed up,
Data can also be collected, convenience is inquired about by the data after collecting or other processing.
When synchronizing processing to data, ripe Tool for Data Warehouse, such as KETTLE instruments etc. can be typically used,
By using each component of the graphical interfaces of synchronization means, data through from source and extraction, change, being loaded onto destination.
When carrying out data syn-chronization using existing Tool for Data Warehouse, the precondition of tool assembly can be typically set.For example circulate
Detect whether some preposition data finishes receiving, if not completing precondition, synchronization means can be in circulation always
For wait state, it is necessary to the manual closing switch of user, operation is more troublesome, and ageing compared with property.
The content of the invention
It is an object of the invention to provide a kind of method of data synchronization, to solve the synchronization that prior art uses graphical interfaces
During instrument, if set precondition do not complete, can the state of board circular wait always, user's manual close operation is more
Trouble, and the problem of poor in timeliness.
In a first aspect, the embodiments of the invention provide a kind of method of data synchronization, methods described includes：
The precondition for needing circular wait set in synchronization means is obtained, and obtains the circulation of the precondition
Waiting time；
The corresponding relation of duration is performed with circulation according to the precondition prestored, judges the precondition
Whether circular wait duration, which is more than or equal to default circulation, performs duration；
When the circular wait duration of the precondition, which is more than or equal to the circulation, performs duration, to the synchronous work
Tool sends END instruction.
With reference in a first aspect, in the first possible implementation of first aspect, when the precondition meet it is default
Data syn-chronization requirement when, methods described also includes：
Within the period where upper once synchronizing time point to current point in time, record what is changed in source database
Data；
The data to be changed in the source database recorded are sent to target database by the synchronization means.
With reference to the possible implementation of the first of first aspect or first aspect, second in first aspect may realization
In mode, the corresponding relation of duration is performed with circulation in the precondition that the basis prestores, is judged described preposition
Whether the circular wait duration of condition was more than or equal to before the step of default circulation performs duration, and methods described is also wrapped
Include：
Obtain the completion time limit of synchronous task, and data type and data volume synchronous required for the synchronous task；
According to the synchronizing speed of the data type counted in advance, when calculating the estimation completed required for the synchronous task
It is long；
According to the completion time limit of the synchronous task and the synchronous estimation duration, the circular wait duration is determined.
With reference to the first possible implementation of first aspect, in the third possible implementation of first aspect, institute
The step of stating the data to be changed in record source database includes：
The modification monitored by trigger in the source database operates, and the trigger includes insertion trigger, deleted
One or more in trigger, modification trigger；
When monitoring that the data in the source database are changed, by the data buffer storage to change to synchronous storage
Area.
With reference to the first possible implementation of first aspect, in the 4th kind of possible implementation of first aspect, institute
Stating the data step to be changed in monitoring source database includes：
The step of data to be changed in the record source database, includes：
The daily record of source database is obtained, according to the modification information recorded in the daily record, determines that the source database occurs
The data of change；
By the data buffer storage to be changed in the source database to synchronous memory block.
Second aspect, the embodiments of the invention provide a kind of data synchronization unit, described device includes：
Information acquisition unit, for obtaining the precondition for needing circular wait set in synchronization means, and obtain
The circular wait duration of the precondition；
Judging unit, for performing the corresponding relation of duration with circulation according to the precondition prestored, judge
Whether the circular wait duration of the precondition, which is more than or equal to default circulation, performs duration；
Instruction sending unit, for when the circular wait duration of the precondition is more than or equal to the circulation and performed
When long, END instruction is sent to the synchronization means.
With reference to second aspect, in the first possible implementation of second aspect, when the precondition meets to preset
Data syn-chronization requirement when, described device also includes：
Data record unit, within the period where upper once synchronizing time point to current point in time, recording source
The data to be changed in database；
Data transmission unit, for the data to be changed in the source database recorded to be sent out by the synchronization means
Deliver to target database.
With reference to the first possible implementation of second aspect, in second of possible implementation of second aspect, institute
Stating device also includes：
Time limit acquiring unit, it is synchronous for obtaining the completion time limit of synchronous task, and required for the synchronous task
Data type and data volume；
Duration acquiring unit is estimated, for the synchronizing speed according to the data type counted in advance, it is described same to calculate completion
The estimation duration that step required by task is wanted；
Circular wait duration acquiring unit, when being estimated for the completion time limit according to the synchronous task and the synchronization
It is long, determine the circular wait duration.
The third aspect, the embodiments of the invention provide a kind of data syn-chronization equipment, including memory, processor and storage
In the memory and the computer program that can run on the processor, computer program described in the computing device
Method of data synchronization described in any one implementation of the above-mentioned first aspects of Shi Shixian.
Fourth aspect, the embodiments of the invention provide a kind of computer-readable recording medium, the computer-readable storage
Media storage has computer program, and the computer program realizes any one reality of above-mentioned first aspect when being executed by processor
Method of data synchronization described in existing mode.
In the present invention, obtain needs the precondition of circular wait in synchronization means, and obtains these preconditions
Corresponding circular wait duration, the corresponding relation of duration is performed with circulation by precondition set in advance, obtains preposition bar
Circulation performs duration corresponding to part, when the waiting time, which is more than default circulation, performs duration, is then sent to synchronization means
END instruction, so that the circular wait operation of synchronization means can be closed timely and effectively, be advantageous to improve staff
The convenience of operation.
Brief description of the drawings
Fig. 1 is the implementation process figure for the method for data synchronization that first embodiment of the invention provides；
Fig. 2 is the implementation process figure that the determination circulation that first embodiment of the invention provides performs duration；
Fig. 3 is the implementation process figure for the method for data synchronization that second embodiment of the invention provides；
Fig. 4 is the structural representation for the data synchronization unit that third embodiment of the invention provides；
Fig. 5 is the structural representation of data syn-chronization equipment provided in an embodiment of the present invention.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, it is right below in conjunction with drawings and Examples
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
The purpose of the embodiment of the present invention is to provide a kind of method of data synchronization, to solve in the prior art to carry out data
When synchronous, due to the synchronization means using graphical interfaces component, after the precondition of graphical interfaces component has been set.Same
Walk in tool operation, the data of some preconditions may be delayed to reach or lost, and cause synchronization means to locate always
In circular wait, system occurs alerting, it is necessary to user's manual off switch, and operation is more troublesome, and the problem of poor in timeliness.
Below in conjunction with the accompanying drawings, the present invention is further illustrated.
Embodiment one：
Fig. 1 shows the implementation process for the method for data synchronization that first embodiment of the invention provides, and details are as follows：
In step S101, the precondition for needing circular wait set in synchronization means is obtained, and described in acquisition
The circular wait duration of precondition.
Specifically, synchronization means described in the embodiment of the present invention, can be that ETL (Extract-Transform-Load, takes out
Take, change, load) instrument, for describing data passing through extraction (extract), conversion (transform) from source terminal, adding
(load) is carried to the process of destination.The synchronization means can be Kettle, Data Stage, Repository
The instruments such as Explorer, Bee load.
Being set in the synchronization means needs the precondition of circular wait, including is set in the graphic assembly of synchronization means
Synchronous data.After the data for needing synchronization are set, operation can be synchronized to data according to the predetermined cycle, wrapped
Include and data are extracted, changed and loaded etc. with operation.
The acquisition of the precondition for needing circular wait, can when synchronization means is set, obtain need it is same
Precondition corresponding to the data of step.Such as in insurance data monthly closing entry synchronization, it is necessary to be completed on the day of Reserve Fund monthly closing entry before
Condition is put, if the same day does not complete, is constantly in wait state, follow-up simultaneously operating can not be carried out.
, can be different according to the specific needs of data of synchronization, setting when it is determined that needing the precondition of circular wait
Precondition with circulation perform duration corresponding relation.The mapping table can be performed to circulation as needed by user
Duration is adjusted, and facilitates user to adjust the duration of circular wait.
In embodiments of the present invention, the circular wait duration step for obtaining the precondition can include：
Synchronous setting requirements in the synchronization means, obtain the operation of graphic assembly corresponding to the precondition
Initial time；
Start to monitor the precondition in the initial time, and the precondition is generated according to the monitoring result
Circular wait duration.
After the operation initial time of graphic assembly corresponding to the precondition is determined, it can be easy to obtain the circulation
The duration of wait.The operation initial time of the graphic assembly, when can be by the startup of the logic flow of the setting of synchronization means
Between determine, or the data process corresponding to the logic flow of synchronization means can be monitored, it is true according to the result of monitoring
The operation initial time of the fixed graphic assembly.
After the initial time is obtained, when the initial time reaches, then start to supervise the precondition
Survey, data input corresponding to the precondition has been detected whether to synchronization means, if it is not, synchronization means is in circulation
Wait state, and timing is carried out to the circular wait duration by calculator.It is several corresponding to the precondition when monitoring
According to synchronization means is input to, then the circular wait duration is reset, and recalculate circular wait duration.
In step s 102, the corresponding relation of duration is performed with circulation according to the precondition prestored, is judged
Whether the circular wait duration of the precondition, which is more than or equal to default circulation, performs duration；
Due to being previously stored with precondition and the corresponding relation of circulation execution duration, set in synchronization means is detected
Precondition, corresponding circulation can be inquired in the mapping table prestored and performs duration.Or the circulation is held
Row duration can be asked for automatically according to historical data.For example inquire about the history number of circular wait duration corresponding to the precondition
According to selecting the prearranged multiple evidence of average circular wait duration, for example select 1.2 times of average average waiting duration to be used as institute
State circulation and perform duration.
Certainly, the circulation performs duration and can also determined according to Fig. 2 methods describeds, specifically includes：
In step s 201, the completion time limit of synchronous task, and data synchronous required for the synchronous task are obtained
Type and data volume.
In the completion time limit of the synchronous task, refer to the expected time for completing data syn-chronization.Completed data syn-chronization when
Between, can be set according to the task of different data syn-chronizations, can also for periodic data syn-chronization task, such as
The data syn-chronization of monthly closing entry, then it can be required according to conventional synchronization, it is automatic to obtain the expected time for completing data syn-chronization.
The data type, continuous data and discrete data can be included, for continuous data and discrete number
According to the time required for synchronization also differs, and therefore, synchronous data can will be needed temporally to make a distinction, can be to not
The data of same type calculate the time required for synchronization respectively.
Certainly, the division for continuous data and discrete data is one of which dividing mode, can also basis
Other dividing modes, required lock in time during distinguishing data syn-chronization.
In step S202, according to the synchronizing speed of the data type counted in advance, calculate and complete the synchronous task institute
The estimation duration needed.
After being made a distinction to data type, the synchronous speed of different data can be obtained according to the synchrodata of history
Degree, according to the size of different types of data and the speed of synchronization, you can calculate the synchronization required for different types of data
Time, so can further try to achieve lock in time required for synchronous data in need.
In step S203, according to the completion time limit of the synchronous task and the synchronous estimation duration, it is determined that described
Circular wait duration.
After the time required for obtaining all data syn-chronizations, with reference to the completion time limit of synchronous task, you can calculate
To the time of circular wait.For example the completion time limit of synchronous task is 24 points, the time required for all data syn-chronizations is 3
30 points of hour, current time are 18 points, then can be obtained when circulation performs a length of 30 minutes.It is, of course, also possible to according to Fig. 2 institutes
The circulation that the method for stating is calculated performs duration, and the circulation searched corresponding to precondition performs duration and is compared, and selects
The shorter time is selected as circulation execution duration.
In step s 103, duration is performed when the circular wait duration of the precondition is more than or equal to the circulation
When, send END instruction to the synchronization means.
When the circular wait duration of the precondition, which is more than or equal to the circulation, performs duration, represent current
The duration of precondition circulation, duration is performed more than circulation set in advance, therefore, it is necessary to terminate current circular wait,
ALM signal is avoided, it is necessary to staff eliminates alarm signal manually, the convenience of data syn-chronization can be improved.
It is described to send END instruction step to the synchronization means and wrap as in embodiment of the invention optional
Include：
END instruction is sent to the synchronization means, closes the switch of circular wait corresponding to the precondition, or,
END instruction is sent to the synchronization means, terminates the operation of the synchronization means.
Wherein, END instruction is sent to the synchronization means, closes current circular wait duration and exceed default feature
The switch corresponding to precondition during executive chairman, so as to terminate the wait to the precondition.It can continue executing with and locate
Manage other synchronous data.
When the data that the precondition is important basic synchronization, then END instruction can be sent to the synchronization means,
Terminate the operation of the synchronization means, and sent to user and terminate synchronous prompt message.
It is worth noting that, the method for the invention be based on existing graphic assembly form synchronization means on the basis of
The monitoring determining program made, by being analyzed and processed to data, it can make it that the operation of synchronization means is relatively reliable, carry
High synchronous operating efficiency.
The present invention obtains the precondition that circular wait is needed in synchronization means, and obtains corresponding to these preconditions
Circular wait duration, the corresponding relation of duration is performed with circulation by precondition set in advance, it is corresponding to obtain precondition
Circulation perform duration, when the waiting time be more than it is default circulation perform duration when, then to synchronization means send end refer to
Order, so that the circular wait operation of synchronization means can be closed timely and effectively, be advantageous to improve staff's operation
Convenience.
Embodiment two：
Fig. 3 shows the implementation process for the method for data synchronization that second embodiment of the invention provides, and details are as follows：
In step S301, the precondition for needing circular wait set in synchronization means is obtained, and described in acquisition
The circular wait duration of precondition.
In step s 302, the corresponding relation of duration is performed with circulation according to the precondition prestored, is judged
Whether the circular wait duration of the precondition, which is more than or equal to default circulation, performs duration.
In step S303, duration is performed when the circular wait duration of the precondition is more than or equal to the circulation
When, send END instruction to the synchronization means.
Step S301-S303 and the step S101-S103 in embodiment one are essentially identical, are not repeated repeat herein.
In step s 304, within the period where upper once synchronizing time point to current point in time, source data is recorded
The data to be changed in storehouse.
In step S305, by the data to be changed in the source database recorded by the synchronization means send to
Target database.
Specifically, monitoring the data to be changed in source database described in the embodiment of the present invention, can specifically use such as
Under type：
1st, monitor modification in the source database by trigger to operate, when monitoring the data in the source database
When changing, by the data buffer storage to change to synchronous memory block, the trigger includes insertion trigger, deletes triggering
One or more in device, modification trigger.
Specifically, by setting trigger in source database, when there is data modification in source database, including data
When any one in insertion, the deletion of data or the rewriting of data occurs, then the writing function in source database is triggered, it is right
The part of data modification is recorded.Data content including record modification, the position where the data content of modification, and will note
The content of record is stored in synchronous memory block.When carrying out data syn-chronization, it is only necessary to which the data in synchronous memory block are sent into mesh
Mark database.
Data modification in the source database, the multiple modification to the data of same position can also be included, in order to carry
The efficiency for the data that high synchronous memory block is stored, can to the content of the data of modification, the storage location of the data of modification with
And the time of modification is recorded, and the data of multiple modification has been carried out for same position, then select last modification when
Between the data of corresponding data as the storage location, so as to avoid the multiple invalid simultaneously operating to changing data, carry
High synchronous efficiency.
2nd, the daily record of source database is obtained, according to the modification information recorded in the daily record, determines the source database hair
The data for changing, by the data buffer storage to be changed in the source database to synchronous memory block.
It is with the difference of mode 1, the synchronous method is to obtain the daily record data in source database, and to the daily record
Data are analyzed, and lookup obtains the daily record modified to source database, according to the position recorded in the daily record, in source number
According to the content that the position is searched in storehouse, and by the content synchronization of lookup to target database.
Certainly, both the above mode is not limited to, the mode based on the contrast of full table can also be included, after lookup obtains modification
Data.By amended data syn-chronization to target database, the efficiency of data syn-chronization can be improved.Especially for database
The synchronization of huge source database, the synchronization times of duplicate data can be greatly reduced, improve the utilization ratio of isochronous resources.
Embodiment three：
Fig. 4 shows the structural representation for the data synchronization unit that third embodiment of the invention provides, and details are as follows：
Data synchronization unit described in the embodiment of the present invention, including：
Information acquisition unit 401, for obtain set in synchronization means need the precondition of circular wait, and obtain
Take the circular wait duration of the precondition；
Judging unit 402, for performing the corresponding relation of duration with circulation according to the precondition prestored, sentence
Whether the circular wait duration for the precondition of breaking, which is more than or equal to default circulation, performs duration；
Instruction sending unit 403, for being held when the circular wait duration of the precondition is more than or equal to the circulation
During row duration, END instruction is sent to the synchronization means.
Preferably, described information acquiring unit includes：
Initial time obtains subelement, for the synchronous setting requirements in the synchronization means, obtains described preposition
The operation initial time of graphic assembly corresponding to condition；
Subelement is monitored, for starting to monitor the precondition in the initial time, and according to the monitoring result
Generate the circular wait duration of the precondition.
Preferably, described device also includes：
Data monitoring unit, for monitoring the data to be changed in source database；
Transmitting element, for the data to change to be sent to target database by the synchronization means.
Preferably, the data monitoring unit is specifically used for：
Modification in the source database is monitored by trigger to operate, when monitoring that the data in the source database send out
During raw modification, by the data buffer storage to change to synchronous memory block, the trigger includes insertion trigger, deletes triggering
One or more in device, modification trigger.
Data synchronization unit described in the embodiment of the present invention, it is corresponding with the method for data synchronization described in embodiment one and two,
This, which is not repeated, repeats.
Fig. 5 shows the schematic diagram of data syn-chronization equipment provided in an embodiment of the present invention.As shown in figure 5, the embodiment
Data syn-chronization equipment 5 includes：Processor 50, memory 51 and it is stored in the memory 51 and can be in the processor 50
The computer program 52 of upper operation, such as the program of data synchronization unit.The processor 50 performs the computer program 52
Step in the above-mentioned each method of data synchronization embodiments of Shi Shixian, such as step 101 shown in Fig. 1 is to 103.It is or described
Processor 50 realizes the function of each module/unit in above-mentioned each device embodiment, such as Fig. 5 when performing the computer program 52
The function of shown module 401 to 403.
Exemplary, the computer program 52 can be divided into one or more module/units, it is one or
Multiple module/units are stored in the memory 51, and are performed by the processor 50, to complete the present invention.Described one
Individual or multiple module/units can be the series of computation machine programmed instruction section that can complete specific function, and the instruction segment is used for
Implementation procedure of the computer program 52 in the data syn-chronization equipment 5 is described.For example, the computer program 52 can be with
It is divided into information acquisition unit, judging unit and instruction sending unit, each unit concrete function is as follows：
Information acquisition unit, for obtaining the precondition for needing circular wait set in synchronization means, and obtain
The circular wait duration of the precondition；
Judging unit, for performing the corresponding relation of duration with circulation according to the precondition prestored, judge
Whether the circular wait duration of the precondition, which is more than or equal to default circulation, performs duration；
Instruction sending unit, for when the circular wait duration of the precondition is more than or equal to the circulation and performed
When long, END instruction is sent to the synchronization means.
The data syn-chronization equipment 5 can be that desktop PC, notebook, palm PC and cloud server etc. calculate
Equipment.The data syn-chronization equipment may include, but be not limited only to, processor 50, memory 51.Those skilled in the art can manage
Solution, Fig. 5 is only the restriction of the example of data syn-chronization equipment 5, not structure paired data synchronizer 5, can be included than diagram
More or less parts, some parts or different parts are either combined, such as the data syn-chronization equipment can also wrap
Include input-output equipment, network access equipment, bus etc..
Alleged processor 50 can be CPU (Central Processing Unit, CPU), can also be
Other general processors, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other PLDs, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor can also be any conventional processor
Deng.
The memory 51 can be the internal storage unit of the data syn-chronization equipment 5, such as data syn-chronization equipment 5
Hard disk or internal memory.The memory 51 can also be the External memory equipment of the data syn-chronization equipment 5, such as the data are same
The plug-in type hard disk being equipped with step equipment 5, intelligent memory card (Smart Media Card, SMC), secure digital (Secure
Digital, SD) card, flash card (Flash Card) etc..Further, the memory 51 can also both include the data
The internal storage unit of synchronizer 5 also includes External memory equipment.The memory 51 is used to store the computer program
And other programs and data needed for the data syn-chronization equipment.The memory 51 can be also used for temporarily storing
Output or the data that will be exported.
In several embodiments provided by the present invention, it should be understood that disclosed apparatus and method, it can be passed through
Its mode is realized.For example, device embodiment described above is only schematical, for example, the division of the unit, only
Only a kind of division of logic function, there can be other dividing mode when actually realizing, such as multiple units or component can be tied
Another system is closed or is desirably integrated into, or some features can be ignored, or do not perform.It is another, it is shown or discussed
Mutual coupling or direct-coupling or communication connection can be the INDIRECT COUPLINGs or logical by some interfaces, device or unit
Letter connection, can be electrical, mechanical or other forms.
The unit illustrated as separating component can be or may not be physically separate, show as unit
The part shown can be or may not be physical location, you can with positioned at a place, or can also be distributed to multiple
On NE.Some or all of unit therein can be selected to realize the mesh of this embodiment scheme according to the actual needs
's.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, can also
That unit is individually physically present, can also two or more units it is integrated in a unit.Above-mentioned integrated list
Member can both be realized in the form of hardware, can also be realized in the form of SFU software functional unit.
If the integrated unit is realized in the form of SFU software functional unit and is used as independent production marketing or use
When, it can be stored in a computer read/write memory medium.Based on such understanding, technical scheme is substantially
The part to be contributed in other words to prior art or all or part of the technical scheme can be in the form of software products
Embody, the computer software product is stored in a storage medium, including some instructions are causing a computer
Equipment (can be personal computer, server, or network equipment etc.) performs the complete of each embodiment methods described of the present invention
Portion or part.And foregoing storage medium includes：USB flash disk, mobile hard disk, read-only storage (ROM, Read-Only Memory),
Random access memory (RAM, Random Access Memory), magnetic disc or CD etc. are various can be with store program codes
Medium.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all essences in the present invention
All any modification, equivalent and improvement made within refreshing and principle etc., should be included in the scope of the protection.
Claims (10)
1. a kind of method of data synchronization, it is characterised in that methods described includes：
The precondition for needing circular wait set in synchronization means is obtained, and obtains the circular wait of the precondition
Duration；
The corresponding relation of duration is performed with circulation according to the precondition prestored, judges the circulation of the precondition
Whether waiting time, which is more than or equal to default circulation, performs duration；
When the circular wait duration of the precondition, which is more than or equal to the circulation, performs duration, sent out to the synchronization means
Send END instruction.
2. method of data synchronization according to claim 1, it is characterised in that when the precondition meets that default data are same
When step requires, methods described also includes：
Within the period where upper once synchronizing time point to current point in time, the number to be changed in source database is recorded
According to；
The data to be changed in the source database recorded are sent to target database by the synchronization means.
3. method of data synchronization according to claim 1 or claim 2, it is characterised in that the basis prestore it is described before
The corresponding relation that condition performs duration with circulation is put, judges whether the circular wait duration of the precondition is more than or equal to institute
Before the step of default circulation performs duration, methods described also includes：
Obtain the completion time limit of synchronous task, and data type and data volume synchronous required for the synchronous task；
According to the synchronizing speed of the data type counted in advance, the estimation duration completed required for the synchronous task is calculated；
According to the completion time limit of the synchronous task and the synchronous estimation duration, the circular wait duration is determined.
4. method of data synchronization according to claim 2, it is characterised in that the number to be changed in the record source database
According to the step of include：
The modification monitored by trigger in the source database operates, and the trigger includes insertion trigger, deletes triggering
One or more in device, modification trigger；
When monitoring that the data in the source database are changed, by the data buffer storage to change to synchronous memory block.
5. method of data synchronization according to claim 2, it is characterised in that the number to be changed in the record source database
According to the step of include：
The daily record of source database is obtained, according to the modification information recorded in the daily record, determines that the source database changes
Data；
By the data buffer storage to be changed in the source database to synchronous memory block.
6. a kind of data synchronization unit, it is characterised in that described device includes：
Information acquisition unit, for obtaining the precondition for needing circular wait set in synchronization means, and described in acquisition
The circular wait duration of precondition；
Judging unit, for performing the corresponding relation of duration with circulation according to the precondition that prestores, described in judgement
Whether the circular wait duration of precondition, which is more than or equal to default circulation, performs duration；
Instruction sending unit, for performing duration when the circular wait duration of the precondition is more than or equal to the circulation
When, send END instruction to the synchronization means.
7. device according to claim 6, it is characterised in that when the precondition meets default data syn-chronization requirement
When, described device also includes：
Data record unit, within the period where upper once synchronizing time point to current point in time, recording source data
The data to be changed in storehouse；
Data transmission unit, for by the data to be changed in the source database recorded by the synchronization means send to
Target database.
8. device according to claim 7, it is characterised in that described device also includes：
Time limit acquiring unit, for obtaining the completion time limit of synchronous task, and data synchronous required for the synchronous task
Type and data volume；
Duration acquiring unit is estimated, for the synchronizing speed according to the data type counted in advance, calculates and completes the synchronization times
Estimation duration required for business；
Circular wait duration acquiring unit, duration is estimated for the completion time limit according to the synchronous task and the synchronization,
Determine the circular wait duration.
9. a kind of data syn-chronization equipment, including memory, processor and it is stored in the memory and can be in the processing
The computer program run on device, it is characterised in that realize such as claim 1 described in the computing device during computer program
The step of to any one of 5 methods described.
10. a kind of computer-readable recording medium, the computer-readable recording medium storage has computer program, and its feature exists
In when the computer program is executed by processor the step of realization such as any one of claim 1 to 5 methods described.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710357020.8A CN107798039B (en) | 2017-05-19 | 2017-05-19 | Data synchronization method and device |
| PCT/CN2018/083047 WO2018210080A1 (en) | 2017-05-19 | 2018-04-13 | Data synchronisation method, apparatus, device, and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710357020.8A CN107798039B (en) | 2017-05-19 | 2017-05-19 | Data synchronization method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107798039A true CN107798039A (en) | 2018-03-13 |
| CN107798039B CN107798039B (en) | 2020-06-05 |
Family
ID=61531119
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710357020.8A Active CN107798039B (en) | 2017-05-19 | 2017-05-19 | Data synchronization method and device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107798039B (en) |
| WO (1) | WO2018210080A1 (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108629002A (en) * | 2018-05-03 | 2018-10-09 | 山东汇贸电子口岸有限公司 | A kind of big data comparison method and device based on kettle |
| WO2018210080A1 (en) * | 2017-05-19 | 2018-11-22 | 平安科技（深圳）有限公司 | Data synchronisation method, apparatus, device, and computer readable storage medium |
| CN109656999A (en) * | 2018-10-11 | 2019-04-19 | 平安科技（深圳）有限公司 | Method of data synchronization, equipment, storage medium and the device of big data quantity |
| CN110290166A (en) * | 2019-05-05 | 2019-09-27 | 平安科技（深圳）有限公司 | Across company-data exchange method, system, device and readable storage medium storing program for executing |
| WO2021147197A1 (en) * | 2020-01-20 | 2021-07-29 | 深圳壹账通智能科技有限公司 | Data synchronization method and apparatus, computer device, and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6112502A (en) * | 1998-02-10 | 2000-09-05 | Diebold, Incorporated | Restocking method for medical item dispensing system |
| US6788997B1 (en) * | 1998-06-01 | 2004-09-07 | Medselect, Inc. | Medical cabinet with adjustable drawers |
| CN101119408A (en) * | 2007-08-16 | 2008-02-06 | 中兴通讯股份有限公司 | Interface, method and communication system for implementing interface data consistency |
| CN102096685A (en) * | 2009-12-11 | 2011-06-15 | 阿里巴巴集团控股有限公司 | Method and device for synchronizing distributive data into data warehouse |
| CN103186624A (en) * | 2011-12-31 | 2013-07-03 | 北京亿阳信通科技有限公司 | Data synchronization method and data synchronization device |
| CN103823797A (en) * | 2012-11-16 | 2014-05-28 | 镇江诺尼基智能技术有限公司 | FTP (file transfer protocol) based real-time industry database data synchronization system |
| CN104246770A (en) * | 2012-04-12 | 2014-12-24 | 诺基亚公司 | Method and apparatus for facilitating switching between devices |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9112808B2 (en) * | 2007-02-06 | 2015-08-18 | At&T Intellectual Property Ii, L.P. | Devices, systems, and methods for providing data |
| US9043283B2 (en) * | 2011-11-01 | 2015-05-26 | International Business Machines Corporation | Opportunistic database duplex operations |
| CN107798039B (en) * | 2017-05-19 | 2020-06-05 | 平安科技（深圳）有限公司 | Data synchronization method and device |
-
2017
- 2017-05-19 CN CN201710357020.8A patent/CN107798039B/en active Active
-
2018
- 2018-04-13 WO PCT/CN2018/083047 patent/WO2018210080A1/en active Application Filing
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6112502A (en) * | 1998-02-10 | 2000-09-05 | Diebold, Incorporated | Restocking method for medical item dispensing system |
| US6788997B1 (en) * | 1998-06-01 | 2004-09-07 | Medselect, Inc. | Medical cabinet with adjustable drawers |
| CN101119408A (en) * | 2007-08-16 | 2008-02-06 | 中兴通讯股份有限公司 | Interface, method and communication system for implementing interface data consistency |
| CN102096685A (en) * | 2009-12-11 | 2011-06-15 | 阿里巴巴集团控股有限公司 | Method and device for synchronizing distributive data into data warehouse |
| CN103186624A (en) * | 2011-12-31 | 2013-07-03 | 北京亿阳信通科技有限公司 | Data synchronization method and data synchronization device |
| CN104246770A (en) * | 2012-04-12 | 2014-12-24 | 诺基亚公司 | Method and apparatus for facilitating switching between devices |
| CN103823797A (en) * | 2012-11-16 | 2014-05-28 | 镇江诺尼基智能技术有限公司 | FTP (file transfer protocol) based real-time industry database data synchronization system |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018210080A1 (en) * | 2017-05-19 | 2018-11-22 | 平安科技（深圳）有限公司 | Data synchronisation method, apparatus, device, and computer readable storage medium |
| CN108629002A (en) * | 2018-05-03 | 2018-10-09 | 山东汇贸电子口岸有限公司 | A kind of big data comparison method and device based on kettle |
| CN109656999A (en) * | 2018-10-11 | 2019-04-19 | 平安科技（深圳）有限公司 | Method of data synchronization, equipment, storage medium and the device of big data quantity |
| CN110290166A (en) * | 2019-05-05 | 2019-09-27 | 平安科技（深圳）有限公司 | Across company-data exchange method, system, device and readable storage medium storing program for executing |
| CN110290166B (en) * | 2019-05-05 | 2023-04-18 | 平安科技（深圳）有限公司 | Cross-cluster data interaction method, system and device and readable storage medium |
| WO2021147197A1 (en) * | 2020-01-20 | 2021-07-29 | 深圳壹账通智能科技有限公司 | Data synchronization method and apparatus, computer device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018210080A1 (en) | 2018-11-22 |
| CN107798039B (en) | 2020-06-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107798039A (en) | A kind of method of data synchronization and device | |
| CN112000737B (en) | Data synchronization method, system, terminal and storage medium based on multi-cloud management | |
| CN112527816B (en) | Data blood relationship analysis method, system, computer equipment and storage medium | |
| US20170109206A1 (en) | Task scheduling method, apparatus, and device | |
| CN108008942B (en) | Method and system for processing data records | |
| CN103533002A (en) | Data processing method and system | |
| US20120324290A1 (en) | Key Based Cluster Log Coalescing | |
| US9727394B2 (en) | Establishing causality order of computer trace records | |
| US10114731B2 (en) | Including kernel object information in a user dump | |
| CN112100239A (en) | Portrait generation method and apparatus for vehicle detection device, server and readable storage medium | |
| CN110874301B (en) | Method and device for acquiring program pause information | |
| CN110716916A (en) | Resource online management method and device, computer equipment and storage medium | |
| CN114172978A (en) | Multi-protocol equipment access method and related device | |
| US20160277477A1 (en) | Information processing apparatus, terminal device, information processing method, and non-transitory computer readable recording medium | |
| CN116089739A (en) | Message pushing method, device, electronic equipment and storage medium | |
| CN112148705A (en) | Data migration method and device | |
| CN108345791B (en) | Processor security detection method, system and detection device | |
| CN110262856B (en) | Application program data acquisition method, device, terminal and storage medium | |
| CN113157306A (en) | Resource packaging management method, device, equipment and storage medium | |
| CN112541825A (en) | Transaction processing method and device based on block chain and electronic equipment | |
| US10345784B2 (en) | Ladder logic graphical programming and textual programming synchronization | |
| CN110020268B (en) | Page returning method and device | |
| CN110750424A (en) | Resource inspection method and device | |
| CN116662276B (en) | Data processing method, device, electronic equipment and storage medium | |
| WO2024066622A1 (en) | Cloud system testing method and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |