CN106412088B - A kind of method of data synchronization and terminal - Google Patents
A kind of method of data synchronization and terminal Download PDFInfo
- Publication number
- CN106412088B CN106412088B CN201610938969.2A CN201610938969A CN106412088B CN 106412088 B CN106412088 B CN 106412088B CN 201610938969 A CN201610938969 A CN 201610938969A CN 106412088 B CN106412088 B CN 106412088B
- Authority
- CN
- China
- Prior art keywords
- data
- task
- data synchronous
- thread
- synchronous task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present invention relates to field of terminal technology, and in particular to a kind of method of data synchronization and terminal.This method includes the data synchronous task that isochronous queue is added in detection；Thread pool is distributed for data synchronous task；The task start time and task for determining data synchronous task execute duration in advance, if the difference of current time and task start time are greater than task and execute duration in advance, start in thread pool thread and are monitored to data synchronous task；Or, all threads opened in thread pool execute data synchronous task if the difference of current time and task start time shift to an earlier date duration no more than task.It can be realized and number of threads is arranged according to data task, the presence that task executes duration in advance just can actually execute data simultaneously operating at the beginning of data synchronous task, and without waiting for the creation process of thread, the treatment effeciency of data synchronous task is improved, the resource occupation of data synchronous task is reduced.
Description
Technical field
The present invention relates to field of terminal technology, and in particular to a kind of method of data synchronization and terminal.
Background technique
During data synchronous service, the basic unit to the processing of data synchronous task is thread, in treatment process
In can grab data from data source by thread, if single thread grabs data, and carry out the processing of data synchronous service.
But for the mode that current journey grabs, such as single thread, treatment effeciency is low, is on the one hand unable to fully
Utilize the process resource of computer, such as multi-core central processing unit；On the other hand, due to needing whether to start data synchronous task
It is inquired, computer resource will be occupied before actual treatment data synchronous task.
Summary of the invention
The embodiment of the invention provides a kind of method of data synchronization and terminal to solve existing use thread crawl data simultaneously
When just occupying resource and program or system crash before treatment effeciency when being handled is low, data synchronous task starts to process
The problem of can not handling in time.
In view of this, first aspect present invention provides a kind of method of data synchronization, this method comprises:
The data synchronous task of isochronous queue is added in detection；
Thread pool is distributed for the data synchronous task；
Determine that the task start time of the data synchronous task and task execute duration in advance, the task shifts to an earlier date duration
It is preset in the data synchronous system for executing the data synchronous task；
If the difference of current time and the task start time of the data synchronous task are greater than task executes duration in advance,
Then start in thread pool thread to be monitored the data synchronous task；Or,
If the difference of current time and the task start time of the data synchronous task shift to an earlier date duration no more than task,
All threads opened in thread pool execute the data synchronous task.
In some embodiments, the maximum thread amount of the data synchronous system is preset threshold N, and N is more than or equal to 2
Integer, it is described for the data synchronous task distribute thread pool include:
The thread pool of default number of threads is distributed for the data synchronous task.
In some embodiments, the method also includes:
Headed by time data synchronous task distribution number of threads be N/2 thread pool；
Have for the distribution of next data synchronous task when time thread pool of the half number of threads of data synchronous task.
In some embodiments, it when the remaining number of threads of data synchronous system is less than critical value, will be added described same
The data synchronous task of step queue is set to wait for state.
In some embodiments, the data synchronous task includes:
The thread will start mark and data information is added in data synchronization log file；
When data synchronize successfully, success flag is added in the data synchronization log file by thread；Or.
When failure of data synchronization or exception, error identification is added in the data synchronization log file by thread.
In some embodiments, when failure of data synchronization or exception, the method also includes by the meter of anomalous counts device
Number increases；
When the counting of the anomalous counts device reaches default outlier threshold, the thread is with abnormal time delay interval to described
Data synchronous task carries out delay process.
In some embodiments, the method also includes when the thread, which successfully completes data, to be synchronized, by the exception
The counting of counter is reset.
In some embodiments, the method also includes:
Exception immediately occurs during thread executes data synchronous task, then the data of synchronous abnormality are added abnormal
Handle queue；And/or
When data synchronous task is completed or system is restarted, have what is determined by data synchronization log file
Start mark and without success flag, or abnormality processing queue is added in data with error identification.
Second aspect of the embodiment of the present invention also provides a kind of terminal, comprising:
Detection module, for detecting the data synchronous task that isochronous queue is added；
Distribution module, for distributing thread pool for the data synchronous task；
Determining module, task start time and task for determining the data synchronous task execute duration in advance, institute
It states task and executes duration in advance and be preset in the data synchronous system for executing the data synchronous task；
Processing module is greater than task for the difference at current time and the task start time of the data synchronous task
Duration is executed in advance, and the thread started in thread pool is monitored the data synchronous task；Or,
For when the difference at current time and the task start time of the data synchronous task shifts to an earlier date no more than task
Long, then all threads opened in the thread pool execute the data synchronous task.
In some embodiments, the maximum thread amount of the data synchronous system is preset threshold N, and N is more than or equal to 2
Integer, the distribution module is specifically used for:
The thread pool of default number of threads is distributed for the data synchronous task.
In some embodiments, the distribution module is also used to:
Headed by time data synchronous task distribution number of threads be N/2 thread pool；
Have for the distribution of next data synchronous task when time thread pool of the half number of threads of data synchronous task.
In some embodiments, the processing module is also used to:
When the remaining number of threads of the data synchronous system is less than critical value, the data of the isochronous queue will be added
Synchronous task is set to wait for state.
In some embodiments, the processing module is specifically used for:
The thread will start mark and data information is added in data synchronization log file；
When data synchronize successfully, success flag is added in the data synchronization log file by thread；Or.
When failure of data synchronization or exception, error identification is added in the data synchronization log file by thread.
In some embodiments, when failure of data synchronization or exception, the processing module is also used to:
The counting of anomalous counts device is increased；
When the counting of the anomalous counts device reaches default outlier threshold, the thread is with abnormal time delay interval to described
Data synchronous task carries out delay process.
In some embodiments, the processing module is also used to:
When the thread, which successfully completes data, to be synchronized, the counting of the anomalous counts device is reset.
In some embodiments, the processing module is also used to:
Exception immediately occurs during thread executes data synchronous task, exception is added in the data of synchronous abnormality
Manage queue；And/or
It, will be true by data synchronization log file when data synchronous task is completed or data synchronous system is restarted
It is fixed have start mark and without success flag, or abnormality processing queue is added in the data with error identification.
As can be seen from the above technical solutions, the embodiment of the present invention has the advantage that in the embodiment of the present invention, in synchronization
When in queue there is data synchronous task to be added, thread pool can be distributed for the data synchronous task being added, include in the thread pool
At least two threads can determine whether out that the job start time of the data synchronous task and task execute duration in advance later, by
The regular hour is needed in thread creation, if therefore executing duration in advance without the task will make data synchronous task that can be delayed
It executes, current time and the task start time of data synchronous task is then subjected to difference, and the difference is shifted to an earlier date with task
It executes duration to compare, indicates not arrive the execution moment also if being greater than, only starting a detection thread can be used for ash
Degree verifying just opens the institute in thread pool if being less than or equal to then indicates to have arrived at the time of need to be implemented the task
There is thread to execute the data synchronous task, it can be seen that on the one hand by number of threads, another party is arranged thus according to data task
Face be again data synchronous task be provided with task execute duration in advance so that just can actually be held at the beginning of data synchronous task
Row data simultaneously operating, and without waiting for the creation process of thread, to improve the treatment effeciency and drop of data synchronous task
The resource occupation of low data synchronous task.
Detailed description of the invention
Fig. 1 is one embodiment figure of the method for data synchronization of the embodiment of the present invention；
Fig. 2 is another implementation example figure of the method for data synchronization of the embodiment of the present invention；
Fig. 3 is one embodiment figure of the terminal of the embodiment of the present invention.
Specific embodiment
The treatment effeciency of data synchronous task is improved the embodiment of the invention provides a kind of method of data synchronization and terminal
And reduce the resource occupation of data synchronous task.
In order to enable those skilled in the art to better understand the solution of the present invention, below in conjunction in the embodiment of the present invention
Attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only
The embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
The model that the present invention protects all should belong in member's every other embodiment obtained without making creative work
It encloses.
It is described in detail separately below.
Description and claims of this specification and term " first ", " second ", " third " " in above-mentioned attached drawing
The (if present)s such as four " are to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should manage
The data that solution uses in this way are interchangeable under appropriate circumstances, so that the embodiments described herein can be in addition to illustrating herein
Or the sequence other than the content of description is implemented.
Referring to Fig. 1, Fig. 1 is one embodiment figure of method of data synchronization of the embodiment of the present invention, this method can include:
101, the data synchronous task of isochronous queue is added in detection.
Wherein, which is the new data synchronous task that isochronous queue is added, and in isochronous queue before
Some data synchronous tasks.
102, thread pool is distributed for the data synchronous task.
After detecting the new data synchronous task that isochronous queue is added, thread pool, the thread pool can be distributed for the task
Middle number of threads can be allocated according to default rule.
Optionally, the maximum thread for executing the data synchronous system of data synchronous task is N, and N is whole more than or equal to 2
Number distributes thread pool concretely for data synchronous task at this time, to distribute default number of threads when time data synchronous task
Thread pool.As can be seen that for the distribution of the thread pool of data synchronous task be according to it is scheduled rule carry out, for example, if
It is that for the first time therefore data synchronous task, the then idle thread for being N with quantity in system can distribute for the data synchronous task
More thread, for example, distributing the thread of maximum total number of threads half or one third to the data synchronous task.
Optionally, the thread pool for being N/2 if the distribution number of threads of data synchronous task for the first time, for being synchronized when time data
The next data synchronous task of task can have for its distribution when time thread pool of the half number of threads of data synchronous task,
In this manner, as being assigned as the half of maximum thread for the first time, then next time is the 1/4 of maximum thread, then next time then can be with
Be 1/8, using this mode, the Thread Count dispensed enabled to be not up to the upper limit of data synchronous system, because without
Appearance system can not handle the problem of either system crash.
Optionally, when the remaining number of threads of data synchronous system is less than preset critical value, the synchronization will be added
The data synchronous task of queue is set to wait for state.In order to further ensure data synchronous system is unlikely to collapse,
When data synchronous system number of threads is less than critical value, the data synchronous task being newly added will be set to wait for state, from
And guarantee the stability of data synchronous system.
103, the task start time and task for determining the data synchronous task execute duration in advance.
In the number of threads distribution of complete paired data synchronous task, it can determine that the task of data synchronous task is held together
Moment and the task of beginning execute duration in advance, and task start time is that data synchronize at the time of actually start to execute, and task mentions
The preceding duration that executes is then that the distribution of the creation and related resource of thread is carried out before data synchronize practical execution, so that reaching
It is synchronous that task start time just can be immediately performed data.
104, judge whether the difference of current time and the task start time of the data synchronous task mention greater than task
Preceding execution duration, if so, executing to step 105, if it is not, thening follow the steps 106.
During this, the difference at current time and the task start time of the data synchronous task is held in advance with task
Row duration is compared, if the difference is greater than task when executing duration in advance, indicates the time for not arriving thread creation also, can be with
It continues waiting for, and if the difference indicates to have arrived time of thread creation, to make when executing duration in advance no more than task
Data synchronous task can be gone on smoothly by obtaining task start time.
105, the thread started in thread pool is monitored the data synchronous task.
In this step, the execution moment for being equivalent to the data synchronous task is not arrived also, therefore only one in starting thread pool
A thread is monitored, which can also carry out the gray scale verifying of production environment.
106, all threads opened in the thread pool execute the data synchronous task.
In this step, due to having arrived at the execution moment of data synchronous task, and thread pool is same for the data
Step task is allocated, and therefore, all threads that can be opened in thread pool at this time execute data synchronous task.
As can be seen that can be the data synchronous task point of addition when in isochronous queue there is data synchronous task to be added
With thread pool, includes at least two threads in the thread pool, can determine whether out the job start time of the data synchronous task later
And task executes duration in advance, since thread creation needs the regular hour, if executing duration in advance just without the task
So that data synchronous task meeting delayed execution, it is poor then to carry out current time and the task start time of data synchronous task for meeting
Value, and the difference is executed into duration with task in advance and is compared, it indicates not arrive the execution moment also if being greater than, only open
A dynamic detection thread can be used for gray scale verifying, if less than or equal to then indicate to have arrived need to be implemented the task when
It carves, all threads just opened in thread pool execute the data synchronous task, it can be seen that on the one hand by appointing thus according to data
Business setting number of threads, is on the other hand provided with task again for data synchronous task and executes duration in advance, appoints so that data are synchronous
Just data simultaneously operating can be actually executed at the beginning of business, and without waiting for the creation process of thread.
Aforesaid way solves the problems, such as that execution efficiency is low and occupies resource, but during synchronous task, works as line
When journey exits due to unknown cause or is seemingly-dead, technical staff can not obtain the information, cause business datum can not synchronize in time and
Processing, additionally due to inevitably there is the system failure in system, after a failure when system reboot, the data of the storage before failure in memory
It can lose, if not handling this exception, will lead to subsequent synchronous service can not be carried out.Below in the embodiment of the present invention
Synchronous task process and the processing that carries out of while occurring abnormal be introduced.
Referring to Fig. 2, the data synchronous task that Fig. 2 is the method for data synchronization of the embodiment of the present invention executes process, in reality
The process that border carries out data synchronous task may is that
201, thread will start mark and data information is added in data synchronization log file；
202, whether data synchronous task is successfully judged, if success, thens follow the steps 203, if failure, holds
Row step 204.
203, success flag is added in the data synchronization log file by thread.
204, error identification is added in the data synchronization log file by thread.
Optionally, in the implementation procedure of synchronous task, after step 204, may also include
205, abnormality processing queue is added in the data with error identification.
Certainly, if the data synchronous abnormality, equally can also be added abnormality processing queue.
As can be seen that by this abnormality processing exception can be carried out to the data of synchronization failure either synchronous abnormality
Reason, so that data synchronous task smoothly completes.
Optionally, when failure of data synchronization or exception, the method also includes,
206, the counting of anomalous counts device is increased.
When an exception occurs, not only abnormality processing queue can be added in the task of synchronization failure or exception, it can also should
The anomalous counts value of thread increases.
207, when the counting of anomalous counts device reaches default outlier threshold, thread is prolonged when executing data and synchronizing with abnormal
When interval to the data synchronous task carry out delay process.
Certainly, if not up to default outlier threshold, then thread execute data it is synchronous when take that be immediately performed data same
Step task.
It is understood that an anomalous counts device can be safeguarded in systems, in failure of data synchronization or exception,
The numerical value of the counter is increased, under original state, the numerical value of the counter is 0,, should when this count down to outlier threshold
For thread in the data that processing grabs, system caused by abnormal stacking is unstable in order to prevent, will not immediately treat the data,
But handled again after abnormal time delay interval, when the data of certain primary thread process synchronize successfully, just meeting
The counting of the anomalous counts device is reset, in this manner, may further ensure that the stability of data synchronous system.
In addition, it is optional, exception immediately occurs during thread executes data synchronous task, then by synchronous abnormality
Abnormality processing queue is added in data；And/or
It, will be true by data synchronization log file when data synchronous task is completed or data synchronous system is restarted
It is fixed have start mark and without success flag, or abnormality processing queue is added in the data with error identification.
As can be seen that the data into abnormality processing queue have various situations, e.g. since thread executes data
Occur during synchronous task it is immediately abnormal, then by exception when data addition abnormality processing queue, in another example, work as data
Synchronous task complete when, data synchronization log file during this subtask can be analyzed, will have start mark and
Abnormality processing queue is added in data without success flag, or with error identification；In another example data synchronous system is restarted
When, at this time due to the loss of data in memory, the partial data in data synchronous task will appear exception, and can equally inquire can
To analyze data synchronization log file during this subtask, will have and start mark and without success flag, or tool
Abnormality processing queue is added in the data of wrong mark.By this processing mode, no matter so that being synchronized by which kind of situation
Abnormal data can obtain appropriate processing, to reach under the extreme conditions such as abnormal timely processing and system failure
Data restore effect.
The method of data synchronization of the embodiment of the present invention is described above, below to the terminal of the embodiment of the present invention into
Row is introduced, referring to Fig. 3, Fig. 3 is one embodiment figure of the terminal of the embodiment of the present invention, wherein the terminal 3 can include:
Detection module 301, for detecting the data synchronous task that isochronous queue is added；
Distribution module 302, for distributing thread pool for the data synchronous task；
Determining module 303, task start time and task for determining the data synchronous task execute duration in advance,
The task executes duration in advance and is preset in the data synchronous system for executing the data synchronous task；
Processing module 304, for being greater than in the difference at current time and the task start time of the data synchronous task
Task executes duration in advance, and the thread started in thread pool is monitored the data synchronous task；Or,
For when the difference at current time and the task start time of the data synchronous task shifts to an earlier date no more than task
Long, then all threads opened in the thread pool execute the data synchronous task.
Wherein, which is able to achieve the function of step 101 in embodiment illustrated in fig. 1, specific detection process and figure
Step 101 is similar in 1 illustrated embodiment, and details are not described herein again；
The distribution module 302 is able to achieve the function of step 102 in embodiment illustrated in fig. 1, shown in specific detection process and Fig. 1
Step 102 is similar in embodiment, and details are not described herein again；
The determining module 303 is able to achieve the function of step 103 in embodiment illustrated in fig. 1, shown in specific detection process and Fig. 1
Step 103 is similar in embodiment, and details are not described herein again；
The processing module 304 is able to achieve the function of step 104 to 106 in embodiment illustrated in fig. 1, specific detection process and figure
Step 104 is to 106 similar in 1 illustrated embodiment, and details are not described herein again.
Optionally, the maximum thread amount of the data synchronous system is preset threshold N, and N is the integer more than or equal to 2,
The distribution module 302 is specifically used for:
The thread pool of default number of threads is distributed for the data synchronous task.
Optionally, the distribution module 302 is also used to:
Headed by time data synchronous task distribution number of threads be N/2 thread pool；
Have for the distribution of next data synchronous task when time thread pool of the half number of threads of data synchronous task.
Optionally, the processing module 304 is also used to:
When the remaining number of threads of the data synchronous system is less than critical value, the data of the isochronous queue will be added
Synchronous task is set to wait for state.
Optionally, the processing module 304 is specifically used for:
The thread will start mark and data information is added in data synchronization log file；
When data synchronize successfully, success flag is added in the data synchronization log file by thread；Or.
When failure of data synchronization or exception, error identification is added in the data synchronization log file by thread.
Optionally, when failure of data synchronization or exception, the processing module 304 is also used to:
The counting of anomalous counts device is increased；
When the counting of the anomalous counts device reaches default outlier threshold, the thread is with abnormal time delay interval to described
Data synchronous task carries out delay process.
Optionally, the processing module 304 is also used to:
When the thread, which successfully completes data, to be synchronized, the counting of the anomalous counts device is reset.
Optionally, the processing module 304 is also used to:
Exception immediately occurs during thread executes data synchronous task, exception is added in the data of synchronous abnormality
Manage queue；And/or
It, will be true by data synchronization log file when data synchronous task is completed or data synchronous system is restarted
It is fixed have start mark and without success flag, or abnormality processing queue is added in the data with error identification.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In several embodiments provided herein, it should be understood that disclosed system, device and method can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit
It divides, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components
It can be combined or can be integrated into another system, or some features can be ignored or not executed.Another point, it is shown or
The mutual coupling, direct-coupling or communication connection discussed can be through some interfaces, the indirect coupling of device or unit
It closes or communicates to connect, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated unit is realized in the form of SFU software functional unit and sells or use as independent product
When, it can store in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially
The all or part of the part that contributes to existing technology or the technical solution can be in the form of software products in other words
It embodies, which is stored in a storage medium, including some instructions are used so that a computer
Equipment (can be personal computer, server or the network equipment etc.) executes the complete of each embodiment the method for the present invention
Portion or part steps.And storage medium above-mentioned includes: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only
Memory), random access memory (RAM, Random Access Memory), magnetic or disk etc. are various can store journey
The medium of sequence code.
The above, the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although referring to before
Stating embodiment, invention is explained in detail, those skilled in the art should understand that: it still can be to preceding
Technical solution documented by each embodiment is stated to modify or equivalent replacement of some of the technical features；And these
It modifies or replaces, the spirit and scope for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution.
Claims (16)
1. a kind of method of data synchronization characterized by comprising
The data synchronous task of isochronous queue is added in detection；
Thread pool is distributed for the data synchronous task；
Determine that the task start time of the data synchronous task and task execute duration in advance, the task executes duration in advance
It is preset in the data synchronous system for executing the data synchronous task；
If the difference of current time and the task start time of the data synchronous task are greater than task executes duration in advance, open
In the thread pool thread is moved to be monitored the data synchronous task；Or,
If the difference of current time and the task start time of the data synchronous task shift to an earlier date duration no more than task, open
All threads in the thread pool execute the data synchronous task.
2. method of data synchronization according to claim 1, which is characterized in that the maximum thread of the data synchronous system
Amount is preset threshold N, and N is the integer more than or equal to 2, described to include: for data synchronous task distribution thread pool
The thread pool of default number of threads is distributed for the data synchronous task.
3. method of data synchronization according to claim 2, which is characterized in that the method also includes:
Headed by time data synchronous task distribution number of threads be N/2 thread pool；
Have for the distribution of next data synchronous task when time thread pool of the half number of threads of data synchronous task.
4. method of data synchronization according to claim 3, which is characterized in that when the remaining thread of the data synchronous system
When quantity is less than critical value, the data synchronous task that the isochronous queue is added is set to wait for state.
5. method of data synchronization according to any one of claim 1 to 4, which is characterized in that the data synchronous task
Include:
The thread will start mark and data information is added in data synchronization log file；
When data synchronize successfully, success flag is added in the data synchronization log file by thread；Or,
When failure of data synchronization or exception, error identification is added in the data synchronization log file by thread.
6. method of data synchronization according to claim 5, which is characterized in that described when failure of data synchronization or exception
Method further includes increasing the counting of anomalous counts device；
When the counting of the anomalous counts device reaches default outlier threshold, the thread is with abnormal time delay interval to the data
Synchronous task carries out delay process.
7. method of data synchronization according to claim 6, which is characterized in that the method also includes, when the thread at
When function completion data are synchronous, the counting of the anomalous counts device is reset.
8. method of data synchronization according to claim 7, which is characterized in that the method also includes:
Exception immediately occurs during thread executes data synchronous task, abnormality processing team is added in the data of synchronous abnormality
Column；And/or
When data synchronous task is completed or data synchronous system is restarted, by what is determined by data synchronization log file
With starting mark and without success flag, or abnormality processing queue is added in data with error identification.
9. a kind of terminal characterized by comprising
Detection module, for detecting the data synchronous task that isochronous queue is added；
Distribution module, for distributing thread pool for the data synchronous task；
Determining module, task start time and task for determining the data synchronous task execute duration in advance, and described
Business executes duration in advance and is preset in the data synchronous system for executing the data synchronous task；
Processing module, for shifting to an earlier date in the difference at current time and the task start time of the data synchronous task greater than task
Duration is executed, starts in the thread pool thread and the data synchronous task is monitored；Or,
Shift to an earlier date duration no more than task for the difference at current time and the task start time of the data synchronous task, then
All threads opened in the thread pool execute the data synchronous task.
10. terminal according to claim 9, which is characterized in that the maximum thread amount of the data synchronous system is pre-
If threshold value N, N are the integer more than or equal to 2, the distribution module is specifically used for:
The thread pool of default number of threads is distributed for the data synchronous task.
11. terminal according to claim 10, which is characterized in that the distribution module is also used to:
Headed by time data synchronous task distribution number of threads be N/2 thread pool；
Have for the distribution of next data synchronous task when time thread pool of the half number of threads of data synchronous task.
12. terminal according to claim 11, which is characterized in that the processing module is also used to:
It is when the remaining number of threads of the data synchronous system is less than critical value, the data that the isochronous queue is added are synchronous
Task is set to wait for state.
13. the terminal according to any one of claim 9 to 12, which is characterized in that the processing module is specifically used for:
The thread will start mark and data information is added in data synchronization log file；
When data synchronize successfully, success flag is added in the data synchronization log file by thread；Or,
When failure of data synchronization or exception, error identification is added in the data synchronization log file by thread.
14. terminal according to claim 13, which is characterized in that when failure of data synchronization or exception, the processing mould
Block is also used to:
The counting of anomalous counts device is increased；
When the counting of the anomalous counts device reaches default outlier threshold, the thread is with abnormal time delay interval to the data
Synchronous task carries out delay process.
15. terminal according to claim 14, which is characterized in that the processing module is also used to:
When the thread, which successfully completes data, to be synchronized, the counting of the anomalous counts device is reset.
16. terminal according to claim 15, which is characterized in that the processing module is also used to:
Exception immediately occurs during thread executes data synchronous task, abnormality processing team is added in the data of synchronous abnormality
Column；And/or
When data synchronous task is completed or data synchronous system is restarted, by what is determined by data synchronization log file
With starting mark and without success flag, or abnormality processing queue is added in data with error identification.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610938969.2A CN106412088B (en) | 2016-10-25 | 2016-10-25 | A kind of method of data synchronization and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610938969.2A CN106412088B (en) | 2016-10-25 | 2016-10-25 | A kind of method of data synchronization and terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106412088A CN106412088A (en) | 2017-02-15 |
| CN106412088B true CN106412088B (en) | 2019-10-18 |
Family
ID=58013060
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610938969.2A Active CN106412088B (en) | 2016-10-25 | 2016-10-25 | A kind of method of data synchronization and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106412088B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107908472A (en) * | 2017-09-30 | 2018-04-13 | 平安科技（深圳）有限公司 | Data synchronization unit, method and computer-readable recording medium |
| CN110602165B (en) * | 2019-08-08 | 2022-09-06 | 数字广东网络建设有限公司 | Government affair data synchronization method, device, system, computer equipment and storage medium |
| CN115017235B (en) * | 2022-06-30 | 2023-07-14 | 上海弘玑信息技术有限公司 | Data synchronization method, electronic device and storage medium |
| CN117009309B (en) * | 2023-05-31 | 2024-05-07 | 合芯科技(苏州)有限公司 | File real-time synchronization method and device based on rsync |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104166590A (en) * | 2013-05-20 | 2014-11-26 | 阿里巴巴集团控股有限公司 | Task scheduling method and system |
| CN104199730A (en) * | 2014-08-29 | 2014-12-10 | 浪潮集团有限公司 | Single-thread multi-task processing method based on synchronous I/O multiplexing mechanism |
| CN104699541A (en) * | 2015-03-30 | 2015-06-10 | 北京奇虎科技有限公司 | Method, device, data transmission assembly and system for synchronizing data |
| CN104750543A (en) * | 2013-12-26 | 2015-07-01 | 杭州华为数字技术有限公司 | Thread creation method, business request processing method and associated device |
| CN105338051A (en) * | 2014-08-15 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Data synchronization method and equipment |
-
2016
- 2016-10-25 CN CN201610938969.2A patent/CN106412088B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104166590A (en) * | 2013-05-20 | 2014-11-26 | 阿里巴巴集团控股有限公司 | Task scheduling method and system |
| CN104750543A (en) * | 2013-12-26 | 2015-07-01 | 杭州华为数字技术有限公司 | Thread creation method, business request processing method and associated device |
| CN105338051A (en) * | 2014-08-15 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Data synchronization method and equipment |
| CN104199730A (en) * | 2014-08-29 | 2014-12-10 | 浪潮集团有限公司 | Single-thread multi-task processing method based on synchronous I/O multiplexing mechanism |
| CN104699541A (en) * | 2015-03-30 | 2015-06-10 | 北京奇虎科技有限公司 | Method, device, data transmission assembly and system for synchronizing data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106412088A (en) | 2017-02-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107291547B (en) | Task scheduling processing method, device and system | |
| CN106412088B (en) | A kind of method of data synchronization and terminal | |
| CN107025205B (en) | Method and equipment for training model in distributed system | |
| CN109814998A (en) | A kind of method and device of multi-process task schedule | |
| CN105468450A (en) | Task scheduling method and system | |
| CN107092521B (en) | Distributed task scheduling method, device and system | |
| CN106933672B (en) | Distributed environment coordinated consumption queue method and device | |
| US20150254105A1 (en) | Data processing system, data processing method, and program | |
| CN110389843B (en) | Service scheduling method, device, equipment and readable storage medium | |
| CN109101342B (en) | Distributed job coordination control method and device, computer equipment and storage medium | |
| CN106598801A (en) | Coroutine monitoring method and apparatus | |
| CN114153580A (en) | Cross-multi-cluster work scheduling method and device | |
| CN109062681A (en) | A kind of execution method, system, device and the storage medium of periodic cycle task | |
| CN111427670A (en) | Task scheduling method and system | |
| CN109871273A (en) | A kind of adaptive task moving method and device | |
| CN110764887A (en) | Task rescheduling method and system, and related equipment and device | |
| CN115658311A (en) | Resource scheduling method, device, equipment and medium | |
| CN105607968A (en) | Incremental backup method and equipment | |
| CN112860401B (en) | Task scheduling method, device, electronic equipment and storage medium | |
| CN114518974A (en) | Checkpoint recovery method, device, equipment and medium for data processing task | |
| US10970098B2 (en) | Methods for sharing input-output device for process automation on a computing machine and devices thereof | |
| CN106598706B (en) | Method and device for improving stability of server and server | |
| CN103761106A (en) | Process control method and process engine | |
| CN115718741A (en) | Automatic cleaning method and device for health data, electronic equipment and storage medium | |
| CN105549911B (en) | The data transmission method and device of NVRAM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |