CN108733471A - A kind of task dissemination system and method - Google Patents
A kind of task dissemination system and method Download PDFInfo
- Publication number
- CN108733471A CN108733471A CN201710278268.5A CN201710278268A CN108733471A CN 108733471 A CN108733471 A CN 108733471A CN 201710278268 A CN201710278268 A CN 201710278268A CN 108733471 A CN108733471 A CN 108733471A
- Authority
- CN
- China
- Prior art keywords
- task
- pending
- performer
- unit
- distributor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of task dissemination system and methods, the system includes setup module, task distributor and task performer, wherein, setup module, for carrying out preload process to pending task, and choose that treated some pending task carries out plus optimistic lock processing, the pending task after the locking is sent to task distributor；The pending task for selecting task performer by preset rules, and is sent to the task performer, task performer, for carrying out task distribution processor to the pending task by task distributor.The real-time of raising task distribution is realized through the invention and reduces the purpose of deadlock.
Description
Technical field
The present invention relates to technical field of data processing, more particularly to one kind of the tasks in parallel distribution mechanisms based on queue
Task dissemination system and method.
Background technology
With the rapid development of development of Mobile Internet technology and the continuous upgrading of network, the task processing of big data quantity is faced
When, if the processing time of task will be elongated using the processing mode of serial task, so operator generally can be to acquisition
Data use parallel processing manner, the effect in order to improve task distribution processor also to propose higher want to parallel processing capability
It asks.
In the prior art, for the main processing side using the concurrent tasks based on multithreading of parallel task processing
Method, i.e., the multiple threads for different tasks being distributed to operating system some process go to handle, in processing procedure in order to avoid
Between multithreading the problem of task contention, it will usually be realized in such a way that database row grade is locked, but row grade lock also has centainly
The shortcomings that, for example expense is big and locks slowly, and deadlock is especially will appear in the case where concurrency is larger, reduces task distribution
Real-time.
Invention content
The above problem, a kind of task dissemination system of present invention offer and method are provided, the distribution of raising task is realized
Real-time and the purpose for reducing deadlock.
To achieve the goals above, according to the first aspect of the invention, a kind of task dissemination system is provided, which sets
Set module, task distributor and task performer, wherein
The setup module for carrying out preload process to pending task, and chooses that treated some waits holding
Row task carries out plus optimistic lock processing, and the pending task after the locking is sent to the task distributor；
The task distributor selects the task performer for passing through preset rules, and described pending is appointed
Business is sent to the task performer, wherein the preset rules are to select the task performer of most lightly loaded；
The task performer, for carrying out task distribution processor to the pending task.
Preferably, the setup module includes pre-add carrier unit and locking unit, wherein
The pre-add carrier unit, for the load quantity according to preset pending task, by the pending task
From being loaded into database in the pending queue of memory；
Unit is locked, it, will for choosing some pending task according to priority in the pending task queue
The pending task is lockked using optimism, and the pending task after the locking is sent to the task
In distributor.
Preferably, the task distributor includes acquiring unit, selecting unit and transmission unit, wherein
The acquiring unit, for judging whether the task performer is carrying out task distribution, if it is not, then obtaining
The task performer；
The selecting unit, the task performer minimum for the selection load in the task performer got are made
For task performer to be operated；
The transmission unit, for pending task to be sent in the task performer to be operated.
Preferably, the setup module further includes unlocking unit, wherein
After the task distributor is distributed processing to the pending task, the unlocking unit, being used for will
The pending task is unlocked, wherein it is described unlock for by the pending task in the pending queue into
Row is deleted.
According to the second aspect of the invention, a kind of task distribution method is provided, this method is suitable for first party of the present invention
Task dissemination system described in face, the task dissemination system include setup module, task distributor and task performer, the party
Method includes：
The setup module carries out preload process to pending task, and some pending task of choosing that treated
It carries out plus optimistic lock is handled, the pending task after the locking is sent to the task distributor；
The task distributor selects the task performer by preset rules, and the pending task is sent
To the task performer, wherein the preset rules are to select the task performer of most lightly loaded；
The task performer carries out task distribution processor to the pending task.
Preferably, the setup module includes pre-add carrier unit and locking unit, wherein the setup module is to pending
Task carry out preload process, and choose that treated some pending task carry out plus optimistic lock processing, by the locking
Pending task afterwards is sent to the task distributor, including：
The pre-add carrier unit according to preset pending task load quantity, by the pending task from data
It is loaded into library in the pending queue of memory；
The locking unit chooses some pending task in the pending task queue according to priority, and will
The pending task is lockked using optimism, and the pending task after the locking is sent to the task
In distributor.
Preferably, the task distributor includes acquiring unit, selecting unit and transmission unit, wherein the task point
Hair device selects the task performer by preset rules, and the pending task is sent to the task performer,
Including：
The acquiring unit judges whether the task performer is carrying out task distribution, if it is not, then described in obtaining
Task performer；
The task performer that the selecting unit selects load minimum in the task performer got is as waiting for
The task performer of operation；
The transmission unit, for pending task to be sent in the task performer to be operated.
Preferably, the setup module further includes unlocking unit, wherein when the task distributor is to described pending
After task is distributed processing,
The pending task is unlocked by the unlocking unit, wherein the unlock is by described pending
Business is deleted in the pending queue.
Compared to the prior art, in the task dissemination system of the present invention, pending is appointed by setup module first
Business preload and add the processing of optimistic lock, can be loaded into pending task from database by preload process
In memory in pending task queue, then obtains and the executing of the task is needed to be lockked with optimistic lock, it can be with by preload process
Ensure that the same pending task is not repeated execution, deadlock can be reduced using the locking processing of optimism lock, then passed through
Task has been distributed is respectively sent to task performer by pending task, and selected task performer is most lightly loaded
Task performer, can improve execution efficiency in this way can reduce accumulation in task distribution procedure and not in time, last task
Actuator is distributed processing to pending task.Finally, it realizes the real-time of raising task distribution and reduces deadlock
Purpose.
Description of the drawings
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of structural schematic diagram for task dissemination system that the embodiment of the present invention one provides；
Fig. 2 is a kind of flow diagram of task distribution method provided by Embodiment 2 of the present invention.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Term " first " and " second " in description and claims of this specification and above-mentioned attached drawing etc. are to be used for area
Not different objects, rather than for describing specific sequence.In addition term " comprising " and " having " and their any deformations,
It is intended to cover and non-exclusive includes.Such as it contains the process of series of steps or unit, method, system, product or sets
It is standby not to be set in the step of having listed or unit, but the step of may include not listing or unit.
Embodiment one
Be a kind of task dissemination system that provides of the embodiment of the present invention one referring to Fig. 1, the system comprises setup module 1,
Task distributor 2 and task performer 3, wherein
The setup module 1 for carrying out preload process to pending task, and chooses that treated some waits holding
Row task carries out plus optimistic lock processing, and the pending task after the locking is sent to the task distributor；
The task distributor 2 selects the task performer for passing through preset rules, and described pending is appointed
Business is sent to the task performer, wherein the preset rules are to select the task performer of most lightly loaded；
The task performer 3, for carrying out task distribution processor to the pending task.
Specifically, the setup module 1 includes pre-add carrier unit 11 and locking unit 12, wherein
The pre-add carrier unit 11 appoints described pending for the load quantity according to preset pending task
Business in database from being loaded into the pending queue of memory；
The locking unit 12, for choosing some pending times according to priority in the pending task queue
Business, and the pending task is lockked using optimism, and the pending task after the locking is sent to
In the task distributor.
Wherein, locking unit can be according to the priority and loading sequence of the pending task in the pending task queue
Then one pending task of selection carries out locking processing, be then issued to task distributor.
Specifically, by pre-add carrier unit by pending task from being loaded into pending task team in memory in database
When in row, can mainly operation be obtained to the pending task data of formulation and be dispensed to the pending task list
In, it can be called when needed.User can be made only to carry out application and use data in this way；Correspondingly,
Asynchronous loading can also be carried out, at this time it needs to be determined that whom user, which is, could carry out preload process.From the angle of user
From the point of view of, no matter being that it is preloaded either with or without promoter, it can apply and take desired data, i.e., when not initiating
When person is that the data of the pending task carry out preload process, then application is to be loaded to data, if
It has initiated to preload, and data have loaded completion, then are directly obtained the data loaded；If data are not complete
At then data consumer needs after the completion of waiting pending datas load, can just obtain data and continue current operation.
The pending task of acquisition is lockked by locking unit, and using the mode of optimism lock, and adopted
Go modification record with the mode of CAS, wherein CAS refer to a kind of shared data in memory that modern CPU is supported extensively into
A kind of special instruction of row operation, the operating process of the instruction is, first, CPU can by memory by data to be altered with
Desired value compares.Then, when the two values are equal, the numerical value in memory can just be replaced with new value by CPU.Otherwise just
It does not operate.Finally, CPU can return to old numerical value.This series of operation is atom.Although they seem complicated,
It is that 5 concurrent mechanisms of Java are basic better than original lock mechanism.
It is understood that in the case of multiple scheduling threads, the preloading data in memory is using different step-lengths
Mode reduces the probability of Data duplication in two schedulers.
Correspondingly, the task distributor 2 includes acquiring unit 21, selecting unit 22 and transmission unit 23, wherein
The acquiring unit 21, for judging whether the task performer is carrying out task distribution, if it is not, then obtaining
Take the task performer；
The selecting unit 22, the task performer minimum for the selection load in the task performer got
As task performer to be operated；
The transmission unit 23, for pending task to be sent in the task performer to be operated.
Specifically, the task performer can be judged whether in carry out task point by acquiring unit and selecting unit
Hair, and the task performer of relative load minimum is selected, the real-time of task distribution can be improved in this way, preferably utilize money
Source.
Correspondingly, the setup module further includes unlocking unit 13, wherein
After the task distributor is distributed processing to the pending task, the unlocking unit 13 is used for
The pending task is unlocked, wherein it is described unlock for by the pending task in the pending queue
It is deleted.
Technical solution disclosed in one through the embodiment of the present invention first carries out pending task by setup module pre-
Load and the processing for adding optimistic lock can be waited for pending task from being loaded into memory in database by preload process
It executes in task queue, then obtains the needing to execute of the task and lockked with optimistic lock, can be ensured by preload process same
A pending task is not repeated execution, can reduce deadlock using the locking processing of optimism lock, then be distributed by task
It rises and pending task is respectively sent to task performer, and the task execution that selected task performer is most lightly loaded
Device, can improve execution efficiency in this way can reduce accumulation in task distribution procedure and not in time, last task performer pair
Pending task is distributed processing.Finally, it realizes the real-time of raising task distribution and reduces the purpose of deadlock.
Embodiment two
Corresponding with a kind of task dissemination system disclosed in the embodiment of the present invention one, the embodiment of the present invention two also provides
A kind of task distribution method, the flow referring to a kind of Fig. 2 task distribution methods provided by the embodiment of the present invention two are illustrated
Figure, the method are suitable for any type task dissemination system described in the embodiment of the present invention one, task distribution system
System includes setup module, task distributor and task performer, and this approach includes the following steps：
S11, the setup module carry out preload process to pending task, and choose that treated some is pending
Task carries out plus optimistic lock processing, and the pending task after the locking is sent to the task distributor；
S12, the task distributor select the task performer by preset rules, and by the pending task
It is sent to the task performer, wherein the preset rules are to select the task performer of most lightly loaded；
S13, the task performer carry out task distribution processor to the pending task.
Correspondingly, the setup module includes pre-add carrier unit and locking unit, wherein the setup module is to pending
Task carry out preload process, and choose that treated some pending task carry out plus optimistic lock processing, by the locking
Pending task afterwards is sent to the task distributor, including：
The pre-add carrier unit according to preset pending task load quantity, by the pending task from data
It is loaded into library in the pending queue of memory；
It locks unit and chooses some pending task according to priority in the pending task queue, and will be described
Pending task is lockked using optimism, and the pending task after the locking is sent to the task and is distributed
In device.
Corresponding, the task distributor includes acquiring unit, selecting unit and transmission unit, wherein the task point
Hair device selects the task performer by preset rules, and the pending task is sent to the task performer,
Including：
The acquiring unit judges whether the task performer is carrying out task distribution, if it is not, then described in obtaining
Task performer；
The task performer that the selecting unit selects load minimum in the task performer got is as waiting for
The task performer of operation；
The transmission unit, for pending task to be sent in the task performer to be operated.
Correspondingly, the setup module further includes unlocking unit, wherein when the task distributor is to described pending
After task is distributed processing,
The pending task is unlocked by the unlocking unit, wherein the unlock is by described pending
Business is deleted in the pending queue.
In the embodiment of the present invention two, pending task is carried out by setup module preloading and adding optimism first
The processing of lock, can be by pending task from being loaded into pending task queue in memory in database by preload process
In, it then obtains the needing to execute of the task and is lockked with optimistic lock, same pending appoint can be ensured by preload process
Business is not repeated execution, can reduce deadlock using the locking processing of optimism lock, then having distributed by task will be pending
Task is respectively sent to task performer, and the task performer that selected task performer is most lightly loaded, in this way can be with
Accumulation in task distribution procedure and not in time can be reduced by improving execution efficiency, and last task performer is to pending task
It is distributed processing.Finally, it realizes the real-time of raising task distribution and reduces the purpose of deadlock.
Each embodiment is described by the way of progressive in this specification, the highlights of each of the examples are with other
The difference of embodiment, just to refer each other for identical similar portion between each embodiment.For device disclosed in embodiment
For, since it is corresponded to the methods disclosed in the examples, so description is fairly simple, related place is said referring to method part
It is bright.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest range caused.
Claims (8)
1. a kind of task dissemination system, which is characterized in that the system comprises setup module, task distributor and task executions
Device, wherein
The setup module for carrying out preload process to pending task, and chooses that treated some pending
Business carries out plus optimistic lock processing, and the pending task after the locking is sent to the task distributor；
The task distributor selects the task performer for passing through preset rules, and the pending task is sent out
It send to the task performer, wherein the preset rules are to select the task performer of most lightly loaded；
The task performer, for carrying out task distribution processor to the pending task.
2. system according to claim 1, which is characterized in that the setup module includes pre-add carrier unit and locks single
Member, wherein
The pre-add carrier unit, for the load quantity according to preset pending task, by the pending task from number
According to being loaded into library in the pending queue of memory；
Unit is locked, it, will be described for choosing some pending task according to priority in the pending task queue
Pending task is lockked using optimism, and the pending task after the locking is sent to the task and is distributed
In device.
3. system according to claim 1, which is characterized in that the task distributor includes acquiring unit, selecting unit
And transmission unit, wherein
The acquiring unit, for judging whether the task performer is carrying out task distribution, if it is not, then described in obtaining
Task performer；
The selecting unit, task performer for selecting load minimum in the task performer got is as waiting for
The task performer of operation；
The transmission unit, for pending task to be sent in the task performer to be operated.
4. system according to claim 2, which is characterized in that the setup module further includes unlocking unit, wherein
After the task distributor is distributed processing to the pending task, the unlocking unit, being used for will be described
Pending task is unlocked, wherein the unlock is to delete the pending task in the pending queue
It removes.
5. a kind of task distribution method, which is characterized in that the method is suitable for the task described in claim 1-4 any one
Dissemination system, the task dissemination system include that setup module, task distributor and task performer, this method include：
The setup module carries out preload process to pending task, and chooses that treated some pending task carries out
Add optimistic lock to handle, the pending task after the locking is sent to the task distributor；
The task distributor selects the task performer by preset rules, and the pending task is sent to institute
State task performer, wherein the preset rules are to select the task performer of most lightly loaded；
The task performer carries out task distribution processor to the pending task.
6. according to the method described in claim 5, it is characterized in that, the setup module includes pre-add carrier unit and locks single
Member, wherein the setup module carries out preload process to pending task, and some pending task of choosing that treated
It carries out plus optimistic lock is handled, the pending task after the locking is sent to the task distributor, including：
The pre-add carrier unit according to preset pending task load quantity, by the pending task from database
It is loaded into the pending queue of memory；
It locks unit and chooses some pending task according to priority in the pending task queue, it will be described pending
Task lockked using optimism, and the pending task after the locking is sent in the task distributor.
7. according to the method described in claim 5, it is characterized in that, the task distributor includes acquiring unit, selecting unit
And transmission unit, wherein the task distributor selects the task performer by preset rules, and will be described pending
Task is sent to the task performer, including：
The acquiring unit judges whether the task performer is carrying out task distribution, if it is not, then obtaining the task
Actuator；
The task performer that the selecting unit selects load minimum in the task performer got is as to be operated
Task performer；
The transmission unit, for pending task to be sent in the task performer to be operated.
8. according to the method described in claim 5, it is characterized in that, the setup module further includes unlocking unit, wherein work as institute
It states after task distributor is distributed processing to the pending task,
The pending task is unlocked by the unlocking unit, wherein the unlock is that the pending task exists
It is deleted in the pending queue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710278268.5A CN108733471A (en) | 2017-04-25 | 2017-04-25 | A kind of task dissemination system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710278268.5A CN108733471A (en) | 2017-04-25 | 2017-04-25 | A kind of task dissemination system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108733471A true CN108733471A (en) | 2018-11-02 |
Family
ID=63934874
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710278268.5A Pending CN108733471A (en) | 2017-04-25 | 2017-04-25 | A kind of task dissemination system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108733471A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101556597A (en) * | 2009-03-06 | 2009-10-14 | 南京航空航天大学 | Self-adaptive optimistic concurrency control method |
| CN104050029A (en) * | 2014-05-30 | 2014-09-17 | 北京先进数通信息技术股份公司 | Task scheduling system |
| CN104731645A (en) * | 2015-03-19 | 2015-06-24 | 蔡树彬 | Task scheduling method and device and data downloading method and device |
| CN105511969A (en) * | 2015-11-25 | 2016-04-20 | 中国船舶工业系统工程研究院 | Inter-process thread mutex method |
| US20170031730A1 (en) * | 2015-07-28 | 2017-02-02 | International Business Machines Corporation | Reentrant read-write lock algorithm |
-
2017
- 2017-04-25 CN CN201710278268.5A patent/CN108733471A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101556597A (en) * | 2009-03-06 | 2009-10-14 | 南京航空航天大学 | Self-adaptive optimistic concurrency control method |
| CN104050029A (en) * | 2014-05-30 | 2014-09-17 | 北京先进数通信息技术股份公司 | Task scheduling system |
| CN104731645A (en) * | 2015-03-19 | 2015-06-24 | 蔡树彬 | Task scheduling method and device and data downloading method and device |
| US20170031730A1 (en) * | 2015-07-28 | 2017-02-02 | International Business Machines Corporation | Reentrant read-write lock algorithm |
| CN105511969A (en) * | 2015-11-25 | 2016-04-20 | 中国船舶工业系统工程研究院 | Inter-process thread mutex method |
Non-Patent Citations (2)
| Title |
|---|
| 杨铸等: "《深入浅出嵌入式底层软件开发》", 31 May 2011, 北京航空航天大学出版社 * |
| 韩景倜，何杰: "《助理云经纪师》", 31 January 2015 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107977376B (en) | Distributed database system and transaction processing method | |
| JP3672208B2 (en) | Hierarchical transaction processing method | |
| Wang et al. | Scaling multicore databases via constrained parallel execution | |
| US8112751B2 (en) | Executing tasks through multiple processors that process different portions of a replicable task | |
| KR101209023B1 (en) | Modified computer architecture with coordinated objects | |
| Jin et al. | A high performance concurrency protocol for smart contracts of permissioned blockchain | |
| US20060048161A1 (en) | Resource allocation method and system | |
| Milman et al. | BQ: A lock-free queue with batching | |
| US20110185360A1 (en) | Multiprocessing transaction recovery manager | |
| Pandey et al. | CART: a real-time concurrency control protocol | |
| Posner et al. | Cooperation vs. coordination for lifeline-based global load balancing in APGAS | |
| US7752399B2 (en) | Exclusion control method and information processing apparatus | |
| Pandey et al. | RACE: A concurrency control protocol for time-constrained transactions | |
| US8473954B2 (en) | Executing operations via asynchronous programming model | |
| CN108733471A (en) | A kind of task dissemination system and method | |
| Atkins et al. | Adaptable concurrency control for atomic data types | |
| US20090235273A1 (en) | Future scheduling by direct representation of possible dependencies | |
| Giakkoupis et al. | Efficient randomized DCAS | |
| de Alfaro et al. | Code aware resource management | |
| Yin et al. | A data-aware workflow scheduling algorithm for heterogeneous distributed systems | |
| Sang et al. | Scalable and serializable networked multi-actor programming | |
| CN113448710B (en) | Distributed application system based on business resources | |
| Lam et al. | The reduced ceiling protocol for concurrency control in real-time databases with mixed transactions | |
| Moschoyiannis et al. | Long-running transactions: semantics, schemas, implementation | |
| Mittal et al. | Dynamic versus static locking in real-time parallel database systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181102 |
|
| RJ01 | Rejection of invention patent application after publication |