CN105607956B - Method for allocating tasks and system in a kind of computer - Google Patents
Method for allocating tasks and system in a kind of computer Download PDFInfo
- Publication number
- CN105607956B CN105607956B CN201610009481.1A CN201610009481A CN105607956B CN 105607956 B CN105607956 B CN 105607956B CN 201610009481 A CN201610009481 A CN 201610009481A CN 105607956 B CN105607956 B CN 105607956B
- Authority
- CN
- China
- Prior art keywords
- task
- equal
- queue
- pool
- computer according
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multi Processors (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present invention provides method for allocating tasks and system in a kind of computer, for the task in m task pool to be distributed to p process, each task pool has n storage unit, method first arranges p process in order, obtain process queue, then in the two-dimensional matrix of the total m × n storage unit composition of m task pool, process is assigned the task to according to putting in order for ranks.The present invention is that the storage unit in multiple tasks pond is regarded as to a two-dimensional matrix, then the element in matrix is sequentially allocated to process, the task pool of the process number, each process that possess each task pool occupied can guarantee equilibrium, to realize the load balancing of process and guarantee that task is efficiently executed.
Description
Technical field
The invention belongs to the method for allocating tasks and system in computer field more particularly to a kind of computer.
Background technique
Require Timing Processing task in many of software systems business scenario, handle the main bodys of these tasks usually by
" process " undertakes.In the case where task amount is small, one " process " can be completed these tasks.But, reliability big in task amount
In demanding situation, it has to be handled using the mode of distributed multithreading.Multithread mode is when it come to arrived, that just leads
The problem of how task distributes between these " processes " related to out.
Quartz is the Java open source projects that OpenSymphony open source group is woven in the field Job scheduling.
The core of Quartz frame is scheduler, it is responsible for managing Quartz application runtime environment.Scheduler is not to do institute on one's own account
Some work, but rely on some very important components in frame.Quartz is not only process and management of process.To ensure
Scalability, Quartz use the framework based on multi-process.When starting, a set of worker process of framework initialization, this is packed into
Journey is scheduled device for executing scheduled task, and such Quartz can concurrently run multiple tasks.Quartz relies on a set of pine
The process pool of coupling manages component to manage thread context.
In brief, Quartz is the distributed timer frame that can run multiple tasks parallel, mature and stable
And using extensive.It is by relevant database and JDBC job storage shared data to realize distributed purpose.
Quartz is powerful, but just because of its power, causes the dependence to environment very harsh, resource disappears
It consumes also higher.Distributed deployment is done using database simultaneously, efficiency is relatively low.Its group scheme and configuration is also more numerous
It is trivial, there is certain upper hand difficulty for new hand.In short, the frame that the support distributed task scheduling that Quartz is a heavyweight executes
Frame uses and needs to spend certain learning cost, although reliably but not efficient.
Summary of the invention
(1) technical problems to be solved
The object of the present invention is to provide the method for allocating tasks and system in a kind of computer, realize the load of process
Equilibrium, and guarantee that task is efficiently executed.
(2) technical solution
The present invention provides the method for allocating tasks in a kind of computer, for the task in m task pool to be distributed to p
Process, m task pool are arranged successively, and each task pool has n storage unit, and method includes:
S1 arranges p process in order, obtains process queue；
Task in j-th of storage unit of i-th of task pool is distributed to k-th of process by S2, if's
Remainder is equal to 0, then k is equal to p, and otherwise k is equal toRemainder, m, n, p, i, j, k are the integer more than or equal to 1,
Wherein, 1≤i≤m, 1≤j≤n, 1≤k≤p.
The present invention also provides the task distribution systems in a kind of computer, for the task in m task pool to be distributed to p
A process, m task pool are arranged successively, and each task pool has n storage unit, and system includes:
Process sorting module arranges p process in order, obtains process queue；
Task in j-th of storage unit of i-th of task pool is distributed to k-th of process by task allocating module, ifRemainder be equal to 0, then k be equal to p, otherwise k is equal toRemainder, m, n, p, i, j, k are big
In the integer for being equal to 1, wherein 1≤i≤m, 1≤j≤n, 1≤k≤p., wherein 1≤i≤m, 1≤j≤n, 1≤k≤p.
(3) beneficial effect
The present invention regards the storage unit in multiple tasks pond as a two-dimensional matrix, then by the element in matrix by suitable
Sequence distributes to process, so that the relations of distribution are cheer and bright, easy to maintain and monitoring；In addition, by the method for salary distribution above,
So that each task pool possess process number, each process the task pool occupied can guarantee equilibrium, to realize process
Load balancing and guarantee task are efficiently executed.
Detailed description of the invention
Fig. 1 is the schematic diagram of the relations of distribution between task pool and process of the invention.
Fig. 2 is the schematic diagram that task distribution is carried out according to the relations of distribution of Fig. 1.
Fig. 3 is the flow chart of the method for allocating tasks in computer provided in an embodiment of the present invention.
Fig. 4 is the structural schematic diagram of the task distribution system in computer provided in an embodiment of the present invention.
Specific embodiment
The present invention provides method for allocating tasks and system in a kind of computer, for by the task in m task pool point
P process of dispensing, each task pool have n storage unit, and method first arranges p process in order, obtain into
Journey queue will according to putting in order for ranks then in the two-dimensional matrix of the total m × n storage unit composition of m task pool
Task distributes to process.The present invention is that the storage unit in multiple tasks pond is regarded as to a two-dimensional matrix, then will be in matrix
Element be sequentially allocated to process, so that the task pool of process number, each process that each task pool possesses occupied
It can guarantee equilibrium, to realize the load balancing of process and guarantee that task is efficiently executed.
A kind of embodiment according to the present invention carries out centralized buffer memory, that is, all task quilts to task
It is centrally stored in unified caching, but is not meant to that these tasks are all placed in a buffer queue, but according to Hash
Algorithm is dispersed in multiple " task pools ", and operation system, which can be used " task management interface " and increase newly or delete into task pool, appoints
Business.One task pool just represents a buffer queue, and the number of task pool is configured according to portfolio.Pass through dispersion in this way
System reliability can be improved in task pool, and being stored by the way of caching being capable of the storage of significant increase task, inquiry, deletion effect
Rate.The present invention regards m × n storage unit in m task pool as a two-dimensional matrix, and method for allocating tasks includes:
S1 arranges p process in order, obtains process queue；
S2 will according to putting in order for ranks in the two-dimensional matrix of the total m × n storage unit composition of m task pool
Task distributes to process, specifically: the task in j-th of storage unit of i-th of task pool is distributed into k-th of process,
In, wherein ifRemainder be equal to 0, then k be equal to p, otherwise k is equal toRemainder, m, n, p, i,
J, k is the integer more than or equal to 1, wherein 1≤i≤m, 1≤j≤n, 1≤k≤p.
The principle of the present invention is, if process number is P, task pool sum is B, and the process number of each task pool is A=P/B,
If A enables A=1, so that survival process can be evenly distributed in less than 1 (task pool sum is greater than survival process number)
Task pool as much as possible round up taking if A was greater than for 1 (task pool sum is less than in survival process number) to A
Whole, this is in order not to allow process progress is excessive to leave unused, to make process load balancing.
As shown in Figs. 1-2, b0-b4 represents 5 task pools, and each task pool has N+1 storage unit, i.e. s0-sN's
Storage location constitutes two-dimensional matrix in this way, for convenience of explanation, each element of matrix number is indicated, in task point
Timing, first distributes the element of the first row of the first row, the element ... of secondary series and so on of reallocation the first row is pressed
The numerical order of 0-14 is allocated each element in figure.As illustrated in fig. 2, it is assumed that having 5 processes and 5 task pools, need
The element that distribution number is 0-14 for convenience of explanation, can be analyzed to three phases in distribution, firstly, by element 0 in figure,
1,2,3,4 is respectively allocated to the 1st, 2,3,4,5 process, then, is respectively allocated to the 1st, 2,3 by element 5 in figure, 6,7,8,9,
4,5 processes, finally, the 1st, 2,3,4,5 process is respectively allocated to by 10,11,12,13,14, in this way, 5 processes obtain
The task that 5 task pools are distributed, also, the task quantity that each process obtains is equal, realize load balancing.
Such method of salary distribution guarantees that the process number and the occupied task pool of each process in each task pool can
Guarantee equilibrium, to realize the load balancing of process and guarantee that task is efficiently executed.
A kind of embodiment according to the present invention, upon step s 2, if a process in p process terminates,
The process is deleted in process queue, and enables p=p-1, then, executes step S1~S2；Alternatively, if there is new process initiation,
By the new process of the addition into the process queue, and p=p+1 is enabled, then, executes the step S1~S2.Here p=
P-1 is that the numerical value of p is instigated to subtract 1, and p=p+1 refers to that the numerical value of p adds 1, then when executing step S1~S2, can make the numerical value of p with
Actual process number is consistent.
A kind of embodiment according to the present invention, before step S1, each process in p process is on startup
The process distributes an identifier.When being ranked up, the identifier of each process in p process is obtained, according to identifier
Size arranges p process, obtains process queue, using first process in process queue as main thread, for holding
Row step S2.Identifier can be a bill, and it is the number of a numeric type that bill, which is the name of process, be increased monotonically, no
It can repeat, as shown in table 1.
| Bill | 1 | 2 | … | N |
| Process 0 | Process 1 | Process 2 | … |
Table 1
Each process can obtain a bill on startup, and it is as follows to obtain bill rule: successively searching since number 0
Whether corresponding bill occupied, until find " N " it is occupied when, then neck ticket success, the ticket of " N " namely process
According to.After progress " success of neck ticket ", it is also necessary to check that the bill is " master note ", that is, number the smallest bill, exist in this way
In process queue, it can be also arranged on first process, thus as host process.And host process is when executing step S2, it is real
It is to save the relations of distribution on border, the relations of distribution refer to the corresponding relationship of element and process in two-dimensional array as shown in Figure 1, main
Thread distributes to corresponding process according to the corresponding relationship, by the task in element.
A kind of embodiment according to the present invention, when each process initiation, can open a timer timing and go to attempt to tie up
All relations of distribution are protected, the purpose designed in this way is to guarantee in the case where original host process exception, and new bill is minimum
Process will become host process continue safeguard the relations of distribution.
A kind of embodiment according to the present invention, host process are not repeatedly to go carry out task distribution, but only exist
Loading unevenness is ability carry out task distribution.In original allocation task, for example survival process number is P, and task pool sum is B, after
Continue because newly-increased process, process exception cause survival number of processes to change (survival information of process is placed in unique caching), leads
Cause load when original allocation task no longer balanced, host process can just execute step S1~S2.
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with specific embodiment, and reference
Attached drawing, the present invention is described in more detail.
Fig. 3 is the method for allocating tasks in computer provided in an embodiment of the present invention, and method includes:
Each process in S1, p processes distributes an identifier on startup, for the process, obtains every in p process
The identifier of a process arranges p process according to the size of identifier, obtains process queue, and will be in process queue
First process as main thread, for executing step S2；
Task in j-th of storage unit of i-th of task pool is distributed to k-th of process by S2, if's
Remainder is equal to 0, then k is equal to p, and otherwise k is equal toRemainder, m, n, p, i, j, k are the integer more than or equal to 1,
Wherein, 1≤i≤m, 1≤j≤n, 1≤k≤p.If a process in p process terminates, deleting in process queue should be into
Journey, and p=p-1 is enabled, then, execute step S1~S2；Alternatively, if there is new process initiation, extremely by the new process of the addition
In process queue, and p=p+1 is enabled, then, executes step S1~S2.
Fig. 4 is the task distribution system in computer provided in an embodiment of the present invention, comprising:
Identifier distribution module on startup for each process in p process distributes a mark for the process
Symbol.
Process sorting module, for obtaining the identifier of each process in the p process, according to the big of the identifier
It is small, the p process is arranged, obtains process queue, and using first process in process queue as main thread.
Task allocating module carries out task distribution by executing the main thread, specifically includes: by i-th task pool
Task in j-th of storage unit distributes to k-th of process, wherein ifRemainder be equal to 0, then k be equal to p,
Otherwise k is equal toRemainder, m, n, p, i, j, k are the integer more than or equal to 1, wherein 1≤i≤m, 1≤j≤
N, 1≤k≤p.
Process manager module is used for after carrying out task distribution, if a process in the p process terminates,
The process is deleted in the process queue, and enables p=p-1；Alternatively, for after carrying out task distribution, if there is new process
Starting, then the new process is added into the process queue, and then enable p=p+1.
In conclusion having efficient, reliable special present invention can apply in large-scale timing task processing system
Property, many abnormal conditions such as system exception delay machine are coped with, accomplish that the runtime adaptively adjusts, while being distributed and being calculated by core
Can method guarantee that the timeliness and harmony of task processing, each distributed system resource for being also able to maintain execution task obtain
Rationally utilize.
Particular embodiments described above has carried out further in detail the purpose of the present invention, technical scheme and beneficial effects
It describes in detail bright, it should be understood that the above is only a specific embodiment of the present invention, is not intended to restrict the invention, it is all
Within the spirit and principles in the present invention, any modification, equivalent substitution, improvement and etc. done should be included in guarantor of the invention
Within the scope of shield.
Claims (12)
1. the method for allocating tasks in a kind of computer, for the task in m task pool to be distributed to p process, feature
It is, the m task pool is arranged successively, and each task pool has n storage unit, and method includes:
S1 arranges the p process in order, obtains process queue；
Task in j-th of storage unit of i-th of task pool is distributed to k-th of process by S2, wherein if
Remainder be equal to 0, then k be equal to p, otherwise k is equal toRemainder, traverse all tasks (i, j) until task distribute
It completes, m, n, p, i, j, k are the integer more than or equal to 1, wherein 1≤i≤m, 1≤j≤n, 1≤k≤p.
2. the method for allocating tasks in computer according to claim 1, which is characterized in that after the step S2,
If a process in the p process terminates, the process is deleted in process queue, and enable p=p-1, if p is not equal to 0,
Then execute the step S1~S2.
3. the method for allocating tasks in computer according to claim 1, which is characterized in that after the step S2,
If there is new process initiation, which is added into the process queue, and enable p=p+1, then, described in execution
Step S1~S2.
4. the method for allocating tasks in computer according to claim 1, which is characterized in that before the step S1,
Include:
Each process in the p process distributes an identifier on startup, for the process.
5. the method for allocating tasks in computer according to claim 4, which is characterized in that the step S1 includes:
The identifier for obtaining each process in the p process carries out the p process according to the size of the identifier
Arrangement, obtains process queue.
6. the method for allocating tasks in computer according to claim 1, which is characterized in that
In the step S1, using first process in the process queue as main thread, for executing the step S2.
7. the task distribution system in a kind of computer, for the task in m task pool to be distributed to p process, feature
It is, the m task pool is arranged successively, and each task pool has n storage unit, and system includes:
Process sorting module arranges the p process in order, obtains process queue；
Task in j-th of storage unit of i-th of task pool is distributed to k-th of process by task allocating module, wherein ifRemainder be equal to 0, then k be equal to p, otherwise k is equal toRemainder, it is straight to traverse all tasks (i, j)
It is assigned to task, m, n, p, i, j, k are the integer more than or equal to 1, wherein 1≤i≤m, 1≤j≤n, 1≤k≤p.
8. the task distribution system in computer according to claim 7, which is characterized in that further include:
Process manager module is used for after carrying out task distribution, if a process in the p process terminates, at this
The process is deleted in process queue, and enables p=p-1.
9. the task distribution system in computer according to claim 7, which is characterized in that further include:
Process manager module, for after carrying out task distribution, if there is new process initiation, by the new process be added to
In the process queue, and then enable p=p+1.
10. the task distribution system in computer according to claim 7, which is characterized in that further include:
Identifier distribution module on startup for each process in the p process distributes a mark for the process
Symbol.
11. the task distribution system in computer according to claim 10, which is characterized in that the process sorting module
The identifier for obtaining each process in the p process arranges the p process according to the size of the identifier,
Obtain process queue.
12. the task distribution system in computer according to claim 7, which is characterized in that
For the process sorting module using first process in the process queue as main thread, the task allocating module is logical
It crosses and executes the main thread to carry out task distribution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610009481.1A CN105607956B (en) | 2016-01-06 | 2016-01-06 | Method for allocating tasks and system in a kind of computer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610009481.1A CN105607956B (en) | 2016-01-06 | 2016-01-06 | Method for allocating tasks and system in a kind of computer |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105607956A CN105607956A (en) | 2016-05-25 |
| CN105607956B true CN105607956B (en) | 2019-06-04 |
Family
ID=55987910
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610009481.1A Active CN105607956B (en) | 2016-01-06 | 2016-01-06 | Method for allocating tasks and system in a kind of computer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105607956B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106201696A (en) * | 2016-07-15 | 2016-12-07 | 东软集团股份有限公司 | Method and apparatus for thread |
| CN106375372B (en) * | 2016-08-23 | 2019-12-06 | 东方网力科技股份有限公司 | big data resource allocation method and device |
| CN106803841B (en) * | 2017-02-14 | 2020-04-10 | 北京奇虎科技有限公司 | Method and device for reading message queue data and distributed data storage system |
| CN108848150A (en) * | 2018-06-06 | 2018-11-20 | 郑州云海信息技术有限公司 | A kind of plan target method and system in cloud management platform |
| CN110795222B (en) * | 2019-10-25 | 2022-03-22 | 北京浪潮数据技术有限公司 | Multithreading task scheduling method, device, equipment and readable medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102929707A (en) * | 2012-11-06 | 2013-02-13 | 无锡江南计算技术研究所 | Parallel task dynamical allocation method |
| CN103279390A (en) * | 2012-08-21 | 2013-09-04 | 中国科学院信息工程研究所 | Parallel processing system for small operation optimizing |
| CN104700255A (en) * | 2013-12-06 | 2015-06-10 | 腾讯科技（北京）有限公司 | Multi-process processing method, device and system |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050034130A1 (en) * | 2003-08-05 | 2005-02-10 | International Business Machines Corporation | Balancing workload of a grid computing environment |
| US8689226B2 (en) * | 2011-05-11 | 2014-04-01 | Hewlett-Packard Development Company, L.P. | Assigning resources to processing stages of a processing subsystem |
-
2016
- 2016-01-06 CN CN201610009481.1A patent/CN105607956B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103279390A (en) * | 2012-08-21 | 2013-09-04 | 中国科学院信息工程研究所 | Parallel processing system for small operation optimizing |
| CN102929707A (en) * | 2012-11-06 | 2013-02-13 | 无锡江南计算技术研究所 | Parallel task dynamical allocation method |
| CN104700255A (en) * | 2013-12-06 | 2015-06-10 | 腾讯科技（北京）有限公司 | Multi-process processing method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105607956A (en) | 2016-05-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105607956B (en) | Method for allocating tasks and system in a kind of computer | |
| US10628419B2 (en) | Many-core algorithms for in-memory column store databases | |
| Zhu et al. | Real-time tasks oriented energy-aware scheduling in virtualized clouds | |
| US8776066B2 (en) | Managing task execution on accelerators | |
| US8595732B2 (en) | Reducing the response time of flexible highly data parallel task by assigning task sets using dynamic combined longest processing time scheme | |
| CN106557367A (en) | For device, the method and apparatus of granular service quality are provided for computing resource | |
| Huang et al. | Yugong: Geo-distributed data and job placement at scale | |
| KR102338849B1 (en) | Method and system for providing stack memory management in real-time operating systems | |
| CN107168779A (en) | A kind of task management method and system | |
| CN106126403A (en) | Oracle database failure analysis methods and device | |
| CN102629220A (en) | Dynamic task allocation and management method | |
| US20160034528A1 (en) | Co-processor-based array-oriented database processing | |
| US10275289B2 (en) | Coexistence of message-passing-like algorithms and procedural coding | |
| CN107316124A (en) | Extensive affairs type job scheduling and processing general-purpose platform under big data environment | |
| CN101216780B (en) | Method and apparatus for accomplishing multi-instance and thread communication under SMP system | |
| US10740317B2 (en) | Using message-passing with procedural code in a database kernel | |
| CN105446812A (en) | Multitask scheduling configuration method | |
| WO2024037132A1 (en) | Workflow processing method and apparatus, and device, storage medium and program product | |
| Vo et al. | HyperFlow: A Heterogeneous Dataflow Architecture. | |
| CN113918305A (en) | Node scheduling method and device, electronic equipment and readable storage medium | |
| Babu et al. | Dynamic colocation algorithm for Hadoop | |
| Kumaresan et al. | Aegeus: An online partition skew mitigation algorithm for mapreduce | |
| Chen et al. | Fangorn: adaptive execution framework for heterogeneous workloads on shared clusters | |
| CN107479945A (en) | A kind of resources of virtual machine dispatching method and device | |
| CN107885834A (en) | A kind of Hadoop big datas component uniformly verifies system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |