CN109669775A - Distributed task dispatching method, system and storage medium - Google Patents
Distributed task dispatching method, system and storage medium Download PDFInfo
- Publication number
- CN109669775A CN109669775A CN201811503014.XA CN201811503014A CN109669775A CN 109669775 A CN109669775 A CN 109669775A CN 201811503014 A CN201811503014 A CN 201811503014A CN 109669775 A CN109669775 A CN 109669775A
- Authority
- CN
- China
- Prior art keywords
- task
- execution machine
- controller
- tasks
- allocated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000006870 function Effects 0.000 claims description 13
- 238000013507 mapping Methods 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 10
- 235000013399 edible fruits Nutrition 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004146 energy storage Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
Abstract
The present invention relates to load dispatch technology, a kind of distributed task dispatching method, system and storage medium are provided.This method comprises: statistics can use the quantity of execution machine；Being arranged each described with the executable task threshold of execution machine and can record each controller of distributed task dispatching system and described can distribute number of tasks with execution machine to each；Task to be allocated is obtained from any controller, the cryptographic Hash of the task to be allocated is calculated according to the attribute information of the task to be allocated；The the first remainder value for calculating the cryptographic Hash Yu the quantity with execution machine, from described with available execution machine corresponding with the first remainder value determining in execution machine；Number of tasks is distributed to the available execution machine of the determination and other available execution machines according to the controller of the task to be allocated is dispatched, selection executes the available execution machine of the task to be allocated.Using the present invention, is conducive to improve task execution efficiency, the dispatching efficiency of distributed task dispatching system can also be improved.
Description
Technical field
The present invention relates to computer task administrative skill field more particularly to a kind of distributed task dispatching methods, system
And computer readable storage medium.
Background technique
It is wherein stored according to scheduling information and dispatching algorithm wherein executes, task scheduling system can be divided into two
Class: distributed task dispatching system and distributed task dispatching system.In distributed task dispatching system, there are a conducts
The controller of hub scheduler, can receive the status information of all execution machines, and all tasks are adjusted by the hub scheduler is unified
Then degree is redistributed to each execution machine and executes up.And distributed task dispatching system has multiple controllers, these controls
Device completes scheduling strategy jointly.The great advantage of distributed task dispatching system is that have good scalability.
Currently, distributed task dispatching system usually utilizes the synchronous communication between multiple controllers, multiple execution machines are realized
Between load balancing, however, dispatching efficiency is unable to get guarantee due to the influence that cannot exclude the factors such as network delay.
Summary of the invention
The main purpose of the present invention is to provide a kind of distributed task dispatching method, system and computer-readable storage mediums
Matter, for improving the dispatching efficiency of distributed task dispatching system.
To achieve the above object, the present invention provides a kind of distributed task dispatching method, is applied to distributed task dispatching
System, this method comprises:
Statistic procedure: obtaining the available mode of execution machine, and statistics can use the quantity of execution machine；
Setting steps: setting is each described with the executable task threshold of the machine of execution and to record the distributed task scheduling tune
Each controller of degree system described can distribute number of tasks with execution machine to each；
Mapping step: task to be allocated is obtained from any controller, calculating according to the attribute information of the task to be allocated should
The cryptographic Hash of task to be allocated；
It calculates step: calculating the first remainder value of the cryptographic Hash Yu the quantity with execution machine, used from described
Available execution machine corresponding with the first remainder value is determined in execution machine；And
Allocation step: when the controller is to when the distributing number of tasks greater than 0 of available execution machine of the determination, by institute
State the available execution machine execution that task to be allocated distributes to the determination；When the controller is to the available execution machine of the determination
When distributing number of tasks no more than 0, the task to be allocated is distributed into other any controller distributing to it
Available execution machine of the number of tasks greater than 0 executes；When the controller to it is described can with execution machine to distribute number of tasks all little
When 0, the available execution machine of random selection one executes the task to be allocated.
Preferably, the attribute information of the task to be allocated includes task names and task's owner name's title.
Preferably, the mapping step includes:
Hash operation is carried out using title of first hash function to the task to be allocated, obtains the first operation result；
Hash operation is carried out using owner title of second hash function to the task to be allocated, obtains the second operation
As a result；
XOR operation is carried out to first operation result and second operation result, obtains third operation result, it will
Cryptographic Hash of the third operation result as the task to be allocated.
Preferably, first hash function and second hash function are one or both of MD5, SHA-1.
Preferably, each controller is recorded in the setting steps described can distribute number of tasks with execution machine to each
Process include the following steps:
The quantity of the executable task threshold and controller is divided by and truncation is rounded, obtains each controller to each
It is described to use the initial value for distributing number of tasks of execution machine；
An one-dimension array is defined on each controller, and the element of the one-dimension array is made to use execution machine one by one with described
It is corresponding, and the initial value of each element and each controller described can distribute the initial of number of tasks with execution machine to each
It is worth equal；
When obtain from some controller task to be allocated is distributed to some available execution machine execution, the controller
The number of tasks of distributing of execution machine can be used to subtract 1 this；
When the available execution machine executes the completion task, which can be used the number of tasks of distributing of execution machine to add 1 this.
Preferably, in the allocation step by the task to be allocated distribute to other any described controllers to it can
The process that available execution machine of the distribution number of tasks greater than 0 executes includes the following steps:
It obtains and is less than the quantity that can use execution machine and the natural number relatively prime with the quantity with execution machine, statistics
The quantity of the natural number of acquisition；
Calculate the second remainder value of the quantity of the cryptographic Hash of the task to be allocated and the natural number of the acquisition, choose with
Second remainder is worth the corresponding natural number as scanning step；
According to the scanning step successively judge the controller to each available execution machine whether distribute number of tasks big
In 0；
When distributing number of tasks greater than 0 of execution machine can be used to some in the controller, by the task to be allocated point
The dispensing available execution machine executes, and terminates the allocation step.
The present invention also provides a kind of distributed task dispatching system, which includes memory, place
Manage device and several execution machines, which is characterized in that include task dispatch in the memory, the task dispatch is described
Processor realizes following steps when executing:
Statistic procedure: obtaining the available mode of execution machine, and statistics can use the quantity of execution machine；
Setting steps: setting is each described with the executable task threshold of the machine of execution and to record the distributed task scheduling tune
Each controller of degree system described can distribute number of tasks with execution machine to each；
Mapping step: task to be allocated is obtained from any controller, calculating according to the attribute information of the task to be allocated should
The cryptographic Hash of task to be allocated；
It calculates step: calculating the first remainder value of the cryptographic Hash Yu the quantity with execution machine, used from described
Available execution machine corresponding with the first remainder value is determined in execution machine；And
Allocation step: when the controller is to when the distributing number of tasks greater than 0 of available execution machine of the determination, by institute
State the available execution machine execution that task to be allocated distributes to the determination；When the controller is to the available execution machine of the determination
When distributing number of tasks no more than 0, the task to be allocated is distributed into other any controller distributing to it
Available execution machine of the number of tasks greater than 0 executes；When the controller to it is described can with execution machine to distribute number of tasks all little
When 0, the available execution machine of random selection one executes the task to be allocated.
The present invention also provides a kind of computer readable storage medium, task tune is stored in the computer readable storage medium
Program is spent, which is performed the arbitrary steps realized in distributed task dispatching method as described above.
Distributed task dispatching method, system and computer readable storage medium provided by the invention are executed by obtaining
The available mode of machine, statistics can use the quantity of execution machine, and then the executable task threshold of each available execution machine of setting, passes through
The mode for defining one-dimension array on the controller records each controller of distributed task dispatching system and described uses to each
Execution machine distributes number of tasks.After obtaining task to be allocated from any controller, according to the attribute of the task to be allocated
Information calculates the cryptographic Hash of the task to be allocated, followed by described in the cryptographic Hash and the quantity calculating with execution machine
The available execution machine of task to be allocated, finally according to dispatch the controller of the task to be allocated to it is described be calculated available hold
Row machine and other available execution machines distribute number of tasks, and selection executes the available execution machine of the task to be allocated.In this hair
In bright, using the cryptographic Hash and it is described can with execute machine quantity calculate the task to be allocated available execution machine be in order to
The identical same generic task of attribute information is preferentially distributed to identical available execution machine to execute, to realize resource multiplex, improves and appoints
Business execution efficiency.In addition, by define one-dimension array record each controller to it is each it is described can with execution machine distribute appoint
Business number is also able to achieve load balancing so that need not synchronize communication between multiple controllers, improves dispatching efficiency.Therefore, sharp
With the present invention, is conducive to improve task execution efficiency, the dispatching efficiency of distributed task dispatching system can also be improved.
Detailed description of the invention
Fig. 1 is the schematic diagram of one embodiment of distributed task dispatching system of the present invention；
Fig. 2 is the Program modual graph of one embodiment of task dispatch in Fig. 1；
Fig. 3 is the flow diagram of one embodiment of distributed task dispatching method of the present invention.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
It should be understood that the specific embodiments described herein are merely illustrative of the present invention, it is not used to limit this
Invention.Based on the embodiments of the present invention, those of ordinary skill in the art are obtained without making creative work
The every other embodiment obtained, shall fall within the protection scope of the present invention.
The present invention provides a kind of distributed task dispatching system.It is distributed task dispatching system of the present invention shown in referring to Fig.1
The schematic diagram for 1 one embodiments of uniting.
The distributed task dispatching system 1 includes memory 11, processor 12, network interface 13 and executes machine cluster
14。
Wherein, the memory 11 includes the readable storage medium storing program for executing of at least one type.At least one type can
Read the non-volatile memory medium that storage medium can be such as flash memory, hard disk, multimedia card, card-type memory.In some implementations
In example, the readable storage medium storing program for executing can be the internal storage unit of the distributed task dispatching system 1, such as the distribution
Host hard drive in task scheduling system 1.In further embodiments, the readable storage medium storing program for executing is also possible to the distribution
The plug-in type hard disk being equipped on the external memory 11 of task scheduling system 1, such as the distributed task dispatching system 1, intelligence
Energy storage card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, flash card (Flash
Card) etc..
In the present embodiment, the readable storage medium storing program for executing of the memory 11 is commonly used in store tasks scheduler program 10.Institute
Stating memory 11 can be also used for temporarily storing the data that has exported or will export.
The processor 12 includes multiple controllers, and multiple controller receives task to be allocated at random and adjusts to it
Degree, each execution machine assigned it in the execution machine cluster 14 execute up.It should be noted that different controllers
It can be deployed in same processor, can also be deployed in different processor (not shown)s.In some embodiments
In, the processor 12 can be central processing unit (Central Processing Unit, CPU), microprocessor or other numbers
According to processing chip, program code for being stored in run memory 11, such as execute the task dispatch 10.
The network interface 13 may include standard wireline interface and wireless interface (such as WI-FI interface).Commonly used in
Communication connection is established between the distributed task dispatching system 1 and other systems or electronic equipment.
The execution machine cluster 14 includes several execution machines.The execution machine can be the various electronics with calculation function
Equipment, including but not limited to server, smart phone, phone wrist-watch, tablet computer, portable computer, desktop PC,
Game station, E-book reader, MP3 player (Moving Picture Experts Group Audio Layer
III), MP4 player (Moving Picture Experts Group Audio Layer IV), etc..In one embodiment
In, when the execution machine is server, which can be rack-mount server, blade server, tower server
Or the one or more of Cabinet-type server etc..In another embodiment, different execution machines can also be deployed in same
In server.
In the present invention, it is assumed for convenience of description that the execution machine executed in a group of planes 14 is same type of execution
Machine, configuration having the same and processing capacity.
Fig. 1 illustrates only the distributed task dispatching system 1 with component 11-14 and task dispatch 10, but answers
What is understood is, it is not required that implements all components shown, the implementation that can be substituted is more or less component.
Optionally, which can also include input unit such as keyboard (Keyboard), voice
Input unit such as microphone (Microphone) etc. has the equipment and instantaneous speech power such as sound of speech identifying function
Sound, earphone etc..User can be configured and adjust to the task dispatch 10 using these devices and equipment.
Optionally, which can also include display, be referred to as display screen or display is single
Member.It can be light-emitting diode display, liquid crystal display, touch-control liquid crystal display and organic light-emitting diodes in some embodiments
Manage (Organic Light-Emitting Diode, OLED) display etc..Display is for being shown in distributed task dispatching
The information that is handled in system 1 and for showing visual user interface.
The distributed task dispatching system 1 can also include radio frequency (Radio Frequency, RF) circuit, sensor, sound
Frequency circuit, wireline interface and wireless interface etc., details are not described herein.
In the above-described embodiments, processor 12 execute memory 11 in store task dispatch 10 when may be implemented as
Lower step:
Statistic procedure: obtaining the available mode of execution machine, and statistics can use the quantity of execution machine；
Setting steps: setting is each described with the executable task threshold of the machine of execution and to record the distributed task scheduling tune
Each controller of degree system described can distribute number of tasks with execution machine to each；
Mapping step: task to be allocated is obtained from any controller, calculating according to the attribute information of the task to be allocated should
The cryptographic Hash of task to be allocated；
It calculates step: calculating the first remainder value of the cryptographic Hash Yu the quantity with execution machine, used from described
Available execution machine corresponding with the first remainder value is determined in execution machine；And
Allocation step: when the controller is to when the distributing number of tasks greater than 0 of available execution machine of the determination, by institute
State the available execution machine execution that task to be allocated distributes to the determination；When the controller is to the available execution machine of the determination
When distributing number of tasks no more than 0, the task to be allocated is distributed into other any controller distributing to it
Available execution machine of the number of tasks greater than 0 executes；When the controller to it is described can with execution machine to distribute number of tasks all little
When 0, the available execution machine of random selection one executes the task to be allocated.
About being discussed in detail for above-mentioned steps, program mould of following Fig. 2 about 10 embodiment of task dispatch is please referred to
The explanation of block figure and Fig. 3 about the flow chart of distributed task dispatching embodiment of the method.
In other embodiments, the task dispatch 10 can be divided into multiple modules, and multiple module is deposited
It is stored in memory 12, and is executed by processor 13, to complete the present invention.The so-called module of the present invention be refer to complete it is specific
The series of computation machine program instruction section of function.
It is the Program modual graph of 10 1 embodiment of task dispatch in Fig. 1 referring to shown in Fig. 2.In the present embodiment, institute
Stating task dispatch 10 can be divided into: statistical module 110, setup module 120, mapping block 130, computing module 140
And distribution module 150.
The statistical module 110, for obtain execution machine can with state, calculate can use execution machine quantity.Specifically,
Statistical module 110 can send the broadcasting packet for determining node device available mode to each execution machine, and according to reception
To response message calculate in the execution machine cluster 14 and can use the quantity of execution machine.Statistical module 110 can be also used for institute
It states available execution machine to be numbered, execution machine number can uniquely can be used with the distribution of execution machine one to be each described, for example, from
0 starts to be numbered with execution machine with incrementing integers to be each described: 0,1,2 ...
The setup module 120, for each executable task threshold that can use execution machine to be arranged, and described in record
Each controller of distributed task dispatching system described can distribute number of tasks with execution machine to each.Specifically, user
Can by setup module 120 according to it is described with the settings such as configuration, the bearing capacity of execution machine it is each described can with execution machine can
Task threshold is executed, then by the number for the controller that task threshold can be performed in this and the distributed task dispatching system 1 includes
Amount is divided by and truncation is rounded, and obtaining each controller described can distribute number of tasks with execution machine to each.For example, it is assumed that institute
The executable task threshold for stating available execution machine is all 10, and there are the distributed task dispatching system 13 to be dispatched for completing
Strategy controller, then each controller to it is each it is described can with execution machine distribute number of tasks be 3.It is understood that
Be, controller to available execution machine distribute number of tasks be variation, when one will obtained from some controller it is to be allocated
When task distributes to some available execution machine execution, which can be used the number of tasks of distributing of execution machine to subtract 1 this, when this can
When executing the completion task with the machine of execution, which can be used the number of tasks of distributing of execution machine to add 1 this.In the present embodiment
In, by defining an one-dimension array on each controller, the element of the one-dimension array is made to use execution machine one by one with described
It is corresponding, and the value of each element and each controller described can use the value for distributing number of tasks of execution machine phase always to each
Deng, that is, when obtain from some controller task to be allocated is distributed to some available execution machine execution, corresponding element
Value subtracts 1, and when this, which can be used execution machine to execute, completes the task, corresponding element value adds 1.
The mapping block 130, for obtaining task to be allocated from any controller, and according to the category of the task to be allocated
Property information calculates the cryptographic Hash of the task to be allocated.Wherein, the attribute information of the task to be allocated includes task names and appoints
Business owner's title.In one embodiment, MD5 operation can be carried out to the title of the task to be allocated, obtains the first fortune
It calculates as a result, the second operation result is obtained, finally by institute again to owner's title progress SHA-1 operation of the task to be allocated
Latter 128 and (128) progress XOR operation of first operation result for stating the second operation result (160), obtain third
Operation result, using the third operation result as the cryptographic Hash of the task to be allocated.It is understood that usual with generic task
Task names having the same, task's owner name is also identical, therefore, will with generic task according to above-mentioned Hash value calculating method
Obtain identical cryptographic Hash.
The computing module 140, for calculating the first remainder value of the cryptographic Hash Yu the quantity with execution machine,
From described with available execution machine corresponding with the first remainder value determining in execution machine.For example, execution machine can be used when described
When number is followed successively by 0,1,2 ... i, if the cryptographic Hash of some task to be allocated and the quantity (i+1) with execution machine
When remainder value is 0, then the available execution machine that number is 0 is available execution machine corresponding with the first remainder value.Because same
The cryptographic Hash of generic task is identical, so, for same generic task, the available execution machine of the determination is identical.The computing module 130
Effect be, pay the utmost attention to distribute to same generic task identical available execution machine and execute, to realize resource multiplex, improve and appoint
Business execution efficiency.
The distribution module 150, for being held according to the controller for dispatching the task to be allocated to the available of the determination
Row machine and other available execution machines distribute number of tasks, and selection executes the available execution machine of the task to be allocated.Specifically
Ground, when the controller is to when the distributing number of tasks greater than 0 of available execution machine of the determination, by the task to be allocated point
The available execution machine determined described in dispensing executes；When the controller distributes number of tasks to the available execution machine of the determination
When no more than 0, by the task to be allocated distribute to other any described controllers to its distribute number of tasks greater than 0 can
It is executed with the machine of execution；When the controller to it is described can be with execution machine when distributing number of tasks and being all not more than 0, random selection one
A available execution machine executes the task to be allocated.Wherein, the task to be allocated is distributed into other any described controllers
The number of tasks process that the available execution machine greater than 0 executes of distributing to it includes following operation:
It obtains and is less than the quantity that can use execution machine and the natural number relatively prime with the quantity with execution machine, statistics
The quantity of the natural number of acquisition；
Calculate the second remainder value of the quantity of the cryptographic Hash of the task to be allocated and the natural number of the acquisition, choose with
Second remainder is worth the corresponding natural number as scanning step；
According to the scanning step successively judge the controller to each available execution machine whether distribute number of tasks big
In 0；
When distributing number of tasks greater than 0 of execution machine can be used to some in the controller, by the task to be allocated point
The dispensing available execution machine executes, and terminates batch operation.
For example, it is assumed that the quantity with execution machine is 10, number 0-9, the then quantity of the natural number of the acquisition
It is 4, is 1,3,7,9 respectively.Task is distributed to it finding other any described controllers for a task to be allocated
When number is greater than 0 available execution machine, scanning step may be a number in 1,3,7,9.According to above-mentioned example, it is assumed that the calculating
The number of the available execution machine for the determination that module 140 is calculated is 2, if scanning step is 1, when the controller pair
When distributing number of tasks no more than 0 of the available execution machine of the determination, distribution module 150 will successively judge the controller pair
Number is 3,4,5 ... 9 available execution machine distributes whether number of tasks is greater than 0, until the controller to it is above-mentioned some
The task to be allocated can be distributed into the available execution machine and executed, terminate to divide with execution machine when distributing number of tasks greater than 0
With operation；If scanning step is 9, when distribute number of tasks of the controller to the available execution machine of the determination is not more than
When 0, distribution module 150 will successively judge the controller be 1 to number, 0,9,8 ... 3 available execution machine distributes
Whether number of tasks is greater than 0, until the controller is to some above-mentioned available execution machine when distributing number of tasks greater than 0, by institute
It states task to be allocated and distributes to available execution machine execution, terminate batch operation.
It should be noted that scanning step is not defined as 1 by the present invention, but met the requirements by way of calculating
Scanning step of the determining number as some task to be allocated is selected in limited natural number (including 1), is to reduce task
Distribute uneven possibility.For example, the available execution machine for the determination that computing module 140 is calculated can for multiclass task
Can be identical, when the available execution machine of the determination is identical, if the scanning step is also identical, task assignment procedure will occur
This multiclass task is gathered in the situation on individual available execution machines, is unable to reach the purpose of load balancing.And pass through above-mentioned calculating
Mode calculate multiclass task scanning step it is not necessarily equal, can reduce task aggregation possibility, be more advantageous to and reach
The purpose of load balancing.
In addition, the present invention also provides a kind of distributed task dispatching methods.Referring to shown in Fig. 3, appoint for the present invention is distributed
The flow diagram of the embodiment of business dispatching method.The processor 12 of distributed task dispatching system 1 executes to be deposited in memory 11
The following steps of distributed task dispatching method are realized when the task dispatch 10 of storage:
Step S300 obtains the available mode of execution machine using the statistical module 110, calculates the number that can use execution machine
Amount.In the present embodiment, by sending broadcasting packet and reception for determining node device available mode to each execution machine
Response message come obtain in the distributed task dispatching system 1 execution machine available mode, and then count can use execution machine
Quantity.Then it can also can be numbered with execution machine to described, execution machine distribution one can be used unique to be each described
It can be numbered with execution machine, for example, can be numbered with execution machine with incrementing integers to be each described since 0: 0,1,2 ... i.
Each executable task threshold that can use execution machine is arranged using the setup module 120 in step S301, and
Recording each controller of the distributed task dispatching system described can distribute number of tasks with execution machine to each.At this
In embodiment, it can be estimated with the configuration information of the machine of execution according to described and each described can be appointed with the executable of execution machine is set
The quantity that the controller that task threshold includes with the distributed task dispatching system 1 can be performed, is then divided by by business threshold value
And truncation is rounded, obtaining each controller described can use the initial value for distributing number of tasks of execution machine to each.Then every
An one-dimension array is defined on a controller, makes the element and the execution machine one-to-one correspondence of the one-dimension array, and is made each
The initial value of element and the above-mentioned each controller being calculated to it is each it is described can with execution machine distribute number of tasks just
Initial value is equal.It is corresponding when obtain from some controller task to be allocated is distributed to some available execution machine execution
Element value subtracts 1, and when this, which can be used execution machine to execute, completes the task, corresponding element value adds 1.
Step S302 obtains task to be allocated from any controller using the mapping block 130, and to be allocated according to this
The attribute information of task calculates the cryptographic Hash of the task to be allocated.In the present embodiment, the attribute information of the task to be allocated
Including task names and task's owner name's title.MD5 operation first is carried out to the title of the task to be allocated, obtains the first operation
As a result, carrying out SHA-1 operation to owner's title of the task to be allocated again, the second operation result is obtained, it finally will be described
Latter 128 of second operation result (160) and (128) progress XOR operation of first operation result obtain third fortune
It calculates as a result, using the third operation result as the cryptographic Hash of the task to be allocated.Because usually having the same with generic task
Task names, task's owner name is also identical, so with generic task by cryptographic Hash having the same.
Step S303 calculates the of the cryptographic Hash and the quantity with execution machine using the computing module 140
One remainder value, from described with available execution machine corresponding with the first remainder value determining in execution machine.Because with generic task
Cryptographic Hash is identical, so, for same generic task, the available execution machine of the determination is identical.It pays the utmost attention to distribute with generic task
It is executed to identical available execution machine, it can be achieved that resource multiplex, improves task execution efficiency.
Step S304, using the distribution module 150 according to the controller of the scheduling task to be allocated to the determination
Available execution machine and other available execution machines distribute number of tasks, selection executes the available execution of the task to be allocated
Machine.Specifically, when the controller is to when the distributing number of tasks greater than 0 of available execution machine of the determination, by described wait divide
The available execution machine for distributing to the determination with task executes；When the controller dividing to the available execution machine of the determination
When being not more than 0 with number of tasks, the task to be allocated is distributed into other any described controllers and distributes number of tasks to it
Available execution machine greater than 0 executes；When the controller to it is described can with execution machine when distributing number of tasks and being all not more than 0,
The available execution machine of random selection one executes the task to be allocated.
In the present embodiment, the task to be allocated is distributed into other any described controllers and distributes task to it
The process that available execution machine of the number greater than 0 executes includes following operation:
Calculate scanning step:
Step=
CoprimeList (NumInvokers) .get (hash%size (coprimerList (NumInvokers)))；
Calculate the number of other any available execution machines:
Invoker number=(homeInvoker+m*step) %NumInvokers；
Judge that the controller distributes whether number of tasks is greater than 0 to currently available execution machine；
If the controller distributes number of tasks greater than 0, by the task to be allocated point to currently available execution machine
The dispensing currently available execution machine executes, end step S304；
If the controller distributes number of tasks no more than 0 to currently available execution machine, m=m+1 is enabled, is calculated another
The number that execution machine can be used, judges that the controller distributes whether number of tasks is greater than 0 to another available execution machine.
Wherein, NumInvokers indicates that the quantity of execution machine can be used, and coprimeList (NumInvokers) indicates to obtain
Less than NumInvokers value and the natural number list relatively prime with NumInvokers value, size (coprimeList
(NumInvokers)) number of natural number in the natural number list, get (hash%size (coprimeList are indicated
(NumInvokers)) it) indicates first with the cryptographic Hash of the task to be allocated to the number of natural number in the natural number list
Remainder calculating is carried out, numerical value corresponding with gained remainder value is then obtained from the natural number list, homeInvoker indicates institute
The number of determining available execution machine is stated, variable m indicates the integer value from 1 to NumInvokers-1.
As m=NumInvokers-1, will complete that described execution machine distributing for the controller can be used to all
Number of tasks with 0 relative size judgement.If the controller can be all not more than to described with the number of tasks of distributing of execution machine
0, then it randomly chooses an available execution machine and executes the task to be allocated.
In addition, the computer readable storage medium can be firmly the present invention also provides a kind of computer readable storage medium
Disk, multimedia card, SD card, flash card, SMC, read-only memory (ROM), Erasable Programmable Read Only Memory EPROM (EPROM), just
Take any one in formula aacompactadisk read onlyamemory (CD-ROM), USB storage etc. or several any combination.It is described
It include task dispatch 10 in computer readable storage medium, which is performed the function of realization, asks
Referring to the above-mentioned introduction about Fig. 2, Fig. 3, details are not described herein.
It should be noted that, in this document, the terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that the process, device, article or the method that include a series of elements not only include those elements, and
And further include other elements that are not explicitly listed, or further include for this process, device, article or method institute it is intrinsic
Element.It in addition, the technical solution between each embodiment can be combined with each other, but must be with ordinary skill people
Based on member can be realized, this technical solution will be understood that when the combination of technical solution appearance is conflicting or cannot achieve
Combination be not present, also not the present invention claims protection scope within.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art
The part contributed out can be embodied in the form of software products, which is stored in one as described above
In storage medium, used including some instructions so that distributed task dispatching system executes side described in each embodiment of the present invention
Method.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of distributed task dispatching method is applied to distributed task dispatching system, which is characterized in that this method comprises:
Statistic procedure: obtaining the available mode of execution machine, and statistics can use the quantity of execution machine；
Setting steps: setting is each described with the executable task threshold of the machine of execution and to record the distributed task dispatching system
Each controller of system described can distribute number of tasks with execution machine to each；
Mapping step: obtaining task to be allocated from any controller, and being calculated according to the attribute information of the task to be allocated should be wait divide
Cryptographic Hash with task；
It calculates step: calculating the first remainder value of the cryptographic Hash Yu the quantity with execution machine, from described with execution
Available execution machine corresponding with the first remainder value is determined in machine；And
Allocation step: when the controller is to the available execution machine of the determination when distributing number of tasks greater than 0, will it is described to
The available execution machine that distribution task distributes to the determination executes；When the controller to the available execution machine of the determination can
When distributing number of tasks no more than 0, the task to be allocated is distributed into other any described controllers and distributes task to it
Available execution machine of the number greater than 0 executes；When the controller can all be not more than 0 with the number of tasks of distributing of execution machine to described
When, the available execution machine of random selection one executes the task to be allocated.
2. distributed task dispatching method as described in claim 1, which is characterized in that the attribute information of the task to be allocated
Including task names and task's owner name's title.
3. distributed task dispatching method as claimed in claim 2, which is characterized in that the mapping step includes:
Hash operation is carried out using title of first hash function to the task to be allocated, obtains the first operation result；
Hash operation is carried out using owner title of second hash function to the task to be allocated, obtains the second operation knot
Fruit；
XOR operation is carried out to first operation result and second operation result, obtains third operation result, by this
Cryptographic Hash of three operation results as the task to be allocated.
4. distributed task dispatching method as claimed in claim 3, which is characterized in that first hash function and described
Two hash functions are one or both of MD5, SHA-1.
5. the distributed task dispatching method as described in any one of Claims 1-4, which is characterized in that the setting step
Each controller is recorded in rapid described can be included the following steps: to each with the process for distributing number of tasks of the machine of execution
The quantity of the executable task threshold and controller is divided by and truncation is rounded, obtains each controller to each described
The initial value for distributing number of tasks of execution machine can be used；
An one-dimension array is defined on each controller, makes the element of the one-dimension array with described with one a pair of execution machine
It answers, and the initial value of each element and each controller described can use the initial value for distributing number of tasks of execution machine to each
It is equal；
When obtain from some controller task to be allocated is distributed to some available execution machine execution, the controller is to this
It can subtract 1 with the number of tasks of distributing of the machine of execution；
When the available execution machine executes the completion task, which can be used the number of tasks of distributing of execution machine to add 1 this.
6. distributed task dispatching method as described in claim 1, which is characterized in that will be described wait divide in the allocation step
Other any described controllers, which are distributed to, with task distributes the number of tasks process packet that the available execution machine greater than 0 executes to it
Include following steps:
It obtains and is less than the quantity that can use execution machine and the natural number relatively prime with the quantity with execution machine, count and obtain
Natural number quantity；
Calculate the second remainder value of the quantity of the cryptographic Hash of the task to be allocated and the natural number of the acquisition, choose with this
Two remainders are worth the corresponding natural number as scanning step；
Successively judge that the controller distributes whether number of tasks is greater than 0 to each available execution machine according to the scanning step；
When distributing number of tasks greater than 0 of execution machine can be used to some in the controller, the task to be allocated is distributed to
The available execution machine executes, and terminates the allocation step.
7. a kind of distributed task dispatching system, including memory, processor and several execution machines, which is characterized in that described to deposit
Include task dispatch in reservoir, which realizes following steps when being executed by the processor:
Statistic procedure: obtaining the available mode of execution machine, and statistics can use the quantity of execution machine；
Setting steps: setting is each described with the executable task threshold of the machine of execution and to record the distributed task dispatching system
Each controller of system described can distribute number of tasks with execution machine to each；
Mapping step: obtaining task to be allocated from any controller, and being calculated according to the attribute information of the task to be allocated should be wait divide
Cryptographic Hash with task；
It calculates step: calculating the first remainder value of the cryptographic Hash Yu the quantity with execution machine, from described with execution
Available execution machine corresponding with the first remainder value is determined in machine；And
Allocation step: when the controller is to the available execution machine of the determination when distributing number of tasks greater than 0, will it is described to
The available execution machine that distribution task distributes to the determination executes；When the controller to the available execution machine of the determination can
When distributing number of tasks no more than 0, the task to be allocated is distributed into other any described controllers and distributes task to it
Available execution machine of the number greater than 0 executes；When the controller can all be not more than 0 with the number of tasks of distributing of execution machine to described
When, the available execution machine of random selection one executes the task to be allocated.
8. distributed task dispatching system as claimed in claim 7, which is characterized in that record each control in the setting steps
Device processed to it is each it is described can with execute machine the process for distributing number of tasks include the following steps:
The quantity of the executable task threshold and controller is divided by and truncation is rounded, obtains each controller to each described
The initial value for distributing number of tasks of execution machine can be used；
An one-dimension array is defined on each controller, makes the element of the one-dimension array with described with one a pair of execution machine
It answers, and the initial value of each element and each controller described can use the initial value for distributing number of tasks of execution machine to each
It is equal；
When obtain from some controller task to be allocated is distributed to some available execution machine execution, the controller is to this
It can subtract 1 with the number of tasks of distributing of the machine of execution；
When the available execution machine executes the completion task, which can be used the number of tasks of distributing of execution machine to add 1 this.
9. distributed task dispatching system as claimed in claim 7, which is characterized in that will be described wait divide in the allocation step
Other any described controllers, which are distributed to, with task distributes the number of tasks process packet that the available execution machine greater than 0 executes to it
Include following steps:
It obtains and is less than the quantity that can use execution machine and the natural number relatively prime with the quantity with execution machine, count and obtain
Natural number quantity；
Calculate the second remainder value of the quantity of the cryptographic Hash of the task to be allocated and the natural number of the acquisition, choose with this
Two remainders are worth the corresponding natural number as scanning step；
Successively judge that the controller distributes whether number of tasks is greater than 0 to each available execution machine according to the scanning step；
When distributing number of tasks greater than 0 of execution machine can be used to some in the controller, the task to be allocated is distributed to
The available execution machine executes, and terminates the allocation step.
10. a kind of computer readable storage medium, which is characterized in that include task schedule in the computer readable storage medium
Program, the task dispatch are performed the distributed task dispatching side realized as described in any one of claim 1 to 6
The step of method.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811503014.XA CN109669775A (en) | 2018-12-10 | 2018-12-10 | Distributed task dispatching method, system and storage medium |
| PCT/CN2019/088631 WO2020119029A1 (en) | 2018-12-10 | 2019-05-27 | Distributed task scheduling method and system, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811503014.XA CN109669775A (en) | 2018-12-10 | 2018-12-10 | Distributed task dispatching method, system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109669775A true CN109669775A (en) | 2019-04-23 |
Family
ID=66143670
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811503014.XA Pending CN109669775A (en) | 2018-12-10 | 2018-12-10 | Distributed task dispatching method, system and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109669775A (en) |
| WO (1) | WO2020119029A1 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110659117A (en) * | 2019-08-23 | 2020-01-07 | 阿里巴巴集团控股有限公司 | Service index task scheduling and executing method, device, system and storage medium |
| CN110716796A (en) * | 2019-09-02 | 2020-01-21 | 中国平安财产保险股份有限公司 | Intelligent task scheduling method and device, storage medium and electronic equipment |
| CN111190710A (en) * | 2019-11-25 | 2020-05-22 | 泰康保险集团股份有限公司 | Task allocation method and device |
| WO2020119029A1 (en) * | 2018-12-10 | 2020-06-18 | 平安科技（深圳）有限公司 | Distributed task scheduling method and system, and storage medium |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114330735A (en) * | 2020-09-30 | 2022-04-12 | 伊姆西Ip控股有限责任公司 | Method, electronic device and computer program product for processing machine learning model |
| CN112463336A (en) * | 2020-12-07 | 2021-03-09 | 深圳集智数字科技有限公司 | Object allocation method, device, equipment and medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622273A (en) * | 2012-02-23 | 2012-08-01 | 中国人民解放军国防科学技术大学 | Self-learning load prediction based cluster on-demand starting method |
| CN102882973A (en) * | 2012-10-11 | 2013-01-16 | 北京邮电大学 | Distributed load balancing system and distributed load balancing method based on peer to peer (P2P) technology |
| CN103150213A (en) * | 2011-12-06 | 2013-06-12 | 北大方正集团有限公司 | Method and device for balancing load |
| CN105786600A (en) * | 2016-02-02 | 2016-07-20 | 北京京东尚科信息技术有限公司 | Task scheduling method and device |
| CN106130920A (en) * | 2016-07-14 | 2016-11-16 | 腾讯科技（深圳）有限公司 | A kind of message forwarding method and device |
| CN107087031A (en) * | 2017-05-10 | 2017-08-22 | 浙江宇视科技有限公司 | A kind of storage resource load-balancing method and device |
| CN107688496A (en) * | 2017-07-24 | 2018-02-13 | 上海壹账通金融科技有限公司 | Task distribution formula processing method, device, storage medium and server |
| CN108268318A (en) * | 2016-12-30 | 2018-07-10 | 华为技术有限公司 | A kind of method and apparatus of distributed system task distribution |
| CN108696581A (en) * | 2018-05-07 | 2018-10-23 | 上海智臻智能网络科技股份有限公司 | Caching method, device, computer equipment and the storage medium of distributed information |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103425519B (en) * | 2012-05-16 | 2016-10-05 | 富士通株式会社 | Distributed computing method and distributed computing system |
| CN105335219A (en) * | 2014-07-08 | 2016-02-17 | 阿里巴巴集团控股有限公司 | Distribution-based task scheduling method and system |
| CN106126332A (en) * | 2016-06-27 | 2016-11-16 | 北京京东尚科信息技术有限公司 | Distributed timing task scheduling system and method |
| CN108762931A (en) * | 2018-05-31 | 2018-11-06 | 康键信息技术（深圳）有限公司 | Method for scheduling task, server based on distributed scheduling system and storage medium |
| CN109669775A (en) * | 2018-12-10 | 2019-04-23 | 平安科技（深圳）有限公司 | Distributed task dispatching method, system and storage medium |
-
2018
- 2018-12-10 CN CN201811503014.XA patent/CN109669775A/en active Pending
-
2019
- 2019-05-27 WO PCT/CN2019/088631 patent/WO2020119029A1/en active Application Filing
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103150213A (en) * | 2011-12-06 | 2013-06-12 | 北大方正集团有限公司 | Method and device for balancing load |
| CN102622273A (en) * | 2012-02-23 | 2012-08-01 | 中国人民解放军国防科学技术大学 | Self-learning load prediction based cluster on-demand starting method |
| CN102882973A (en) * | 2012-10-11 | 2013-01-16 | 北京邮电大学 | Distributed load balancing system and distributed load balancing method based on peer to peer (P2P) technology |
| CN105786600A (en) * | 2016-02-02 | 2016-07-20 | 北京京东尚科信息技术有限公司 | Task scheduling method and device |
| CN106130920A (en) * | 2016-07-14 | 2016-11-16 | 腾讯科技（深圳）有限公司 | A kind of message forwarding method and device |
| CN108268318A (en) * | 2016-12-30 | 2018-07-10 | 华为技术有限公司 | A kind of method and apparatus of distributed system task distribution |
| CN107087031A (en) * | 2017-05-10 | 2017-08-22 | 浙江宇视科技有限公司 | A kind of storage resource load-balancing method and device |
| CN107688496A (en) * | 2017-07-24 | 2018-02-13 | 上海壹账通金融科技有限公司 | Task distribution formula processing method, device, storage medium and server |
| CN108696581A (en) * | 2018-05-07 | 2018-10-23 | 上海智臻智能网络科技股份有限公司 | Caching method, device, computer equipment and the storage medium of distributed information |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020119029A1 (en) * | 2018-12-10 | 2020-06-18 | 平安科技（深圳）有限公司 | Distributed task scheduling method and system, and storage medium |
| CN110659117A (en) * | 2019-08-23 | 2020-01-07 | 阿里巴巴集团控股有限公司 | Service index task scheduling and executing method, device, system and storage medium |
| CN110716796A (en) * | 2019-09-02 | 2020-01-21 | 中国平安财产保险股份有限公司 | Intelligent task scheduling method and device, storage medium and electronic equipment |
| CN111190710A (en) * | 2019-11-25 | 2020-05-22 | 泰康保险集团股份有限公司 | Task allocation method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020119029A1 (en) | 2020-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109669775A (en) | Distributed task dispatching method, system and storage medium | |
| CN109684080A (en) | Centralized method for scheduling task, system and storage medium | |
| CN108229767B (en) | Order picking task distribution method, device and system | |
| CN104243405B (en) | A kind of request processing method, apparatus and system | |
| CN108762931A (en) | Method for scheduling task, server based on distributed scheduling system and storage medium | |
| CN107360206A (en) | A kind of block chain common recognition method, equipment and system | |
| CN109933420A (en) | Node tasks dispatching method, electronic equipment and system | |
| CN109634748A (en) | Cluster resource dispatching method, device, equipment and computer readable storage medium | |
| CN108683720A (en) | A kind of container cluster service configuration method and device | |
| CN109906421A (en) | Processor core based on thread importance divides | |
| CN102147746A (en) | Dynamic thread pool management | |
| CN108399101A (en) | The methods, devices and systems of scheduling of resource | |
| CN103873587B (en) | A kind of method and device that scheduling is realized based on cloud platform | |
| CN107451147A (en) | A kind of method and apparatus of kafka clusters switching at runtime | |
| CN109634915A (en) | File dispositions method, Cloud Server, system and storage medium | |
| CN110457138A (en) | Management method, device and the electronic equipment of game server cluster | |
| CN107341020A (en) | Implementation method and device, the desktop cloud system and terminal device of video card virtualization | |
| CN109725991A (en) | Task processing method, device, equipment and readable storage medium storing program for executing | |
| CN103713852A (en) | Method for processing information, service platform and electronic devices | |
| CN108551585A (en) | A kind of barrage method of calibration, device, server and medium | |
| CN113079139B (en) | Block chain-based consensus group master node determination method, device and system | |
| CN104753938B (en) | A kind of information processing method and electronic equipment | |
| CN113285886B (en) | Bandwidth allocation method and device, electronic equipment and readable storage medium | |
| CN109658043A (en) | Collection data processing method, device, equipment and storage medium | |
| CN110427273A (en) | Scheduling memory method, apparatus, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |