CN106371918A - Task cluster scheduling management method and apparatus - Google Patents
Task cluster scheduling management method and apparatus Download PDFInfo
- Publication number
- CN106371918A CN106371918A CN201610709610.8A CN201610709610A CN106371918A CN 106371918 A CN106371918 A CN 106371918A CN 201610709610 A CN201610709610 A CN 201610709610A CN 106371918 A CN106371918 A CN 106371918A
- Authority
- CN
- China
- Prior art keywords
- task
- execution
- terminal
- workflow
- scheduling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
Abstract
The invention provides a task cluster scheduling management method and apparatus. The method comprises a workflow scheduling step of performing scheduling on pre-defined workflow which comprises multiple groups of tasks with an executive dependence sequence, a task allocation step of allocating a group of tasks without a forward dependence relationship to at least one task execution terminal for execution, a monitoring step of monitoring an executive state of the group of tasks without the forward dependence relationship in the at least one task execution terminal, and a task scheduling step of scheduling residual tasks according to the executive dependence relationship of the tasks when it is monitored that the execution of the group of tasks without the forward dependence relationship is finished, and repeating the task allocation step, the monitoring step and the task scheduling step until the execution of all the tasks in the workflow is finished. A task execution sequence is established by setting the executive dependence relationship among the tasks, so that the technical problems of resource waste and low task execution efficiency caused by disordered task allocation can be effectively avoided.
Description
Technical field
The present invention relates to computer task colony dispatching management domain, in particular to a kind of task colony dispatching pipe
Reason method and device.
Background technology
Mass computing task usually includes many subtasks, needs to have executed all subtasks in mass computing task
Finish, this mass computing task could be finished.As only considered to carry out point subtask in the implementation procedure of many subtasks
Cloth process, it is understood that there may be problems with, some of which subtask need to rely on the result of other subtasks, even if by these
Subtask is distributed to execution these subtasks of terminal and also cannot execute at once, and the execution terminal being distributed these subtasks can be empty
Put, have a strong impact on the efficiency of tasks carrying.
Content of the invention
In order to overcome above-mentioned deficiency of the prior art, the technical problem to be solved is to provide a kind of task-set
Group's schedule management method and device, it can enter row order according to the execution dependence between task and task to task and hold
OK, improve the efficiency of tasks carrying.
The present invention one better embodiment provides a kind of task cluster schedule management method, is applied to and tasks carrying terminal
The task management terminal of communication is it is characterised in that methods described includes:
Workflow schedule step, is scheduling to predefined workflow, and described predefined workflow includes many
There is the task of execution dependence order, a group task of wherein no preposition dependence is to need execute at first to appoint between group
Business；
Task distributing step, a group task of described no preposition dependence is distributed at least one tasks carrying terminal and holds
OK；
Monitor step, the group task monitoring described no preposition dependence is in described at least one tasks carrying terminal
Practice condition；
Task scheduling step, when the group task execution listening to described no preposition dependence completes, according to task
Execution relies on sequential scheduling residue task；
Repeat described task distributing step, monitor step and task scheduling step, all of in described workflow
Business execution completes.
Another preferred embodiment of the present invention also provides a kind of task cluster schedule management method, is applied to communicate each other
The tasks carrying terminal connecting and task management terminal, methods described includes:
Workflow schedule step, the predefined workflow of described task management terminal-pair is scheduling, described fixed in advance
The workflow of justice include multigroup between there is the task of execution dependence order, a group task of wherein no preposition dependence is need
Executing at first of task；
Task distributing step, a group task of described no preposition dependence is distributed at least by described task management terminal
One tasks carrying terminal execution；
Task execution step, described at least one tasks carrying terminal executes a group task of described no preposition dependence；
Monitor step, a group task of no preposition dependence described in described task management terminal monitoring is described at least one
Practice condition in tasks carrying terminal；
Task scheduling step, described task management terminal is listening to a group task execution of described no preposition dependence
When completing, rely on sequential scheduling residue task according to tasks carrying；
Iterative task distributing step, task execution step, monitoring step and task scheduling step, until described workflow is held
Row completes.
Another preferred embodiment of the present invention also provides a kind of task cluster dispatching managing device, is applied to tasks carrying eventually
It is characterised in that described device includes in the task management terminal of end communication:
Workflow-deployment module, for being scheduling to predefined workflow, described predefined workflow packages
Include multigroup between there is the task of execution dependence order, a group task of wherein no preposition dependence needs to execute at first
Task；
Task distribution module, for being distributed at least one tasks carrying eventually by a group task of described no preposition dependence
End execution；
Monitor module, for monitoring a group task of described no preposition dependence in described at least one tasks carrying terminal
On practice condition；
Task scheduling modules, during for completing in the group task execution listening to described no preposition dependence, according to
Tasks carrying relies on sequential scheduling residue task；
Wherein, described task distribution module, monitoring module and task scheduling modules circulate execution, successively until described work
In flowing, all of tasks carrying completes.
In terms of existing technologies, the method have the advantages that
Set up the sequencing of tasks carrying by the execution dependence between setting task, appoint first carrying out
After the completion of business execution, just the other tasks depending on this task can be distributed to execution terminal and be executed, can be prevented effectively from
The wasting of resources that causes and the low technical problem of tasks carrying efficiency because of the unordered distribution of task.
Brief description
In order to be illustrated more clearly that the technical scheme of the embodiment of the present invention, below will be attached to use required in embodiment
Figure is briefly described it will be appreciated that the following drawings illustrate only certain embodiments of the present invention, and it is right to be therefore not construed as
The restriction of scope, for those of ordinary skill in the art, on the premise of not paying creative work, can also be according to this
A little accompanying drawings obtain other related accompanying drawings.
Fig. 1 be task management terminal provided in an embodiment of the present invention with least one tasks carrying terminal interact schematic diagram；
Fig. 2 is the block diagram of task management terminal provided in an embodiment of the present invention；
A kind of function mould of task cluster dispatching managing device shown in Fig. 2 that Fig. 3 provides for first embodiment of the invention
Block block diagram；
Fig. 4 is the particular flow sheet of the task cluster schedule management method that first embodiment of the invention provides；
Fig. 5 is the schematic diagram of workflow in the specific example that first embodiment of the invention provides；
Fig. 6 is the schematic diagram of the scheduler cross-certification that first embodiment of the invention provides；
Fig. 7 is the particular flow sheet of the task cluster schedule management method that second embodiment of the invention provides.
In above-mentioned accompanying drawing, each reference is corresponding entitled:
Specific embodiment
Below in conjunction with accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Ground description is it is clear that described embodiment is only a part of embodiment of the present invention, rather than whole embodiments.Generally exist
The assembly of the embodiment of the present invention described and illustrated in accompanying drawing can be arranged with various different configurations and design herein.Cause
This, be not intended to limit claimed invention to the detailed description of the embodiments of the invention providing in the accompanying drawings below
Scope, but it is merely representative of the selected embodiment of the present invention.Based on embodiments of the invention, those skilled in the art are not doing
The every other embodiment being obtained on the premise of going out creative work, broadly falls into the scope of protection of the invention.
It should also be noted that similar label and letter expression similar terms in following accompanying drawing, therefore, once a certain Xiang Yi
It is defined in individual accompanying drawing, then do not need it to be defined further and explains in subsequent accompanying drawing.Meanwhile, the present invention's
In description, term " first ", " second " etc. are only used for distinguishing description, and it is not intended that indicating or hint relative importance.
Refer to Fig. 1, be that the task management terminal 100 that present pre-ferred embodiments provide is led to tasks carrying terminal 200
The interactive schematic diagram of letter.Described task management terminal 100 can be communicated with described tasks carrying terminal 200 by network 300,
With the data cube computation realized between described task management terminal 100 and described tasks carrying terminal 200 or interact.
Refer to Fig. 2, be the block diagram that Fig. 1 is a kind of task management terminal 100 provided in an embodiment of the present invention.Institute
State task management terminal 100 and include task cluster dispatching managing device 110, memorizer 111, storage control 112, processor
113 and communication unit 114.
Described memorizer 111, storage control 112, processor 113 and each element of communication unit 114 are direct each other
Or be indirectly electrically connected with, to realize transmission or the interaction of data.For example, these elements can pass through one or more each other
Communication bus or holding wire are realized being electrically connected with.Described task cluster dispatching managing device 110 includes at least one can be with software
Or the form of firmware (firmware) is stored in described memorizer 111 or is solidificated in the operation of described task management terminal 100
Software function module in system (operating system, os).Described memorizer 111 stores application program.Described process
Device 113 is used for executing the executable module of storage in described memorizer 111, for example described task cluster dispatching managing device 110
Included software function module and computer program etc..
Wherein, described memorizer 111 may be, but not limited to, random access memory (random access
Memory, ram), read only memory (read only memory, rom), programmable read only memory (programmable
Read-only memory, prom), erasable read-only memory (erasable programmable read-only
Memory, eprom), electricallyerasable ROM (EEROM) (electric erasable programmable read-only
Memory, eeprom) etc..Wherein, memorizer 111 be used for storage program, described processor 113 after receiving execute instruction,
Execution described program.Described processor 113 and other possible assemblies can control in described storage to the access of memorizer 111
Carry out under the control of device 112.
Described processor 113 is probably a kind of IC chip, has the disposal ability of signal.Above-mentioned processor
113 can be general processor, including central processing unit (central processing unit, cpu), network processing unit
(network processor, np) etc.；Can also be digital signal processor (dsp)), special IC (asic), ready-made
Programmable gate array (fpga) or other PLDs, discrete gate or transistor logic, discrete hardware group
Part.Can realize or execute disclosed each method in the embodiment of the present invention, step and logic diagram.General processor is permissible
It is microprocessor or this processor can also be any conventional processor etc..
Described communication unit 114 is used for setting up described task management terminal 100 by described network 300 and tasks carrying is whole
End 200 communication connections between the two, thus realize described task management terminal 100 hold in described tasks carrying terminal 200
The management of row task.
First embodiment
Refer to Fig. 3, Fig. 3 is task cluster dispatching managing device shown in Fig. 2 110 that present pre-ferred embodiments provide
Functional block diagram.Described task cluster dispatching managing device 110 includes workflow-deployment module 1101, task distribution module
1102nd, module 1103 and task scheduling modules 1104 are monitored.Fig. 4 is the task colony dispatching pipe that present pre-ferred embodiments provide
The flow chart of reason method.The method and step of the related flow process of methods described can be by described workflow-deployment module 1101, task
Distribution module 1102, monitor module 1103 and task scheduling modules 1104 to realize.Below how modules are specifically realized
Idiographic flow is described in detail.
Step s111, described workflow-deployment module 1101 is scheduling to predefined workflow.
In the present embodiment, described predefined workflow include multigroup between exist execution dependence order task.
Specifically, described task management terminal can be pre-defined to workflow by visualization interface, its
In, the execution sequencing between definition task and task is included, but not limited to the predefined content of workflow.Lift
Example illustrates it is assumed that it is now desired to being processed the information of trade company for the use of other platforms, wherein, the information of trade company
Including but not limited to merchant transaction report data statistics, activity data report form statistics and trade company's member's report data statistics.In order to
Make data compatible or consistent between different platform, Data Control can be carried out in the example shown by way of white list.This
Sample just develops and a data work flow: obtains white list trade company -- > and proposes basic data -- > data secondary operations -- > and raw
Become business datum.The course of processing of data is referred to Fig. 5, according to Fig. 5, it is known that above-mentioned workflow is by 8 tasks
Composition.Task dependence is: first carries out task [extracting white list trade company id], treats that this tasks carrying executes after terminating again
[newly-increased white list trade company id inserts other platform service forms], after tasks carrying terminates, more simultaneously and concurrently executes task [trade company
Transaction report data statistics], [activity data report form statistics] and [trade company member's report data count], treat task activity datagram
After list processing terminates, then execution task [scene marketing report form statistics], execution terminates followed by execution [scene marketing form knot
Fruit is put in storage].Task [terminating data mart modeling, rehabilitation] needs to wait merchant transaction report data statistics, trade company's member's form
Data statisticss, scene marketing form result are put in storage could start to process after the whole process of these three tasks terminates.Understanding work
After mutual relation between task in stream, you can to define task on the visualization interface in described task management terminal, and
Set up the execution dependence between task.
In the present embodiment, described visualization interface can also arrange following functions.Resource distribution function, above-mentioned to appoint
Business definition adopt this resource distribution functional realiey, described resource distribution function can also include configuration timer-triggered scheduler,
Real time inspection server registration situation etc..Control of authority function, realizes function privilege, data permission by user, role, authority
Configuration management.Elastic dilatation function, completes dilatation by automation services login mechanism, and the dilatation of execution terminal needs two steps complete
Become, first pass through resources configuration management interface editing tasks carrying terminal quantity, then increase tasks carrying terminal disposition node
Register machine by automation services and complete dilatation.Real-time status monitoring function, the running status of monitor in real time workflow and task,
The process progress of Real-time Feedback task, run succeeded or failure situation and run time-out warning situations such as.
In the present embodiment, after user carries out to workflow predefining, task cluster dispatching managing device 110 basis
By figure whether this dependence is assembled into a figure by workflow, and whether the workflow of definition is correct, whether there is closed loop,
Meet the feature of directed acyclic graph to be judged.Specific evaluation algorithm can be using Depth Priority Algorithm and topology row
Sequence algorithm, wherein workflow in the process of implementation, are processed by directed acyclic graph is changed into a topological queue.
Step s112, task is distributed at least one tasks carrying terminal execution by task distribution module 1102.
In the present embodiment, a group task of no preposition dependence is distributed to by described task distribution module 1102 first
At least one tasks carrying terminal execution.
Specifically, a group task of described no preposition dependence refers to that the execution of this group task needs not rely on other
Business implementing result it is however generally that, in the present embodiment, a group task of described no preposition dependence refers in workflow
First task of needing execution.Such as, the task in the example above " extracting white list trade company id ".
Task, when carrying out task distribution, is also carried out burst and processes to obtain number of tasks by described task distribution module 1102
According to burst, and by described task data burst be sent to described at least one tasks carrying terminal execution.
In the present embodiment, the quantity of task data burst be execute this task execution terminal data and registration execute end
Smaller value in terminal number amount.
Wherein, the execution terminal quantity of task can be defined by the visualization interface in described task management terminal 100:
M is it is also possible to obtain registration execution terminal quantity: r by reading zookeeper.Described data fragmentation piece number p=min m,
r}.
Step s113, monitors module 1103 and monitors execution shape in described at least one tasks carrying terminal for the described task
Condition.
In the present embodiment, log system holding to the data fragmentation in described at least one tasks carrying terminal can be passed through
Row daily record is collected, to realize the monitoring to the practice condition stated at least one tasks carrying terminal.
Specifically, by being transformed to log system so that the daily record producing in described execution terminal is according to workflow
Execution address carry out packet aggregation, the daily record collected is arranged by final log system, make one execution address correspond to
One journal file.
In the present embodiment, described task management terminal adopts the scheduling mechanism based on event, by the thing of zookeeper
Part monitors and achieves event synchronization in cluster, by the encapsulation again to event, it is to avoid same event is many in cluster
Secondary response.
Step s114, task scheduling modules 1104, when listening to described tasks carrying and completing, judge whether also in need
The task of execution, relies on sequential scheduling residue task when also sometimes entering step s112 according to tasks carrying, iterative task is distributed
Step s112, monitoring step s113 and task scheduling step s114, in described workflow, all of tasks carrying completes.When
When not needing executing of task, terminate flow process.
In the case of there is execution dependence between multiple tasks, according to the sequencing executing in workflow
Task carries out order and carries out distributed scheduling execution.
In the present embodiment, described task cluster schedule management method can also include: choose master for each workflow and hold
Row terminal, described main execution terminal has workflow described in right scheduling.
Specifically, judge the weighted value size of all online execution terminals, if weighted value is different, select weighted value maximum
Execution terminal based on execute terminal；If weighted value is identical, randomly chooses one from all online execution terminals and execute end
End is as main execution terminal；If cannot randomly choosing from all online execution terminals and obtaining main execution terminal, by work
The execution terminal flowing down first registration is as described main execution terminal.
In the present embodiment, an execution terminal is randomly choosed as main execution terminal using random function, wherein, described
Random function can be realized using conventional random distribution algorithm (such as, concordance hash algorithm).
Specifically, described execution terminal carries out intersecting registration under each workflow, by lower for workflow first registration
Execution terminal is as described main execution terminal, or the weighted value according to registration execution terminal under each workflow, selects weighted value
Maximum execution terminal is as described main execution terminal.
In the present embodiment, the execution terminal only executing terminal based on one-tenth just has this workflow of right scheduling, in order to avoid
, by same execution terminal as main execution terminal, described task cluster dispatching managing device is to execution terminal for most workflows
On scheduler carry out load balance process.Specifically, refer to Fig. 6, be to carry out selecting main execution using intersection logon mode
The schematic diagram of terminal, carries out main execution terminal by the way of intersecting registration for workflow flow1, flow2 and flow3 and divides
Join.By intersecting login mechanism, the load balancing of the preliminary scheduler realized, it is to avoid the scheduling of all working stream falls and holds at
In row terminal.
In the present embodiment, can smoothly timely execute to ensure task, described task cluster schedule management method is adopted
With the mode of failover, the task of execution failure is shifted.Specifically, described monitoring module 1103 is listening to task
When task data burst in actuating station executes unsuccessfully, the task data burst of described execution failure is scheduled to other tasks and holds
Row terminal executes.
Second embodiment
The present embodiment provides a kind of task cluster schedule management method, and this task cluster schedule management method is applied to mutually
Between communication connection tasks carrying terminal 200 and task management terminal 100, methods described includes:
Step s211, described task management terminal 100 is scheduling to predefined workflow, described predefined
Workflow include multigroup between exist execution dependence order task, a group task of wherein no preposition dependence is to need
First carrying out of task.
Step s212, task is distributed at least one tasks carrying terminal execution by described task management terminal 100.
In the present embodiment, a group task of no preposition dependence is distributed to by described task management terminal 100 first
At least one tasks carrying terminal execution.
Step s213, described at least one tasks carrying terminal 200 executes described task.
Step s214, described task management terminal 100 monitors described task in described at least one tasks carrying terminal 200
Practice condition.
Step s215, described task management terminal 100, when listening to described tasks carrying and completing, judges whether also to need
Task to be executed, relies on sequential scheduling residue task when also sometimes entering step s212 according to tasks carrying, iterative task divides
Send out step s212, task execution step s213, monitor step s214 and step s215, all of task in described workflow
Execution completes.When the task of not needing to execute, terminate flow process.
In sum, the present invention implements task cluster schedule management method and the device providing, between setting task
Execution dependence set up the sequencing of tasks carrying, after the completion of should first carrying out tasks carrying, just can will rely on
Other tasks in this task are distributed to execution terminal and are executed, and can be prevented effectively from the money that causes because of the unordered distribution of task
The technical problem that source wastes and tasks carrying efficiency is low.
It should be understood that disclosed equipment and method are it is also possible to pass through in several embodiments provided herein
Other modes are realized.Apparatus embodiments described above are only schematically, for example, the flow chart in accompanying drawing and block diagram
Show the equipment of multiple embodiments according to the present invention, the architectural framework in the cards of method and computer program product,
Function and operation.At this point, each square frame in flow chart or block diagram can represent the one of a module, program segment or code
Part, a part for described module, program segment or code comprises holding of one or more logic function for realizing regulation
Row instruction.It should also be noted that at some as in the implementation replaced, the function of being marked in square frame can also be to be different from
The order being marked in accompanying drawing occurs.For example, two continuous square frames can essentially execute substantially in parallel, and they are sometimes
Can execute in the opposite order, this is depending on involved function.It is also noted that it is every in block diagram and/or flow chart
The combination of the square frame in individual square frame and block diagram and/or flow chart, can be with the special base of the function of execution regulation or action
System in hardware to be realized, or can be realized with combining of computer instruction with specialized hardware.
In addition, each functional module in each embodiment of the present invention can integrate one independent portion of formation
Divide or modules individualism is it is also possible to two or more modules are integrated to form an independent part.
If described function realized using in the form of software function module and as independent production marketing or use when, permissible
It is stored in a computer read/write memory medium.Based on such understanding, technical scheme is substantially in other words
Partly being embodied in the form of software product of part that prior art is contributed or this technical scheme, this meter
Calculation machine software product is stored in a storage medium, including some instructions with so that a computer equipment (can be individual
People's computer, server, or network equipment etc.) execution each embodiment methods described of the present invention all or part of step.
And aforesaid storage medium includes: u disk, portable hard drive, read only memory (rom, read-only memory), random access memory are deposited
Reservoir (ram, random access memory), magnetic disc or CD etc. are various can be with the medium of store program codes.
It should be noted that herein, such as first and second or the like relational terms are used merely to a reality
Body or operation are made a distinction with another entity or operation, and not necessarily require or imply these entities or deposit between operating
In any this actual relation or order.And, term " inclusion ", "comprising" or its any other variant are intended to
Comprising of nonexcludability, wants so that including a series of process of key elements, method, article or equipment and not only including those
Element, but also include other key elements being not expressly set out, or also include for this process, method, article or equipment
Intrinsic key element.In the absence of more restrictions, the key element that limited by sentence "including a ..." it is not excluded that
Also there is other identical element including in the process of described key element, method, article or equipment.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for the skill of this area
For art personnel, the present invention can have various modifications and variations.All within the spirit and principles in the present invention, made any repair
Change, equivalent, improvement etc., should be included within the scope of the present invention.It should also be noted that similar label and letter exist
Representing similar terms in figure below, therefore, once being defined in a certain Xiang Yi accompanying drawing, being then not required in subsequent accompanying drawing
It is defined further and to be explained.
The above, the only specific embodiment of the present invention, but protection scope of the present invention is not limited thereto, and any
Those familiar with the art the invention discloses technical scope in, change or replacement can be readily occurred in, all should contain
Cover within protection scope of the present invention.Therefore, protection scope of the present invention should described be defined by scope of the claims.
Claims (10)
1. a kind of task cluster schedule management method is it is characterised in that be applied to the task management with tasks carrying terminal communication
Terminal is it is characterised in that methods described includes:
Workflow schedule step, is scheduling to predefined workflow, described predefined workflow include multigroup it
Between exist execution dependence order task, a group task of wherein no preposition dependence is the task of needing to execute at first；
Task distributing step, a group task of described no preposition dependence is distributed at least one tasks carrying terminal execution；
Monitor step, monitor the execution in described at least one tasks carrying terminal for the group task of described no preposition dependence
Situation；
Task scheduling step, when the group task execution listening to described no preposition dependence completes, according to tasks carrying
Rely on sequential scheduling residue task；
Repeat described task distributing step, monitor step and task scheduling step, all of task is held in described workflow
Row completes.
2. task cluster schedule management method as claimed in claim 1 is it is characterised in that in described task distributing step:
By task carry out burst process obtain task data burst, by described task data burst be distributed to described at least one task
Execution terminal execution.
3. task cluster schedule management method as claimed in claim 2 is it is characterised in that the quantity of described task data burst
Execution terminal quantity and registration for executing this task execute the smaller value in terminal quantity.
4. task cluster schedule management method as claimed in claim 1 is it is characterised in that described task scheduling step includes:
When task data burst on listening to tasks carrying end executes unsuccessfully, by the task data burst of described execution failure
It is scheduled to other tasks carrying terminal execution.
5. task cluster schedule management method as claimed in claim 1 is it is characterised in that methods described also includes:
By the visualization interface in described task management terminal, described workflow is defined, wherein, described workflow
Definition includes the execution sequencing between definition task and task.
6. task cluster schedule management method as claimed in claim 1 is it is characterised in that methods described also includes:
Choose main execution terminal for each workflow, described main execution terminal has the authority dispatching described workflow.
7. task cluster schedule management method as claimed in claim 6 is it is characterised in that described choose master for each workflow
The step of execution terminal includes:
Judge the weighted value size of all online execution terminals, if weighted value is different, select the maximum execution terminal of weighted value
Based on execute terminal；If weighted value is identical, randomly chooses an execution terminal from all online execution terminals and hold as master
Row terminal；If cannot randomly choosing from all online execution terminals and obtaining main execution terminal, by lower for workflow first
The execution terminal of registration is as described main execution terminal.
8. a kind of task cluster schedule management method is it is characterised in that be applied to the tasks carrying communicating to connect each other eventually
End and task management terminal, methods described includes:
Workflow schedule step, the predefined workflow of described task management terminal-pair is scheduling, described predefined
Workflow include multigroup between exist execution dependence order task, a group task of wherein no preposition dependence is to need
First carrying out of task；
Task distributing step, a group task of described no preposition dependence is distributed at least one by described task management terminal
Business execution terminal execution；
Task execution step, described at least one tasks carrying terminal executes a group task of described no preposition dependence；
Monitor step, described in described task management terminal monitoring, a group task of no preposition dependence is in described at least one task
Practice condition in execution terminal；
Task scheduling step, described task management terminal completes in the group task execution listening to described no preposition dependence
When, rely on sequential scheduling residue task according to tasks carrying；
Iterative task distributing step, task execution step, monitoring step and task scheduling step, until described workflow execution is complete
Become.
9. a kind of task cluster dispatching managing device is it is characterised in that be applied to the task management with tasks carrying terminal communication
It is characterised in that described device includes in terminal:
Workflow-deployment module, for being scheduling to predefined workflow, described predefined workflow includes many
There is the task of execution dependence order, a group task of wherein no preposition dependence is to need execute at first to appoint between group
Business；
Task distribution module, holds for a group task of described no preposition dependence is distributed at least one tasks carrying terminal
OK；
Monitor module, for monitoring a group task of described no preposition dependence in described at least one tasks carrying terminal
Practice condition；
Task scheduling modules, during for completing in the group task execution listening to described no preposition dependence, according to task
Execution relies on sequential scheduling residue task；
Wherein, described task distribution module, monitoring module and task scheduling modules circulate execution, successively until described workflow
In all of tasks carrying complete.
10. task cluster dispatching managing device as claimed in claim 9 it is characterised in that:
Task is carried out burst process and obtains task data burst by described task distribution module, and described task data burst is distributed
To described at least one tasks carrying terminal execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610709610.8A CN106371918B (en) | 2016-08-23 | 2016-08-23 | Task cluster schedule management method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610709610.8A CN106371918B (en) | 2016-08-23 | 2016-08-23 | Task cluster schedule management method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106371918A true CN106371918A (en) | 2017-02-01 |
| CN106371918B CN106371918B (en) | 2019-10-18 |
Family
ID=57878914
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610709610.8A Active CN106371918B (en) | 2016-08-23 | 2016-08-23 | Task cluster schedule management method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106371918B (en) |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107908469A (en) * | 2017-12-18 | 2018-04-13 | 北京搜狐新媒体信息技术有限公司 | A kind of method for scheduling task and system |
| CN108319499A (en) * | 2018-02-07 | 2018-07-24 | 麒麟合盛网络技术股份有限公司 | Method for scheduling task and device |
| CN108537543A (en) * | 2018-03-30 | 2018-09-14 | 百度在线网络技术（北京）有限公司 | Method for parallel processing, device, equipment and the storage medium of block chain data |
| CN108595316A (en) * | 2018-04-20 | 2018-09-28 | 星环信息科技（上海）有限公司 | Life cycle management method, manager, equipment and the medium of Distributed Application |
| CN108762932A (en) * | 2018-05-31 | 2018-11-06 | 安徽四创电子股份有限公司 | A kind of cluster task scheduling system and processing method |
| CN108958920A (en) * | 2018-07-13 | 2018-12-07 | 众安在线财产保险股份有限公司 | A kind of distributed task dispatching method and system |
| CN109165088A (en) * | 2018-09-06 | 2019-01-08 | 华泰证券股份有限公司 | It is a kind of based on elastic-job frame can single machine elasticity dilatation task sharding method |
| CN109522100A (en) * | 2017-09-19 | 2019-03-26 | 阿里巴巴集团控股有限公司 | Real-time calculating task method of adjustment and device |
| CN110187958A (en) * | 2019-06-04 | 2019-08-30 | 上海燧原智能科技有限公司 | A kind of task processing method, device, system, equipment and storage medium |
| CN110231991A (en) * | 2019-05-31 | 2019-09-13 | 新华三大数据技术有限公司 | A kind of method for allocating tasks, device, electronic equipment and readable storage medium storing program for executing |
| CN110428453A (en) * | 2019-07-30 | 2019-11-08 | 深圳云天励飞技术有限公司 | Data processing method, device, data processing equipment and storage medium |
| CN110443587A (en) * | 2019-08-12 | 2019-11-12 | 北京神舟航天软件技术有限公司 | A kind of workflow schedule method |
| CN110502343A (en) * | 2019-08-23 | 2019-11-26 | 深圳市新系区块链技术有限公司 | A kind of resource allocation methods, system, device and computer readable storage medium |
| CN110519381A (en) * | 2019-08-29 | 2019-11-29 | 深圳前海微众银行股份有限公司 | Processing method, device, terminal device and the storage medium of distributed task scheduling |
| CN110659116A (en) * | 2019-08-15 | 2020-01-07 | 中国科学院计算机网络信息中心 | Big data ETL task scheduling method |
| CN110704183A (en) * | 2019-09-18 | 2020-01-17 | 深圳前海大数金融服务有限公司 | Data processing method, system and computer readable storage medium |
| CN110780981A (en) * | 2018-07-31 | 2020-02-11 | 北京科杰信息技术有限公司 | Task scheduling management system and method for hadoop big data platform |
| CN111367642A (en) * | 2020-03-09 | 2020-07-03 | 中国铁塔股份有限公司 | Task scheduling execution method and device |
| CN111506659A (en) * | 2020-04-20 | 2020-08-07 | 杭州数澜科技有限公司 | Data synchronization method, system and medium |
| CN111553545A (en) * | 2020-07-13 | 2020-08-18 | 四川新网银行股份有限公司 | Method for optimizing batch task scheduling dependence |
| CN111858013A (en) * | 2020-06-19 | 2020-10-30 | 中科院计算所西部高等技术研究院 | Workflow job scheduling control method |
| CN112650657A (en) * | 2019-10-10 | 2021-04-13 | 浙江大搜车软件技术有限公司 | Business system analysis method and device based on stream computing |
| CN112748999A (en) * | 2021-01-28 | 2021-05-04 | 新疆睿思高科信息科技有限公司 | Cluster scheduling system and method |
| CN113010280A (en) * | 2021-02-19 | 2021-06-22 | 北京字节跳动网络技术有限公司 | Distributed task processing method, system, device, equipment and medium |
| CN113821326A (en) * | 2021-09-24 | 2021-12-21 | 北京天融信网络安全技术有限公司 | Job scheduling method and device, electronic equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101216911A (en) * | 2008-01-04 | 2008-07-09 | 清华大学 | Dynamic workflow model subdivision method supporting distributed execution |
| CN104503833A (en) * | 2014-12-22 | 2015-04-08 | 广州唯品会网络技术有限公司 | Task scheduling optimization method and device |
| CN104965754A (en) * | 2015-03-31 | 2015-10-07 | 腾讯科技（深圳）有限公司 | Task scheduling method and task scheduling apparatus |
-
2016
- 2016-08-23 CN CN201610709610.8A patent/CN106371918B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101216911A (en) * | 2008-01-04 | 2008-07-09 | 清华大学 | Dynamic workflow model subdivision method supporting distributed execution |
| CN104503833A (en) * | 2014-12-22 | 2015-04-08 | 广州唯品会网络技术有限公司 | Task scheduling optimization method and device |
| CN104965754A (en) * | 2015-03-31 | 2015-10-07 | 腾讯科技（深圳）有限公司 | Task scheduling method and task scheduling apparatus |
Cited By (39)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109522100A (en) * | 2017-09-19 | 2019-03-26 | 阿里巴巴集团控股有限公司 | Real-time calculating task method of adjustment and device |
| CN109522100B (en) * | 2017-09-19 | 2023-03-31 | 阿里巴巴集团控股有限公司 | Real-time computing task adjusting method and device |
| CN107908469B (en) * | 2017-12-18 | 2020-08-28 | 北京搜狐新媒体信息技术有限公司 | Task scheduling method and system |
| CN107908469A (en) * | 2017-12-18 | 2018-04-13 | 北京搜狐新媒体信息技术有限公司 | A kind of method for scheduling task and system |
| CN108319499B (en) * | 2018-02-07 | 2021-06-04 | 麒麟合盛网络技术股份有限公司 | Task scheduling method and device |
| CN108319499A (en) * | 2018-02-07 | 2018-07-24 | 麒麟合盛网络技术股份有限公司 | Method for scheduling task and device |
| CN108537543A (en) * | 2018-03-30 | 2018-09-14 | 百度在线网络技术（北京）有限公司 | Method for parallel processing, device, equipment and the storage medium of block chain data |
| CN108595316A (en) * | 2018-04-20 | 2018-09-28 | 星环信息科技（上海）有限公司 | Life cycle management method, manager, equipment and the medium of Distributed Application |
| CN108595316B (en) * | 2018-04-20 | 2020-02-21 | 星环信息科技（上海）有限公司 | Lifecycle management method, manager, device, and medium for distributed application |
| CN108762932A (en) * | 2018-05-31 | 2018-11-06 | 安徽四创电子股份有限公司 | A kind of cluster task scheduling system and processing method |
| CN108958920A (en) * | 2018-07-13 | 2018-12-07 | 众安在线财产保险股份有限公司 | A kind of distributed task dispatching method and system |
| CN108958920B (en) * | 2018-07-13 | 2021-04-06 | 众安在线财产保险股份有限公司 | Distributed task scheduling method and system |
| CN110780981B (en) * | 2018-07-31 | 2022-03-25 | 北京科杰科技有限公司 | Task scheduling management system and method for hadoop big data platform |
| CN110780981A (en) * | 2018-07-31 | 2020-02-11 | 北京科杰信息技术有限公司 | Task scheduling management system and method for hadoop big data platform |
| CN109165088A (en) * | 2018-09-06 | 2019-01-08 | 华泰证券股份有限公司 | It is a kind of based on elastic-job frame can single machine elasticity dilatation task sharding method |
| CN109165088B (en) * | 2018-09-06 | 2022-04-19 | 华泰证券股份有限公司 | Elastic expansion task segmentation method based on elastic-jobframe and capable of achieving single machine |
| CN110231991A (en) * | 2019-05-31 | 2019-09-13 | 新华三大数据技术有限公司 | A kind of method for allocating tasks, device, electronic equipment and readable storage medium storing program for executing |
| CN110187958B (en) * | 2019-06-04 | 2020-05-05 | 上海燧原智能科技有限公司 | Task processing method, device, system, equipment and storage medium |
| CN110187958A (en) * | 2019-06-04 | 2019-08-30 | 上海燧原智能科技有限公司 | A kind of task processing method, device, system, equipment and storage medium |
| CN110428453B (en) * | 2019-07-30 | 2020-12-15 | 深圳云天励飞技术有限公司 | Data processing method, data processing device, data processing equipment and storage medium |
| CN110428453A (en) * | 2019-07-30 | 2019-11-08 | 深圳云天励飞技术有限公司 | Data processing method, device, data processing equipment and storage medium |
| CN110443587A (en) * | 2019-08-12 | 2019-11-12 | 北京神舟航天软件技术有限公司 | A kind of workflow schedule method |
| CN110659116A (en) * | 2019-08-15 | 2020-01-07 | 中国科学院计算机网络信息中心 | Big data ETL task scheduling method |
| CN110502343B (en) * | 2019-08-23 | 2022-05-06 | 深圳市新系区块链技术有限公司 | Resource allocation method, system, device and computer readable storage medium |
| CN110502343A (en) * | 2019-08-23 | 2019-11-26 | 深圳市新系区块链技术有限公司 | A kind of resource allocation methods, system, device and computer readable storage medium |
| CN110519381A (en) * | 2019-08-29 | 2019-11-29 | 深圳前海微众银行股份有限公司 | Processing method, device, terminal device and the storage medium of distributed task scheduling |
| CN110519381B (en) * | 2019-08-29 | 2023-08-25 | 深圳前海微众银行股份有限公司 | Distributed task processing method and device, terminal equipment and storage medium |
| CN110704183A (en) * | 2019-09-18 | 2020-01-17 | 深圳前海大数金融服务有限公司 | Data processing method, system and computer readable storage medium |
| CN110704183B (en) * | 2019-09-18 | 2021-01-08 | 深圳前海大数金融服务有限公司 | Data processing method, system and computer readable storage medium |
| CN112650657A (en) * | 2019-10-10 | 2021-04-13 | 浙江大搜车软件技术有限公司 | Business system analysis method and device based on stream computing |
| CN111367642A (en) * | 2020-03-09 | 2020-07-03 | 中国铁塔股份有限公司 | Task scheduling execution method and device |
| CN111367642B (en) * | 2020-03-09 | 2023-08-22 | 中国铁塔股份有限公司 | Task scheduling execution method and device |
| CN111506659A (en) * | 2020-04-20 | 2020-08-07 | 杭州数澜科技有限公司 | Data synchronization method, system and medium |
| CN111858013A (en) * | 2020-06-19 | 2020-10-30 | 中科院计算所西部高等技术研究院 | Workflow job scheduling control method |
| CN111553545B (en) * | 2020-07-13 | 2020-10-27 | 四川新网银行股份有限公司 | Method for optimizing batch task scheduling dependence |
| CN111553545A (en) * | 2020-07-13 | 2020-08-18 | 四川新网银行股份有限公司 | Method for optimizing batch task scheduling dependence |
| CN112748999A (en) * | 2021-01-28 | 2021-05-04 | 新疆睿思高科信息科技有限公司 | Cluster scheduling system and method |
| CN113010280A (en) * | 2021-02-19 | 2021-06-22 | 北京字节跳动网络技术有限公司 | Distributed task processing method, system, device, equipment and medium |
| CN113821326A (en) * | 2021-09-24 | 2021-12-21 | 北京天融信网络安全技术有限公司 | Job scheduling method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106371918B (en) | 2019-10-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106371918A (en) | Task cluster scheduling management method and apparatus | |
| Sriram | SPECI, a simulation tool exploring cloud-scale data centres | |
| Cardoso et al. | Quality of service for workflows and web service processes | |
| Toosi et al. | Revenue maximization with optimal capacity control in infrastructure as a service cloud markets | |
| CN103761614A (en) | Project progress management method based on critical chain | |
| US8560359B2 (en) | System and methods for modeling consequences of events | |
| CN103986669B (en) | Assessment method of resource allocation strategy in cloud computing | |
| Singh et al. | Energy based efficient resource scheduling: a step towards green computing | |
| CN110928655A (en) | Task processing method and device | |
| CN1545674A (en) | Business process policy object | |
| US11070430B2 (en) | Persona/individual based actions based on community specific trigger | |
| CN111984426A (en) | Task scheduling method and device, electronic equipment and storage medium | |
| CN105005501B (en) | A kind of second order optimizing and scheduling task method towards cloud data center | |
| CN107957945B (en) | Monetary fund system automatic performance test method, device, equipment and storage medium | |
| Li et al. | MapReduce task scheduling in heterogeneous geo-distributed data centers | |
| CN105204947A (en) | Hybrid cloud computing resource management system based on commercial bank model | |
| CN112801464A (en) | Process management method and system | |
| CN115471215B (en) | Business process processing method and device | |
| CN115237592A (en) | Mixed cloud service flow scheduling method based on privacy perception | |
| Denkena et al. | Process planning and scheduling with multiagent systems | |
| Cao et al. | Online cost-rejection rate scheduling for resource requests in hybrid clouds | |
| Makhijani et al. | Quest: Queue simulation for content moderation at scale | |
| CN113626209A (en) | State machine-based event-driven method and system applied to transaction field | |
| CA2832806A1 (en) | Method for optimizing schedule compression in projects using accelerating and overlapping of activities | |
| Gulati et al. | Proposing security requirement prioritization framework |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221110 Address after: 037, Floor 3, Building 1, No. 88, Nongda South Road, Haidian District, Beijing 100,089 Patentee after: Yunting Zhitong (Beijing) Information Technology Co.,Ltd. Address before: Room 301, floor 3, building 1, yard 88, Nongda South Road, Haidian District, Beijing 100084 Patentee before: BEIJING YUNZONG INFORMATION TECHNOLOGY CO.,LTD. |