CN108762910A - A kind of distributed task dispatching method and system - Google Patents
A kind of distributed task dispatching method and system Download PDFInfo
- Publication number
- CN108762910A CN108762910A CN201810575719.6A CN201810575719A CN108762910A CN 108762910 A CN108762910 A CN 108762910A CN 201810575719 A CN201810575719 A CN 201810575719A CN 108762910 A CN108762910 A CN 108762910A
- Authority
- CN
- China
- Prior art keywords
- task
- resource
- goal
- unit
- queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
Abstract
This application discloses a kind of distributed task dispatching method and system, the goal task for meeting trigger condition is put into task queue by task trigger, and goal task is time driving generic task and/or event-driven generic task；Task dispatcher obtains the task schedule information of goal task after the notification information for receiving task queue publication, resource requirement configuration information corresponding with goal task type is loaded, and carries the resource requirements request of resource requirement configuration information to Resource Scheduler initiation；Resource Scheduler initiates to carry the establishing resource unit application of resource requirement configuration information to resource management platform according to resource requirements request；Resource management platform creates the task run unit of goal task according to the resource requirement configuration information in establishing resource unit application；The localization of task performer control task running unit runs the goal task.Realize the United Dispatching of different type task.
Description
Technical field
The present invention relates to task scheduling technique fields, more particularly to a kind of distributed task dispatching method and system.
Background technology
The business support system resource level of traditional entire operator is supported for minicomputer, after a whole set of Operation Support System
Platform batch processing task produces multiple-task application framework, such as the task based on time type, period according to time, event
Frame Task, the frame TF of real time processing tasks of task processing, support event driven foreground file to import task processing block
Frame BP supports event driven Scheduling Framework VM of workflow task etc..
The type of these task frameworks scheduling, application operation frame are different from.It is disposed currently based on minicomputer cluster, it is whole
A clustered machine quantity is because machine processing ability is big, and entire cluster scale is within controlled range, and each generic task is in every machine
The task process individually disposed corresponding Scheduling Framework on device and specifically executed, the tasks of all kinds of processing types reach it is thousands of even
Up to ten thousand, the problem of deployment and operation and maintenance of these tasks on minicomputer are all artificial treatment, O&M, challenges increasingly serious.
Currently, in order to solve the problems, such as the O&M of minicomputer cluster deployment, resource level is substituted small-sized using x86 servers
Machine.The machine quantity of decades of times is expanded compared to minicomputer in entire machine resources cluster level.It is former under the adjustment of this framework
The task of all kinds of task frameworks of first the machine deployment is compared to the quantity that original minicomputer has disposed hundreds of times extending transversely.These classes
The task scheduling frame of type is all deployed on every machine, and the upper task run of machine is completed in scheduling, although also supporting the machine
Task is split according to certain fragment rule on device, to realize task processing concurrency extension.But due to the money of x86 servers
The processing capacity in source pond, every machine all weakens much compared to minicomputer, also anxious in task processing capacity to be required to prop up
Transverse direction, the ability of Longitudinal Extension are held, the requirement that extensive background task calculates distributionization is coped with.
Invention content
In view of this, the present invention provides a kind of distributed task dispatching method and system, different type task is realized
United Dispatching, support vertical, the horizontal extension of task computation, improve task treatment effeciency.
In order to achieve the above-mentioned object of the invention, specific technical solution provided by the invention is as follows：
A kind of distributed task dispatching method, including：
The goal task for meeting trigger condition is put into task queue by task trigger, and the goal task drives for the time
Dynamic generic task and/or event-driven generic task；
Task dispatcher obtains the task of the goal task after the notification information for receiving the task queue publication
Scheduling information loads resource requirement configuration information corresponding with the goal task type, and is taken to Resource Scheduler initiation
Resource requirements request with the resource requirement configuration information；
Resource Scheduler carries the resource requirement to resource management platform initiation according to the resource requirements request and matches
The establishing resource unit application of confidence breath；
Resource management platform is according to the resource requirement configuration information in the establishing resource unit application, described in establishment
The task run unit of goal task；
Task performer controls the task run unit localization and runs the goal task.
Preferably, subscribing relationship is pre-established between the task queue and the task dispatcher；The method is also wrapped
It includes：
When the task queue receives the goal task, make an announcement information to the task dispatcher.
Preferably, the task dispatcher obtains the target after the notification information for receiving the task queue publication
The task schedule information of task, including：
The task dispatcher traverses after the notification information for receiving the task queue publication in the task queue
Task data；
According to the operating status in the task data in the task queue, it is to wait for operating status that batch, which obtains operating status,
Goal task task schedule information；
Or according to the task priority in the task data in the task queue, obtain the target of super objective priority
The task schedule information of task.
Preferably, the resource requirements request and the establishing resource unit application all include the task of the goal task
Mark；The task performer controls the task run unit localization and runs the goal task, including：
The task performer establishes registration connection with the task queue；
The task performer controls task identification of the task run unit according to the goal task, from described
The task data of the goal task is obtained in business queue, and the goal task is run according to task data localization.
Preferably, the method further includes：
When task run unit localization runs the goal task completion, the task performer is to described
Queue initiation task status of being engaged in updates application, is set as the operating status of goal task described in the task queue to complete shape
State；
The goal task is run in task run unit localization there are when abnormal conditions, the task performer
Task status update application is initiated to the task queue, the operating status of goal task described in the task queue is arranged
For abnormality.
Preferably, the method further includes：
The task dispatcher carries out periodic scan to the state of each task in the task queue；
For the task of completion status, the task dispatcher carries completion status to Resource Scheduler initiation and appoints
The resource unit of business mark destroys request, and the Resource Scheduler initiates TU task unit destruction to the resource management platform asks
It asks, the resource management platform destroys corresponding task running unit, and discharges the localization configuration money of corresponding task running unit
Source；
For the task of abnormality, the task dispatcher startup separator treatment mechanism retries task scheduling or opens automatically
Moving work Intervention Mechanism.
Preferably, the task dispatcher startup separator treatment mechanism retries task scheduling automatically, including：
When failure, which retries configured number, is not up to the upper limit, the task dispatcher initiates to carry to the Resource Scheduler
There is the resource release request of abnormality task identification；
The Resource Scheduler initiates TU task unit to the resource management platform and destroys request；
The resource management platform destroys corresponding task running unit, the localization configuration of release corresponding task running unit
Resource, and start other and destroyed TU task unit of the TU task unit with same asset configuration, operation again has described different
The goal task of normal state task mark.
A kind of distributed task dispatching system, including：Task trigger, task dispatcher, Resource Scheduler, resource management
Platform and task performer；
The task trigger, for the goal task for meeting trigger condition to be put into task queue, the target is appointed
Business is time driving generic task and/or event-driven generic task；
The task dispatcher is appointed for obtaining the target after the notification information for receiving the task queue publication
The task schedule information of business, loads corresponding with goal task type resource requirement configuration information, and to the resource
Scheduler initiates to carry the resource requirements request of the resource requirement configuration information；
The Resource Scheduler, for initiating to carry to the resource management platform according to the resource requirements request
State the establishing resource unit application of resource requirement configuration information；
The resource management platform, for matching confidence according to the resource requirement in the establishing resource unit application
Breath, creates the task run unit of the goal task；
The task performer controls the task run unit localization and runs the goal task.
Preferably,
The task performer is additionally operable to when task run unit localization runs the goal task completion,
Task status update application is initiated to the task queue, the operating status of goal task described in the task queue is arranged
For completion status；The goal task is run there are when abnormal conditions, to the task in task run unit localization
Task status update application is initiated in queue, sets the operating status of goal task described in the task queue to abnormal shape
State.
Preferably,
The task dispatcher is additionally operable to carry out periodic scan to the state of each task in the task queue；
For the task of completion status, the task dispatcher is additionally operable to carry to Resource Scheduler initiation
Request is destroyed at the resource unit of state task mark, the Resource Scheduler initiates TU task unit to the resource management platform
Request is destroyed, the resource management platform destroys corresponding task running unit, and discharges the localization of corresponding task running unit
Configure resource；
For the task of abnormality, the task dispatcher is additionally operable to startup separator treatment mechanism and retries task automatically
Scheduling starts manual intervention mechanism.
Compared with the existing technology, beneficial effects of the present invention are as follows：
Different types of task is summarized as time drive by a kind of distributed task dispatching method and system provided by the invention
Different types of task scheduling mode is integrated into unified method for scheduling task by dynamic generic task or event-driven generic task.Task
Scheduler loads the corresponding resource requirement configuration information of goal task type, and carries the money to Resource Scheduler initiation
The resource requirements request of source demand configuration information, Resource Scheduler are initiated according to the resource requirements request to resource management platform
Carry the establishing resource unit application of the resource requirement configuration information；Resource management platform is according to the establishing resource unit
The resource requirement configuration information in application, creates the task run unit of the goal task.Unified resource scheduling is integrated,
Resource Scheduler shields the difference of bottom different type resource, and such as containerization, virtualization resource, resource management platform is according to difference
The resource requirement configuration information of task creates different task run units, and parallelization handles task, maximumlly promotes task
Concurrent processing performance supports the horizontal and vertical dilatation ability of task.
Description of the drawings
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is a kind of distributed task dispatching system architecture diagram disclosed by the embodiments of the present invention；
Fig. 2 is a kind of distributed task dispatching method flow diagram disclosed by the embodiments of the present invention；
Fig. 3 is failure handling mechanisms schematic diagram disclosed by the embodiments of the present invention；
Fig. 4 is a kind of distributed task dispatching system structure diagram disclosed by the embodiments of the present invention.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Referring to Fig. 1, in distributed task dispatching Organization Chart disclosed by the invention, including task visualization domain, task tune
Spend domain, scheduling of resource domain and resource domains.Wherein, task visualization domain provides task visual console, and there is task to configure work(
Energy, Mission Monitor function and task daily record function, interfaceization support task data configuration operation, support each of task configuration data
Class is inquired.The consistency of data in task physical library and task queue is supported in task dispatching zone, and task dispatcher is to task team
The meeting trigger condition in row of the task is scheduled.Task dispatcher passes through the Resource Scheduler scheduler task in scheduling of resource domain
Resource needed for executing, resource management platform application of the scheduling of resource domain into resource domains create task run unit, resourceoriented
Different types of resource in domain, such as Docker containers, virtual machine (VM, Openstack) shield the difference of underlying resource, pass through
Resource Scheduler is adapted to bottom mainstream managing computing resources interface, realizes task and the matched scheduling of computing resource.Scheduling of resource
Each TU task unit in domain is the running example of resource domains, establishes the connection with task queue by task performer, reports
Task status.
A kind of distributed task dispatching method disclosed by the invention, the two-stage tune detached using task scheduling and scheduling of resource
Degree pattern, wherein scheduling of resource can cope with virtualization or containerization resource management platform on x86 servers, support not
The scheduling of same type resource.
The disclosed distributed task dispatching method in order to which the present invention is further explained, referring to Fig. 2, the method includes with
Lower step：
S101：The goal task for meeting trigger condition is put into task queue by task trigger, and the goal task is
Time driving generic task and/or event-driven generic task；
It should be noted that by existing different types of task, such as batch, immediate processing mode in terms of timeliness；When
Between period in terms of driving, date Hour Minute Second unit time driving generic task；In terms of event-driven；Outside imports data, outer
Portion's service interface notification event driving task, unified integration are supported simultaneously at time driving generic task and event-driven generic task
Third party's task interface is docked.
For time driving generic task, it is specially to meet time triggered condition to meet trigger condition；For event-driven class
Task, it is specially to meet trigger conditions to meet trigger condition.
Task in task queue includes task identification, task priority, task run state, task names etc., wherein
Task run state includes：Wait for state, completion status and abnormality in operating status, operation.When task trigger will meet
When the goal task of trigger condition is put into task queue, the operating status of the goal task is to wait for operating status.
Task trigger is scanned task, and the goal task for meeting trigger condition is put into task queue.
S102：Task dispatcher obtains the goal task after the notification information for receiving the task queue publication
Task schedule information loads resource requirement configuration information corresponding with the goal task type, and is sent out to Resource Scheduler
Act the resource requirements request for carrying the resource requirement configuration information；
Subscribing relationship is pre-established between task queue and task dispatcher, when the task queue receives the target
When task, make an announcement information to the task dispatcher.
Task queue is according to preset subscription rules, when task queue changes, will real-time informing subscriber (i.e.
Task dispatcher).The pattern that task dispatcher actively goes inquiry is avoided, active inquiry, which needs to be spaced, constantly goes to scan, from
And reduce the task amount of task dispatcher.What the realization of subscribing relationship actually used is a subscription in redis memory banks
The function of making an announcement, many distributed important cores are all the similar this abilities used, are also equipped with the industry of this ability
Boundary's component of increasing income has such as zookeeper etc..
The task dispatcher traverses after the notification information for receiving the task queue publication in the task queue
Task data；
According to the operating status in the task data in the task queue, it is to wait for operating status that batch, which obtains operating status,
Goal task task schedule information；
Or according to the task priority in the task data in the task queue, obtain the target of super objective priority
The task schedule information of task.
It should be noted that when task dispatcher is scheduled goal task, the operating status of the goal task
It is set as state in operation.
Each type of task has preset corresponding resource requirement configuration information, including：CPU numbers, are deposited memory size
Store up size etc..
Task dispatcher initiates to carry the resource requirements request of the resource requirement configuration information, Shen to Resource Scheduler
The task run unit to match with resource requirement configuration information please be create, if Resource Scheduler is because of inadequate resource, resource tune
When the reasons such as degree is abnormal return to exception information, task dispatcher supports retry mechanism, and (such as task execution fails, according to reporting heartbeat
Information carries out rescheduling execution), manual intervention mechanism.Support clean-up task queue task managing detailed catalogue under abnormal conditions, really
Protect task data consistency.
It should also be noted that, the resource requirements request further includes the goal task mark and the goal task class
Type.
S103：Resource Scheduler carries the resource need according to the resource requirements request to resource management platform initiation
Ask the establishing resource unit application of configuration information；
Resource Scheduler initiates establishing resource unit application to resource management platform, and Resource Scheduler is supported different types of
Resource type, such as Docker containerization resource management platform Mesos, Kubernates, virtualization resource management platform
Openstack, vmware etc..
S104：Resource management platform is according to the resource requirement configuration information in the establishing resource unit application, wound
Build the task run unit of the goal task；
Resource management platform creates the task run unit of respective resources demand, such as according to establishing resource unit application
Docker container instances, virtual machine instance etc., while being identified according to goal task, start and applies image program accordingly.
S105：Task performer controls the task run unit localization and runs the goal task.
The task performer establishes registration connection with the task queue；
The task performer controls task identification of the task run unit according to the goal task, from described
The task data of the goal task is obtained in business queue, and the goal task is run according to task data localization.
Preferably, when task run unit localization runs the goal task completion, the task performer
Task status update application is initiated to the task queue, the operating status of goal task described in the task queue is arranged
For completion status；
The goal task is run in task run unit localization there are when abnormal conditions, the task performer
Task status update application is initiated to the task queue, the operating status of goal task described in the task queue is arranged
For abnormality.
The task dispatcher carries out periodic scan to the state of each task in the task queue；
For the task of completion status, the task dispatcher carries completion status to Resource Scheduler initiation and appoints
The resource unit of business mark destroys request, and the Resource Scheduler initiates TU task unit destruction to the resource management platform asks
It asks, the resource management platform destroys corresponding task running unit, and discharges the localization configuration money of corresponding task running unit
Source；
For the task of abnormality, the task dispatcher startup separator treatment mechanism retries task scheduling or opens automatically
Moving work Intervention Mechanism.
Wherein, the task dispatcher startup separator treatment mechanism retries task scheduling automatically, including：
When failure, which retries configured number, is not up to the upper limit, the task dispatcher initiates to carry to the Resource Scheduler
There is the resource release request of abnormality task identification；
The Resource Scheduler initiates TU task unit to the resource management platform and destroys request；
The resource management platform destroys corresponding task running unit, the localization configuration of release corresponding task running unit
Resource, and start other and destroyed TU task unit of the TU task unit with same asset configuration, operation again has described different
The goal task of normal state task mark.
Specifically, referring to Fig. 3, Fig. 3 is failure handling mechanisms schematic diagram disclosed by the embodiments of the present invention.
A. it can register and report heartbeat message, task monitoring module can be spaced and sweep to task queue in task performer operation
Retouch task performer state；
B. if task heartbeat is because task performer operation exception or network problem cause state not update,
Configured number is retried according to failure, calls the release of Resource Scheduler application abnormal task resource, when avoiding fault self-recovery, task is held
Row device causes task data to reprocess because the case where network reason " seemingly-dead " occurs；
C. Resource Scheduler can initiate TU task unit to resource management platform and destroy request；
D. resource management platform is destroyed according to TU task unit and is asked, and destroys actual container, computing resource is discharged, is opened simultaneously
It moves identical TU task unit and then runs the task, realize task failure self-healing.
A kind of distributed task dispatching method disclosed in the present embodiment, time driving class is summarized as by different types of task
Different types of task scheduling mode is integrated into unified method for scheduling task by task or event-driven generic task.Task scheduling
Device loads the corresponding resource requirement configuration information of goal task type, and carrying the resource to Resource Scheduler initiation need to
The resource requirements request of configuration information, Resource Scheduler is asked to initiate to carry to resource management platform according to the resource requirements request
There is the establishing resource unit application of the resource requirement configuration information；Resource management platform is according to the establishing resource unit application
In the resource requirement configuration information, create the task run unit of the goal task.Integrate unified resource scheduling, resource
Scheduler shields the difference of bottom different type resource, and such as containerization, virtualization resource, resource management platform is according to different task
Resource requirement configuration information create different task run units, parallelization handles task, and maximized to promote task concurrent
Process performance supports the horizontal and vertical dilatation ability of task.
Based on a kind of distributed task dispatching method disclosed in above-described embodiment, referring to Fig. 4, the present embodiment corresponds to disclosure
A kind of distributed task dispatching system, specifically includes：Task trigger 401, task dispatcher 402, Resource Scheduler 403,
Resource management platform 404 and task performer 405；
The task trigger 401, for the goal task for meeting trigger condition to be put into task queue, the target
Task is time driving generic task and/or event-driven generic task；
The task dispatcher 402, for obtaining the mesh after the notification information for receiving the task queue publication
The task schedule information of mark task, loads corresponding with goal task type resource requirement configuration information, and to described
Resource Scheduler 403 initiates to carry the resource requirements request of the resource requirement configuration information；
The Resource Scheduler 403, for being taken to the initiation of the resource management platform 404 according to the resource requirements request
Establishing resource unit application with the resource requirement configuration information；
The resource management platform 404, for being configured according to the resource requirement in the establishing resource unit application
Information creates the task run unit of the goal task；
The task performer 405 controls the task run unit localization and runs the goal task.
Preferably,
The task performer 405 is additionally operable to run the goal task completion in task run unit localization
When, task status update application is initiated to the task queue, by the operating status of goal task described in the task queue
It is set as completion status；The goal task is run in task run unit localization there are when abnormal conditions, to described
Task status update application is initiated in task queue, and the operating status of goal task described in the task queue is set as abnormal
State.
Preferably,
The task dispatcher 402 is additionally operable to carry out periodic scan to the state of each task in the task queue；
For the task of completion status, the task dispatcher 402 is additionally operable to take to the initiation of the Resource Scheduler 403
Resource unit with completion status task identification destroys request, and the Resource Scheduler 403 is to the resource management platform 404
It initiates TU task unit and destroys request, the resource management platform 404 destroys corresponding task running unit, and discharges corresponding task fortune
The localization of row unit configures resource；
For the task of abnormality, the task dispatcher 402, be additionally operable to startup separator treatment mechanism retry automatically appoint
Business scheduling starts manual intervention mechanism.
A kind of distributed task dispatching system disclosed in the present embodiment, time driving class is summarized as by different types of task
Different types of task scheduling mode is integrated into unified method for scheduling task by task or event-driven generic task.Task scheduling
Device loads the corresponding resource requirement configuration information of goal task type, and carrying the resource to Resource Scheduler initiation need to
The resource requirements request of configuration information, Resource Scheduler is asked to initiate to carry to resource management platform according to the resource requirements request
There is the establishing resource unit application of the resource requirement configuration information；Resource management platform is according to the establishing resource unit application
In the resource requirement configuration information, create the task run unit of the goal task.Integrate unified resource scheduling, resource
Scheduler shields the difference of bottom different type resource, and such as containerization, virtualization resource, resource management platform is according to different task
Resource requirement configuration information create different task run units, parallelization handles task, and maximized to promote task concurrent
Process performance supports the horizontal and vertical dilatation ability of task.
The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present invention.
Various modifications to these embodiments will be apparent to those skilled in the art, as defined herein
General Principle can be realized in other embodiments without departing from the spirit or scope of the present invention.Therefore, of the invention
It is not intended to be limited to the embodiments shown herein, and is to fit to and the principles and novel features disclosed herein phase one
The widest range caused.
Claims (10)
1. a kind of distributed task dispatching method, which is characterized in that including：
The goal task for meeting trigger condition is put into task queue by task trigger, and the goal task is time driving class
Task and/or event-driven generic task；
Task dispatcher obtains the task scheduling of the goal task after the notification information for receiving the task queue publication
Information loads resource requirement configuration information corresponding with the goal task type, and is carried to Resource Scheduler initiation
The resource requirements request of the resource requirement configuration information；
Resource Scheduler carries the resource requirement to resource management platform initiation according to the resource requirements request and matches confidence
The establishing resource unit application of breath；
Resource management platform creates the target according to the resource requirement configuration information in the establishing resource unit application
The task run unit of task；
Task performer controls the task run unit localization and runs the goal task.
2. according to the method described in claim 1, it is characterized in that, between the task queue and the task dispatcher in advance
Establish subscribing relationship；The method further includes：
When the task queue receives the goal task, make an announcement information to the task dispatcher.
3. according to the method described in claim 1, it is characterized in that, the task dispatcher is receiving the task queue hair
The task schedule information of the goal task is obtained after the notification information of cloth, including：
The task dispatcher traverses appointing in the task queue after the notification information for receiving the task queue publication
Business data；
According to the operating status in the task data in the task queue, it is to wait for the mesh of operating status that batch, which obtains operating status,
The task schedule information of mark task；
Or according to the task priority in the task data in the task queue, obtain the goal task of super objective priority
Task schedule information.
4. according to the method described in claim 1, it is characterized in that, the resource requirements request and establishing resource unit Shen
Please all include the task identification of the goal task；The task performer controls the task run unit localization operation institute
Goal task is stated, including：
The task performer establishes registration connection with the task queue；
The task performer controls task identification of the task run unit according to the goal task, from the task team
The task data of the goal task is obtained in row, and the goal task is run according to task data localization.
5. according to the method described in claim 1, it is characterized in that, the method further includes：
When task run unit localization runs the goal task and completes, the task performer is to the task team
Row initiate task status update application, set the operating status of goal task described in the task queue to completion status；
The goal task is run in task run unit localization there are when abnormal conditions, and the task performer is to institute
It states task queue and initiates task status update application, the operating status of goal task described in the task queue is set as different
Normal state.
6. according to the method described in claim 5, it is characterized in that, the method further includes：
The task dispatcher carries out periodic scan to the state of each task in the task queue；
For the task of completion status, the task dispatcher carries completion status task mark to Resource Scheduler initiation
The resource unit of knowledge destroys request, and the Resource Scheduler initiates TU task unit to the resource management platform and destroys request, institute
It states resource management platform and destroys corresponding task running unit, and discharge the localization configuration resource of corresponding task running unit；
For the task of abnormality, the task dispatcher startup separator treatment mechanism retries task scheduling or starts people automatically
Work Intervention Mechanism.
7. according to the method described in claim 6, it is characterized in that, the task dispatcher startup separator treatment mechanism weighs automatically
Task scheduling is tried, including：
When failure, which retries configured number, is not up to the upper limit, the task dispatcher carries different to Resource Scheduler initiation
The resource release request of normal state task mark；
The Resource Scheduler initiates TU task unit to the resource management platform and destroys request；
The resource management platform destroys corresponding task running unit, the localization configuration money of release corresponding task running unit
Source, and start other and destroyed TU task unit of the TU task unit with same asset configuration, operation again has the exception
The goal task of state task mark.
8. a kind of distributed task dispatching system, which is characterized in that including：Task trigger, task dispatcher, scheduling of resource
Device, resource management platform and task performer；
The task trigger, for the goal task for meeting trigger condition to be put into task queue, the goal task is
Time driving generic task and/or event-driven generic task；
The task dispatcher, for obtaining the goal task after the notification information for receiving the task queue publication
Task schedule information, loads corresponding with goal task type resource requirement configuration information, and to the scheduling of resource
Device initiates to carry the resource requirements request of the resource requirement configuration information；
The Resource Scheduler, for carrying the money to resource management platform initiation according to the resource requirements request
The establishing resource unit application of source demand configuration information；
The resource management platform, for according to the resource requirement configuration information in the establishing resource unit application, wound
Build the task run unit of the goal task；
The task performer controls the task run unit localization and runs the goal task.
9. system according to claim 8, which is characterized in that
The task performer is additionally operable to when task run unit localization runs the goal task completion, to institute
It states task queue and initiates task status update application, set the operating status of goal task described in the task queue to
At state；The goal task is run there are when abnormal conditions, to the task queue in task run unit localization
Task status update application is initiated, sets the operating status of goal task described in the task queue to abnormality.
10. system according to claim 9, which is characterized in that
The task dispatcher is additionally operable to carry out periodic scan to the state of each task in the task queue；
For the task of completion status, the task dispatcher is additionally operable to carry to Resource Scheduler initiation and completes shape
The resource unit of state task identification destroys request, and the Resource Scheduler is initiated TU task unit to the resource management platform and destroyed
Request, the resource management platform destroy corresponding task running unit, and discharge the localization configuration of corresponding task running unit
Resource；
For the task of abnormality, the task dispatcher is additionally operable to startup separator treatment mechanism and retries task scheduling automatically
Or start manual intervention mechanism.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810575719.6A CN108762910A (en) | 2018-06-06 | 2018-06-06 | A kind of distributed task dispatching method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810575719.6A CN108762910A (en) | 2018-06-06 | 2018-06-06 | A kind of distributed task dispatching method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108762910A true CN108762910A (en) | 2018-11-06 |
Family
ID=63999778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810575719.6A Pending CN108762910A (en) | 2018-06-06 | 2018-06-06 | A kind of distributed task dispatching method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108762910A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109450687A (en) * | 2018-11-14 | 2019-03-08 | 沈文策 | A kind of data distributing method, device, electronic equipment and storage medium |
| CN110442446A (en) * | 2019-06-29 | 2019-11-12 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | The method of processing high-speed digital signal data flow in real time |
| CN110688206A (en) * | 2019-09-05 | 2020-01-14 | 平安科技（深圳）有限公司 | Timing task scheduling method and device, computer equipment and storage medium |
| CN110795225A (en) * | 2019-10-31 | 2020-02-14 | 华夏银行股份有限公司 | Method and system for executing scheduling task |
| CN111061548A (en) * | 2019-11-29 | 2020-04-24 | 西安四叶草信息技术有限公司 | Safety scanning task scheduling method and scheduler |
| CN111626835A (en) * | 2020-04-27 | 2020-09-04 | 口碑(上海)信息技术有限公司 | Task configuration method, device, system, storage medium and computer equipment |
| CN111669357A (en) * | 2019-03-08 | 2020-09-15 | 厦门网宿有限公司 | Method for batch processing of haproxy network isolation space and haproxy proxy server |
| WO2021057514A1 (en) * | 2019-09-24 | 2021-04-01 | 中兴通讯股份有限公司 | Task scheduling method and apparatus, computer device, and computer readable medium |
| CN113119127A (en) * | 2021-04-20 | 2021-07-16 | 大连宏燊软件技术有限公司 | Task scheduling method and system for RPA robot |
| CN114726869A (en) * | 2022-04-02 | 2022-07-08 | 中国建设银行股份有限公司 | Resource management method and device, storage medium and electronic equipment |
| CN114896050A (en) * | 2022-07-12 | 2022-08-12 | 北京云枢创新软件技术有限公司 | Task scheduling method and system based on cluster resources |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103810023A (en) * | 2014-03-06 | 2014-05-21 | 中国科学院信息工程研究所 | Distributed application intelligent deployment method and system in cloud platform |
| CN105159767A (en) * | 2015-09-06 | 2015-12-16 | 北京京东尚科信息技术有限公司 | Method and device for realizing distributed scheduling on the basis of zookeeper |
| CN106980543A (en) * | 2017-04-05 | 2017-07-25 | 福建智恒软件科技有限公司 | The distributed task dispatching method and device triggered based on event |
| CN107168779A (en) * | 2017-03-31 | 2017-09-15 | 咪咕互动娱乐有限公司 | A kind of task management method and system |
| CN107656805A (en) * | 2017-10-20 | 2018-02-02 | 广东电网有限责任公司电力调度控制中心 | A kind of electric power data job scheduling method based on Hadoop platform |
| CN107766129A (en) * | 2016-08-17 | 2018-03-06 | 北京金山云网络技术有限公司 | A kind of task processing method, apparatus and system |
| CN107783818A (en) * | 2017-10-13 | 2018-03-09 | 北京百度网讯科技有限公司 | Deep learning task processing method, device, equipment and storage medium |
| CN108121599A (en) * | 2016-11-30 | 2018-06-05 | 杭州海康威视数字技术股份有限公司 | A kind of method for managing resource, apparatus and system |
-
2018
- 2018-06-06 CN CN201810575719.6A patent/CN108762910A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103810023A (en) * | 2014-03-06 | 2014-05-21 | 中国科学院信息工程研究所 | Distributed application intelligent deployment method and system in cloud platform |
| CN105159767A (en) * | 2015-09-06 | 2015-12-16 | 北京京东尚科信息技术有限公司 | Method and device for realizing distributed scheduling on the basis of zookeeper |
| CN107766129A (en) * | 2016-08-17 | 2018-03-06 | 北京金山云网络技术有限公司 | A kind of task processing method, apparatus and system |
| CN108121599A (en) * | 2016-11-30 | 2018-06-05 | 杭州海康威视数字技术股份有限公司 | A kind of method for managing resource, apparatus and system |
| CN107168779A (en) * | 2017-03-31 | 2017-09-15 | 咪咕互动娱乐有限公司 | A kind of task management method and system |
| CN106980543A (en) * | 2017-04-05 | 2017-07-25 | 福建智恒软件科技有限公司 | The distributed task dispatching method and device triggered based on event |
| CN107783818A (en) * | 2017-10-13 | 2018-03-09 | 北京百度网讯科技有限公司 | Deep learning task processing method, device, equipment and storage medium |
| CN107656805A (en) * | 2017-10-20 | 2018-02-02 | 广东电网有限责任公司电力调度控制中心 | A kind of electric power data job scheduling method based on Hadoop platform |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109450687A (en) * | 2018-11-14 | 2019-03-08 | 沈文策 | A kind of data distributing method, device, electronic equipment and storage medium |
| CN111669357B (en) * | 2019-03-08 | 2023-03-24 | 厦门网宿有限公司 | Method for batch processing of haproxy network isolation space and haproxy proxy server |
| CN111669357A (en) * | 2019-03-08 | 2020-09-15 | 厦门网宿有限公司 | Method for batch processing of haproxy network isolation space and haproxy proxy server |
| CN110442446A (en) * | 2019-06-29 | 2019-11-12 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | The method of processing high-speed digital signal data flow in real time |
| CN110442446B (en) * | 2019-06-29 | 2022-12-13 | 西南电子技术研究所（中国电子科技集团公司第十研究所） | Method for real-time processing high-speed digital signal data stream |
| CN110688206A (en) * | 2019-09-05 | 2020-01-14 | 平安科技（深圳）有限公司 | Timing task scheduling method and device, computer equipment and storage medium |
| WO2021057514A1 (en) * | 2019-09-24 | 2021-04-01 | 中兴通讯股份有限公司 | Task scheduling method and apparatus, computer device, and computer readable medium |
| CN110795225A (en) * | 2019-10-31 | 2020-02-14 | 华夏银行股份有限公司 | Method and system for executing scheduling task |
| CN111061548A (en) * | 2019-11-29 | 2020-04-24 | 西安四叶草信息技术有限公司 | Safety scanning task scheduling method and scheduler |
| CN111061548B (en) * | 2019-11-29 | 2024-04-05 | 西安四叶草信息技术有限公司 | Safe scanning task scheduling method and scheduler |
| CN111626835A (en) * | 2020-04-27 | 2020-09-04 | 口碑(上海)信息技术有限公司 | Task configuration method, device, system, storage medium and computer equipment |
| CN111626835B (en) * | 2020-04-27 | 2024-02-02 | 口碑(上海)信息技术有限公司 | Task configuration method, device, system, storage medium and computer equipment |
| CN113119127A (en) * | 2021-04-20 | 2021-07-16 | 大连宏燊软件技术有限公司 | Task scheduling method and system for RPA robot |
| CN114726869A (en) * | 2022-04-02 | 2022-07-08 | 中国建设银行股份有限公司 | Resource management method and device, storage medium and electronic equipment |
| CN114896050A (en) * | 2022-07-12 | 2022-08-12 | 北京云枢创新软件技术有限公司 | Task scheduling method and system based on cluster resources |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108762910A (en) | A kind of distributed task dispatching method and system | |
| CN103049321B (en) | Intelligent Task treating apparatus and intelligent task disposal route | |
| US8898620B2 (en) | System and method for application process automation over a computer network | |
| CN103037019B (en) | A kind of distributed data acquisition system and method based on cloud computing | |
| CN110247954A (en) | A kind of dispatching method and system of distributed task scheduling | |
| CN104917792B (en) | The cluster management method and system of democratic autonomy | |
| CN112379995A (en) | DAG-based unitized distributed scheduling system and method | |
| WO2019020514A1 (en) | Edge devices and associated networks utilising microservices | |
| JP2005527916A (en) | System and method for work list forecasting | |
| CN105095327A (en) | Distributed ELT system and scheduling method | |
| CN106815063B (en) | Control platform of automation equipment with multiple interaction channels | |
| CN108132837A (en) | A kind of distributed type assemblies dispatch system and method | |
| CN107168779A (en) | A kind of task management method and system | |
| CN108874549A (en) | resource multiplexing method, device, terminal and computer readable storage medium | |
| CN107656804B (en) | Process pool system and method | |
| CZ2003563A3 (en) | System for controlling a network-building element | |
| CN112910952B (en) | Distributed task scheduling method and device, storage medium and electronic device | |
| CN112948077A (en) | Batch processing method, device, equipment and storage medium | |
| Dadam et al. | The ADEPT WfMS project at the university of Ulm | |
| CN107196987A (en) | The switching method and server of a kind of pair of centring system | |
| CN108038008A (en) | Self-aided terminal framework | |
| US20160246648A1 (en) | Information technology resource planning | |
| CN106598726A (en) | Multi-task management system and distributed deployment method thereof | |
| CN112131188B (en) | Batch file distribution processing method and device | |
| EP1659521A1 (en) | A production scheduling method and system, related computer program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181106 |