CN107766136A - A kind of method of task cluster management and running - Google Patents
A kind of method of task cluster management and running Download PDFInfo
- Publication number
- CN107766136A CN107766136A CN201710922771.XA CN201710922771A CN107766136A CN 107766136 A CN107766136 A CN 107766136A CN 201710922771 A CN201710922771 A CN 201710922771A CN 107766136 A CN107766136 A CN 107766136A
- Authority
- CN
- China
- Prior art keywords
- task
- actuator
- control centre
- address
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Cardiology (AREA)
- Multi Processors (AREA)
- Debugging And Monitoring (AREA)
Abstract
A kind of method of task cluster management and running of the present invention, the server node for performing task scheduling is separated into control centre and actuator, the control centre includes scheduling pond, Quartz schedulers, readjustment service and dispatching log, schedule information is responsible for, dispatch request is sent according to the configuration of schedule information；The actuator includes server, tasks carrying queue, task object, the server is responsible for receiving the heartbeat detection information of control centre, task triggering information, actuator receives the mission bit stream of control centre by server, it is medium pending to be packaged into task object addition tasks carrying queue, reach and perform the time, task object starts to perform, and final task implementing result is returned into control centre by the way that the specified interface of control centre is asynchronous.Control centre of the present invention decouples with task, improves system availability and stability, while dispatch systematic function and be no longer limited by task.It is absorbed in due to task and performs operation, is developed and safeguard and be simpler and efficient.
Description
Technical field
The present invention relates to a kind of method of task cluster management and running.
Technical background
The situation of internet project access timed task is more and more, and quratz is presently the most maturation, using most extensive
Java task scheduling frameworks, the flexible of powerful configuration accounts for critical role in enterprise applies.It is big with Distributed Design
Scale is used, and timer is distributed and imperative.Quartz is in fact also to support aggregated structure in itself.Such as Fig. 1 institutes
Show, under quartz cluster modes, each server node is immediately synchronous all task statuses that will and be currently running
To central database, when any server node triggers calling task again, analysed whether from central database different
Server node be currently running same task (task of same timing point i.e. of the same name), it is other if there is same task
Server node on the situation run, then current server node ignores the task of this operation.
Following deficiency be present in above-mentioned quartz cluster modes：
1st, scheduling logic and task coupling be in same project, as task quantity gradually increases, scheduling logic is gradual
In the case of exacerbation, business will be greatly limited by by dispatching the performance of system；
2nd, needing persistence business (QuartzJobBean), system is invasive quite tight into underlying services database table
Weight.
The content of the invention
It is an object of the invention to provide a kind of method of task cluster management and running, and scheduling and task are separated, will
All tasks have carried out distributed management, during task scheduling, task can be allowed to be efficiently performed by configuration, multiple tasks
It will not be interfered during execution.
A kind of method of task cluster management and running of the present invention, the server node for performing task scheduling is separated into scheduling
Center and actuator, the control centre include scheduling pond, Quartz schedulers, readjustment service and dispatching log, are responsible for adjusting
Information is spent, dispatch request is sent according to the configuration of schedule information；The actuator includes server, tasks carrying queue, task pair
As the server is responsible for receiving the heartbeat detection information of control centre, task triggering information, and actuator receives tune by server
The mission bit stream at degree center, be packaged into task object add tasks carrying queue it is medium pending, reach perform the time, task pair
As starting to perform, and by final task implementing result control centre is returned to by the way that the specified interface of control centre is asynchronous.
Described task scheduling comprises the following specific steps that：
Actuator registered address in first configuration schedules central dispatching pond and tasks carrying knot before step 1, actuator start
Fruit feedback addresses；
When step 2, actuator start, the IP of each actuator, port information are registered to by scheduling by the registered address
Center, actuator address list is formed in the scheduling pond of control centre；
Step 3, control centre's interval threshold time sweep actuator address list, and sent out one by one to actuator address list
Whether cloth heartbeat packet, detection actuator can use, and actuator uploads the task information list of oneself in heartbeat packet, the mission bit stream
The JobHandler in pond is correspondingly dispatched in list, and same actuator can configure multiple JobHandler, to perform different appoint
Business；Control centre updates the task information list of the actuator by the task information list of the upload, passes through above-mentioned heartbeat
Journey, control centre save the address of all available actuators and the task information list of each actuator schedulable；For
The actuator of detection failure removes from actuator address list in preset heart beat cycle, and the removed actuator excludes
Control centre can be reregistered with after exception.
Quartz tasks are added in the database that step 4, user are shared by Quartz client end APs I to control centre to appoint
Business information；
Step 5, control centre travel through actuator list after detecting Quartz some task triggerings, and detect actuator
Task information list, if the task in the task information list of some actuator, the address come up is registered according to actuator,
The execute instruction of the task, and Asynchronous Reception implementing result are sent to the actuator, an actuator is changed if failure is performed
Retry, until tasks carrying success；
Step 6, actuator receive the mission bit stream of control centre, are packaged into task object, add execution queue wait and hold
OK；
Step 7, task object etc. are to be achieved to be performed the time and performs, and final task implementing result is passed through into control centre
Specified interface asynchronous return to control centre.
Control centre decouples with task in the present invention, improves system availability and stability, while dispatch systematic function
It is no longer limited by task.Because task is absorbed in the operation such as execution of task, develops and safeguard and be simpler and efficient.
Brief description of the drawings
Fig. 1 is the schematic diagram of traditional scheduling and tasks carrying；
Fig. 2 is the schematic diagram that present invention scheduling separates with task.
The present invention is further described below in conjunction with drawings and examples.
Embodiment
As shown in figure 1, a kind of method of task cluster management and running of the present invention, will perform the server node of task scheduling
Control centre and actuator are separated into, the control centre includes scheduling pond, Quartz schedulers, readjustment service and dispatching log,
Schedule information is responsible for, sends dispatch request according to the configuration of schedule information, itself does not undertake service code；The actuator bag
Server (can be the preferred servers such as jetty, tomcat), tasks carrying queue, task object are included, the server is responsible for connecing
Heartbeat detection information, the task triggering information of control centre are received, actuator receives the mission bit stream of control centre by server,
It is pending to be packaged into task object, addition tasks carrying queue etc., reaches execution time task object and starts to perform, and will be final
The asynchronous specific step for returning to control centre, including following task scheduling of specified interface that task action result passes through control centre
Suddenly：
Actuator registered address in first configuration schedules central dispatching pond and tasks carrying knot before step 1, actuator start
Fruit feedback addresses；
When step 2, actuator start, the IP of each actuator, port information are registered to by scheduling by the registered address
Center, actuator address list is formed in the scheduling pond of control centre；
5 seconds step 3, control centre interval scanning actuator address lists, and the heart is issued one by one to actuator address list
Bag is jumped, whether detection actuator can use, and actuator uploads the task information list of oneself in heartbeat packet, the task information list
JobHandler in corresponding diagram 2, same actuator can configure multiple JobHandler, to perform different tasks；Scheduling
The task information list of the actuator is updated centrally through the task information list of the upload, passes through above-mentioned Heart Beat procedure, scheduling
Center saves the address of all available actuators and the task information list of each actuator schedulable；For in 3 heartbeats
The actuator of detection failure removes from actuator address list in cycle, and the removed actuator can weigh after excluding exception
New registration is to control centre.
Quartz tasks are added in the database that step 4, user are shared by Quartz client end APs I to control centre to appoint
Business information；
Step 5, control centre travel through actuator list after detecting Quartz tasks (such as task A) triggering, and detect and hold
The task information list of row device, if there is task A in the task information list of some actuator, come up according to actuator registration
Address, task A execute instruction, and Asynchronous Reception implementing result are sent to the actuator, one is changed if failure is performed
Actuator retries, until tasks carrying success；
Step 6, actuator receive the mission bit stream of control centre, are packaged into task object, add execution queue wait and hold
OK；
Step 7, task object etc. are to be achieved to be performed the time and performs, and final task implementing result is passed through into control centre
Specified interface asynchronous return to control centre.
It is described above, only it is present pre-ferred embodiments, is not intended to limit the scope of the present invention, therefore
Any subtle modifications, equivalent variations and modifications that every technical spirit according to the present invention is made to above example, still belong to
In the range of technical solution of the present invention.
Claims (2)
- A kind of 1. method of task cluster management and running, it is characterised in that：The server node for performing task scheduling is separated into Control centre and actuator, the control centre include scheduling pond, Quartz schedulers, readjustment service and dispatching log, are responsible for pipe Schedule information is managed, dispatch request is sent according to the configuration of schedule information；The actuator includes server, tasks carrying queue, appointed Business object, the server are responsible for receiving the heartbeat detection information of control centre, task triggering information, and actuator is received by server To the mission bit stream of control centre, being packaged into task object, to add tasks carrying queue medium pending, reaches and performs the time, appoints Business object starts to perform, and final task implementing result is returned into control centre by the way that the specified interface of control centre is asynchronous.
- 2. the method for a kind of task cluster management and running according to claim 1, it is characterised in that task scheduling is included such as Lower specific steps：Actuator registered address and task action result before step 1, actuator start in first configuration schedules central dispatching pond is anti- Present address；When step 2, actuator start, the IP of each actuator, port information are registered to by control centre by the registered address, Actuator address list is formed in the scheduling pond of control centre；Step 3, control centre's interval threshold time sweep actuator address list, and the heart is issued one by one to actuator address list Bag is jumped, whether detection actuator can use, and actuator uploads the task information list of oneself in heartbeat packet, the task information list JobHandler in corresponding scheduling pond, same actuator can configure multiple JobHandler, to perform different tasks；Adjust Degree updates the task information list of the actuator centrally through the task information list of the upload, by above-mentioned Heart Beat procedure, adjusts Degree center saves the address of all available actuators and the task information list of each actuator schedulable；For preset The actuator of detection failure removes from actuator address list in heart beat cycle, after the removed actuator excludes exception Control centre can be reregistered with.Quartz jobs are added in the database that step 4, user are shared by Quartz client end APs I to control centre to believe Breath；Step 5, control centre travel through actuator list after detecting Quartz some task triggerings, and detect the task of actuator Information list, if the task in the task information list of some actuator, the address come up is registered according to actuator, to this Actuator sends the execute instruction of the task, and Asynchronous Reception implementing result, and changing an actuator if failure is performed retries, Until tasks carrying success；Step 6, actuator receive the mission bit stream of control centre, are packaged into task object, and it is pending to add execution queue etc.；Step 7, task object etc. are to be achieved to be performed the time and performs, and the finger that final task implementing result is passed through into control centre Stationary interface is asynchronous to return to control centre.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710922771.XA CN107766136A (en) | 2017-09-30 | 2017-09-30 | A kind of method of task cluster management and running |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710922771.XA CN107766136A (en) | 2017-09-30 | 2017-09-30 | A kind of method of task cluster management and running |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107766136A true CN107766136A (en) | 2018-03-06 |
Family
ID=61267320
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710922771.XA Pending CN107766136A (en) | 2017-09-30 | 2017-09-30 | A kind of method of task cluster management and running |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107766136A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108681777A (en) * | 2018-05-07 | 2018-10-19 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of the machine learning program operation based on distributed system |
| CN108900379A (en) * | 2018-07-09 | 2018-11-27 | 广东神马搜索科技有限公司 | Distributed network business scheduling method, calculates equipment and storage medium at device |
| CN108958915A (en) * | 2018-06-28 | 2018-12-07 | 中国建设银行股份有限公司 | Timed task executes method and device |
| CN109582446A (en) * | 2018-10-11 | 2019-04-05 | 中国建设银行股份有限公司 | Quasi real time asynchronous batch processing system, method, apparatus and storage medium |
| CN109905459A (en) * | 2019-01-16 | 2019-06-18 | 平安科技（深圳）有限公司 | A kind of data transmission method and device |
| CN110764884A (en) * | 2018-07-27 | 2020-02-07 | 北京国双科技有限公司 | Task scheduling method and system and task system |
| CN111143044A (en) * | 2019-12-10 | 2020-05-12 | 浙江力石科技股份有限公司 | Task scheduling management system, method, device and storage medium thereof |
| CN111459639A (en) * | 2020-04-03 | 2020-07-28 | 杭州趣维科技有限公司 | Distributed task management platform and method supporting global multi-machine-room deployment |
| CN111782360A (en) * | 2020-06-28 | 2020-10-16 | 中国工商银行股份有限公司 | Distributed task scheduling method and device |
| CN111966474A (en) * | 2020-08-11 | 2020-11-20 | 武汉遂意语联网络科技有限公司 | Method and device for controlling timing task |
| CN113590347A (en) * | 2021-07-09 | 2021-11-02 | 平安国际智慧城市科技股份有限公司 | Remote task regulation and control method, device, equipment and storage medium |
| CN113641480A (en) * | 2021-08-27 | 2021-11-12 | 四川中电启明星信息技术有限公司 | Task scheduling system and method based on Kubernetes cluster |
| CN113867911A (en) * | 2021-09-26 | 2021-12-31 | 聚好看科技股份有限公司 | Task scheduling method and device and micro-service system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103279385A (en) * | 2013-06-01 | 2013-09-04 | 北京华胜天成科技股份有限公司 | Method and system for scheduling cluster tasks in cloud computing environment |
| CN104750549A (en) * | 2015-04-13 | 2015-07-01 | 飞狐信息技术（天津）有限公司 | Computational task processing device, method and system |
| CN105094961A (en) * | 2015-08-11 | 2015-11-25 | 北京思特奇信息技术股份有限公司 | Task scheduling management system based on quartz frame and method thereof |
| CN106648845A (en) * | 2016-09-23 | 2017-05-10 | 努比亚技术有限公司 | Distributed type task scheduling system and method |
| CN106817408A (en) * | 2016-12-27 | 2017-06-09 | 中国银联股份有限公司 | A kind of distributed server cluster dispatching method and device |
-
2017
- 2017-09-30 CN CN201710922771.XA patent/CN107766136A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103279385A (en) * | 2013-06-01 | 2013-09-04 | 北京华胜天成科技股份有限公司 | Method and system for scheduling cluster tasks in cloud computing environment |
| CN104750549A (en) * | 2015-04-13 | 2015-07-01 | 飞狐信息技术（天津）有限公司 | Computational task processing device, method and system |
| CN105094961A (en) * | 2015-08-11 | 2015-11-25 | 北京思特奇信息技术股份有限公司 | Task scheduling management system based on quartz frame and method thereof |
| CN106648845A (en) * | 2016-09-23 | 2017-05-10 | 努比亚技术有限公司 | Distributed type task scheduling system and method |
| CN106817408A (en) * | 2016-12-27 | 2017-06-09 | 中国银联股份有限公司 | A kind of distributed server cluster dispatching method and device |
Non-Patent Citations (3)
| Title |
|---|
| 许雪里: "XXL-JOB官方文档", 《GITHUB》 * |
| 许雪里: "分布式任务调度平台xxl-job", 《微信》 * |
| 许雪里: "分布式任务调度平台XXL-JOB（最新版本V1.4.0）", 《GITHUB》 * |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108681777A (en) * | 2018-05-07 | 2018-10-19 | 北京京东尚科信息技术有限公司 | A kind of method and apparatus of the machine learning program operation based on distributed system |
| CN108681777B (en) * | 2018-05-07 | 2021-07-20 | 北京京东尚科信息技术有限公司 | Method and device for running machine learning program based on distributed system |
| CN108958915A (en) * | 2018-06-28 | 2018-12-07 | 中国建设银行股份有限公司 | Timed task executes method and device |
| CN108900379A (en) * | 2018-07-09 | 2018-11-27 | 广东神马搜索科技有限公司 | Distributed network business scheduling method, calculates equipment and storage medium at device |
| CN108900379B (en) * | 2018-07-09 | 2020-12-29 | 阿里巴巴（中国）有限公司 | Distributed network service scheduling method, device, computing equipment and storage medium |
| CN110764884A (en) * | 2018-07-27 | 2020-02-07 | 北京国双科技有限公司 | Task scheduling method and system and task system |
| CN109582446A (en) * | 2018-10-11 | 2019-04-05 | 中国建设银行股份有限公司 | Quasi real time asynchronous batch processing system, method, apparatus and storage medium |
| CN109905459B (en) * | 2019-01-16 | 2022-06-28 | 平安科技（深圳）有限公司 | Data transmission method and device |
| CN109905459A (en) * | 2019-01-16 | 2019-06-18 | 平安科技（深圳）有限公司 | A kind of data transmission method and device |
| CN111143044A (en) * | 2019-12-10 | 2020-05-12 | 浙江力石科技股份有限公司 | Task scheduling management system, method, device and storage medium thereof |
| CN111143044B (en) * | 2019-12-10 | 2023-08-29 | 浙江力石科技股份有限公司 | Task scheduling management system, method, device and storage medium thereof |
| CN111459639A (en) * | 2020-04-03 | 2020-07-28 | 杭州趣维科技有限公司 | Distributed task management platform and method supporting global multi-machine-room deployment |
| CN111459639B (en) * | 2020-04-03 | 2023-10-20 | 杭州小影创新科技股份有限公司 | Distributed task management platform and method supporting global multi-machine room deployment |
| CN111782360A (en) * | 2020-06-28 | 2020-10-16 | 中国工商银行股份有限公司 | Distributed task scheduling method and device |
| CN111782360B (en) * | 2020-06-28 | 2023-08-11 | 中国工商银行股份有限公司 | Distributed task scheduling method and device |
| CN111966474A (en) * | 2020-08-11 | 2020-11-20 | 武汉遂意语联网络科技有限公司 | Method and device for controlling timing task |
| CN111966474B (en) * | 2020-08-11 | 2024-04-02 | 武汉遂意语联网络科技有限公司 | Method and device for controlling timing task |
| CN113590347A (en) * | 2021-07-09 | 2021-11-02 | 平安国际智慧城市科技股份有限公司 | Remote task regulation and control method, device, equipment and storage medium |
| CN113641480A (en) * | 2021-08-27 | 2021-11-12 | 四川中电启明星信息技术有限公司 | Task scheduling system and method based on Kubernetes cluster |
| CN113641480B (en) * | 2021-08-27 | 2023-12-15 | 四川中电启明星信息技术有限公司 | Task scheduling system and method based on Kubernetes cluster group |
| CN113867911A (en) * | 2021-09-26 | 2021-12-31 | 聚好看科技股份有限公司 | Task scheduling method and device and micro-service system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107766136A (en) | A kind of method of task cluster management and running | |
| CN112035228B (en) | Resource scheduling method and device | |
| JP6164747B2 (en) | Method for flow control in a collaborative environment and for reliable communication | |
| CN104935672B (en) | Load balancing service high availability implementation method and equipment | |
| EP1276050A2 (en) | Registry service for use in a distributed processing framework system and methods for implementing the same | |
| CN105791354A (en) | Job scheduling method and cloud scheduling server | |
| US20090037775A1 (en) | Messaging system based group joint debugging system and method | |
| WO2016045403A1 (en) | Runtime environment configuration method, device and system | |
| US9329858B2 (en) | Managing access to resource versions in shared computing environments | |
| CN111641515A (en) | VNF life cycle management method and device | |
| US20130283291A1 (en) | Managing Business Process Messaging | |
| JP2003188877A (en) | System for managing distributed network infrastructure services | |
| CN107483245A (en) | It is a kind of that realization device is actively gathered based on karaf_agent | |
| US9342369B2 (en) | System and method for increasing throughput of a PaaS system | |
| CN101262498A (en) | A distributed message call method and device | |
| CN113312165A (en) | Task processing method and device | |
| US20140101320A1 (en) | Information processing system, control method, management apparatus and computer-readable recording medium | |
| CN106060189B (en) | A kind of distribution domain name registration system | |
| US20100325198A1 (en) | System and Method for Providing User Context Support in a Native Transaction Platform | |
| CN107508787B (en) | A kind of task executing method, apparatus and system | |
| US7085853B2 (en) | System and method for a distributed shell in a java environment | |
| WO2016139621A1 (en) | Methods and systems for requesting access to limited service instances | |
| JP2009515474A (en) | Independent message store and message transport agent | |
| CA3061451A1 (en) | Task based service management platform | |
| CN106550002A (en) | A kind of paas clouds mandatory system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180306 |
|
| WD01 | Invention patent application deemed withdrawn after publication |