CN106980543A - The distributed task dispatching method and device triggered based on event - Google Patents

The distributed task dispatching method and device triggered based on event Download PDF

Info

Publication number
CN106980543A
CN106980543A CN201710217652.4A CN201710217652A CN106980543A CN 106980543 A CN106980543 A CN 106980543A CN 201710217652 A CN201710217652 A CN 201710217652A CN 106980543 A CN106980543 A CN 106980543A
Authority
CN
China
Prior art keywords
task
message
module
distributed
tasks carrying
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710217652.4A
Other languages
Chinese (zh)
Other versions
CN106980543B (en
Inventor
李贵生
谢远勇
汪钦堤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujian Zhiheng Environmental Engineering Technology Co ltd
Original Assignee
FUJIAN ZHIHENG SOFTWARE TECHNOLOGY Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by FUJIAN ZHIHENG SOFTWARE TECHNOLOGY Co Ltd filed Critical FUJIAN ZHIHENG SOFTWARE TECHNOLOGY Co Ltd
Priority to CN201710217652.4A priority Critical patent/CN106980543B/en
Publication of CN106980543A publication Critical patent/CN106980543A/en
Application granted granted Critical
Publication of CN106980543B publication Critical patent/CN106980543B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Debugging And Monitoring (AREA)
  • Telephonic Communication Services (AREA)

Abstract

The invention provides a kind of distributed task dispatching method and device triggered based on event, it is related to technical field of data processing, the task requests message in distributed task scheduling Queue module is received using listening mode by task scheduling engine module, and task requests message is parsed, obtain task registration message, whether the execution time and dependence for judging task registration message meet preset trigger condition, and in the case where meeting, tasks carrying message is obtained according to the task type of task registration message, tasks carrying message is sent to performing module, so that performing module is performed to tasks carrying message, and obtain execution state and execution journal.The present invention can lift the task scheduling ability of high concurrent, effectively save server resource, meet big data real-time task scheduling requirement.

Description

The distributed task dispatching method and device triggered based on event
Technical field
The present invention relates to technical field of data processing, more particularly, to the distributed task dispatching method triggered based on event And device.
Background technology
The crossing to merge of message technology and economic society has triggered data rapidly to increase, and data have turned into national basis and fought Omit resource.The national big data strategy of implementation is distinctly claimed in Chinese ' 13 ' development plans, propulsion data resources open is shared. What State Council printed and distributed《Promote big data developing activity outline》Also require that and promote government department's data sharing energetically, promote public steadily Data resource is opened altogether, big data infrastructure construction of making overall planning.
As the task scheduling system of one of big data system infrastructure construction, to the concurrent capability of task, task is adjusted Process performance of degree etc. proposes requirements at the higher level.And traditional task scheduling system is mainly adjusted using time-based timed task Degree method, no matter whether the dependence of tasks carrying meets, as long as meeting time point, must just trigger tasks carrying, hold Poll judges dependence condition again during row, so causes substantial amounts of server resource to waste, degraded performance, and can not meet The real-time task scheduling requirement of big data.
The content of the invention
In view of this, it is an object of the invention to provide the distributed task dispatching method and device triggered based on event, To lift the task scheduling ability of high concurrent, server resource is effectively saved, big data real-time task scheduling requirement is met.
In a first aspect, the embodiments of the invention provide a kind of distributed task dispatching method triggered based on event, wherein, Applied to task scheduling engine module, methods described includes:
Distributed task scheduling Queue module is monitored, monitoring information is obtained;
Judge the distributed task scheduling Queue module under using master-salve distributed message structure according to the monitoring information With the presence or absence of task requests message;
If it is present message call is sent to the distributed task scheduling Queue module, so that the distributed task scheduling team Row module sends the task requests message using the message queue mode of first in first out;
The task requests message that the distributed task scheduling Queue module is sent is received, and to the task requests message Parsed, obtain task registration message;
Whether the execution time and dependence for judging the task registration message meet preset trigger condition;
If met, tasks carrying message is obtained according to the task type of the task registration message;
The tasks carrying message is sent to performing module, so that the performing module is entered to the tasks carrying message Row is performed, and obtains execution state and execution journal.
With reference in a first aspect, the embodiments of the invention provide the possible embodiment of the first of first aspect, wherein, institute Stating method also includes:
Receive the execution journal that the performing module is sent and the execution for carrying out judging to obtain to the execution state As a result;
The implementing result and the execution journal are recorded;
The implementing result is sent to task AM access module by the distributed task scheduling Queue module, so that described Business AM access module judges whether tasks carrying completes according to the implementing result, and by the implementing result in the case of completion Feed back to task requests side.
With reference in a first aspect, the embodiments of the invention provide the possible embodiment of second of first aspect, wherein, institute Stating so that the performing module is performed to the tasks carrying message, and obtain execution state and execution journal includes:
So that the performing module is called according to the tasks carrying message to configuration processor, instruction message is obtained, And the tasks carrying message is performed according to the instruction message, obtain the execution state and the execution journal.
With reference in a first aspect, the embodiments of the invention provide the possible embodiment of the third of first aspect, wherein, institute Stating task type includes timed task, and the task type according to the task registration message obtains tasks carrying message package Include:
It is the timed task according to the task type of the task registration message, the task is obtained according to timing mode Perform message.
With reference in a first aspect, the embodiments of the invention provide the possible embodiment of the 4th of first aspect kind, wherein, institute Stating task type also includes event task, and the task type according to the task registration message obtains tasks carrying message package Include:
It is the event task according to the task type of the task registration message, the task is obtained according to event schema Perform message.
Second aspect, the embodiment of the present invention also provides a kind of distributed task dispatching device triggered based on event, wherein, Described device includes:
Task scheduling engine module, for monitoring distributed task scheduling Queue module, obtains monitoring information, according to institute State monitoring information and judge that the distributed task scheduling Queue module please with the presence or absence of task under using master-salve distributed message structure Message is sought, and in case of presence, message call is sent to the distributed task scheduling Queue module, described distributed is received The task requests message that business Queue module is sent, and the task requests message is parsed, obtain task registration and disappear Whether breath, the execution time and dependence for judging the task registration message meets preset trigger condition, and in the feelings met Under condition, tasks carrying message is obtained according to the task type of the task registration message, the tasks carrying message is sent to Performing module;
The distributed task scheduling Queue module, disappears for there are the task requests using master-salve distributed message structure Breath, and the task requests message is sent using the message queue mode of first in first out;
The performing module, for being performed to the tasks carrying message, and obtains execution state and execution journal.
With reference to second aspect, the embodiments of the invention provide the possible embodiment of the first of second aspect, wherein, institute Task scheduling engine module is stated to be additionally operable to:
Receive the execution journal that the performing module is sent and the execution for carrying out judging to obtain to the execution state As a result;
The implementing result and the execution journal are recorded;
The implementing result is sent to task AM access module by the distributed task scheduling Queue module, so that described Business AM access module judges whether tasks carrying completes according to the implementing result, and by the implementing result in the case of completion Feed back to task requests side.
With reference to second aspect, the embodiments of the invention provide the possible embodiment of second of second aspect, wherein, institute Task scheduling engine module is stated to be additionally operable to:
So that the performing module is called according to the tasks carrying message to configuration processor, instruction message is obtained, And the tasks carrying message is performed according to the instruction message, obtain the execution state and the execution journal.
With reference to second aspect, the embodiments of the invention provide the possible embodiment of the third of second aspect, wherein, institute Stating task type includes timed task, and the task scheduling engine module is additionally operable to:
It is the timed task according to the task type of the task registration message, the task is obtained according to timing mode Perform message.
With reference to second aspect, the embodiments of the invention provide the possible embodiment of the 4th of second aspect kind, wherein, institute Stating task type also includes event task, and the task scheduling engine module is additionally operable to:
It is the event task according to the task type of the task registration message, the task is obtained according to event schema Perform message.
The embodiment of the present invention brings following beneficial effect:
The distributed task dispatching method and device triggered based on event that the present invention is provided, passes through task scheduling engine mould Block is monitored distributed task scheduling Queue module, obtains monitoring information, and distributed task scheduling queue mould is judged according to monitoring information Block whether there is task requests message under using master-salve distributed message structure, and in case of presence, appoint to distribution Business Queue module sends message call, is appointed with making distributed task scheduling Queue module be sent using the message queue mode of first in first out Business request message, receives the task requests message that distributed task scheduling Queue module is sent, and task requests message is parsed, Task registration message is obtained, whether the execution time and dependence for judging task registration message meet preset trigger condition, and In the case where meeting, tasks carrying message is obtained according to the task type of task registration message, tasks carrying message is sent To performing module, so that performing module is performed to tasks carrying message, and execution state and execution journal are obtained.The present invention The task scheduling ability of high concurrent can be lifted, server resource is effectively saved, big data real-time task scheduling requirement is met.
Other features and advantages of the present invention will be illustrated in the following description, also, partly be become from specification Obtain it is clear that or being understood by implementing the present invention.The purpose of the present invention and other advantages are in specification, claims And specifically noted structure is realized and obtained in accompanying drawing.
To enable the above objects, features and advantages of the present invention to become apparent, preferred embodiment cited below particularly, and coordinate Appended accompanying drawing, is described in detail below.
Brief description of the drawings
, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical scheme of the prior art The accompanying drawing used required in embodiment or description of the prior art is briefly described, it should be apparent that, in describing below Accompanying drawing is some embodiments of the present invention, for those of ordinary skill in the art, before creative work is not paid Put, other accompanying drawings can also be obtained according to these accompanying drawings.
Fig. 1 is the distributed task dispatching method flow diagram triggered based on event that the embodiment of the present invention one is provided;
Fig. 2 is the implementing result feedback method flow chart that the embodiment of the present invention one is provided;
Fig. 3 is the distributed task dispatching method signaling diagram triggered based on event that the embodiment of the present invention two is provided;
Fig. 4 is a kind of distributed task dispatching schematic device triggered based on event that the embodiment of the present invention three is provided;
Fig. 5 is another distributed task dispatching device signal triggered based on event that the embodiment of the present invention four is provided Figure.
Icon:
100- task AM access modules;200- distributed task scheduling Queue modules;300- task scheduling engine modules;400- is performed Module;410- tasks carrying proxy modules;420- tasks carrying program modules.
Embodiment
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with accompanying drawing to the present invention Technical scheme be clearly and completely described, it is clear that described embodiment is a part of embodiment of the invention, rather than Whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art are not making creative work premise Lower obtained every other embodiment, belongs to the scope of protection of the invention.
At present, as the task scheduling system of one of big data system infrastructure construction, to the concurrent capability of task, appoint Process performance of business scheduling etc. proposes requirements at the higher level.And traditional task scheduling system is mainly appointed using time-based timing Business dispatching method, no matter whether the dependence of tasks carrying meets, as long as meeting time point, must just trigger tasks carrying, Poll judges dependence condition again in the process of implementation, so causes substantial amounts of server resource to waste, degraded performance, and can not Meet the real-time task scheduling requirement of big data.
Based on this, a kind of distributed task dispatching method and device triggered based on event provided in an embodiment of the present invention, The task scheduling ability of high concurrent can be lifted, server resource is effectively saved, big data real-time task scheduling requirement is met.
For ease of understanding the present embodiment, first to disclosed in the embodiment of the present invention it is a kind of based on event trigger Distributed task dispatching method describes in detail.
Embodiment one:
Fig. 1 is the distributed task dispatching method flow diagram triggered based on event that the embodiment of the present invention one is provided.
Reference picture 1, using task scheduling engine module as executive agent, the distributed task dispatching method triggered based on event Comprise the following steps:
Step S110, monitors distributed task scheduling Queue module, obtains monitoring information;
Step S120, judges distributed task scheduling Queue module under using master-salve distributed message structure according to monitoring information With the presence or absence of task requests message;If it is present performing step S130;If it does not exist, then repeating step S110;
Step S130, message call is sent to distributed task scheduling Queue module, so that distributed task scheduling Queue module is used The message queue mode of first in first out sends task requests message;
Specifically, task requests side initiates task requests message, task AM access module, which is responsible for accessing all task requests, to disappear Breath, and task requests message is submitted to distributed task scheduling Queue module.Task requests message includes timed task and event is appointed Business, timed task refers to, according to regular hour frequency, once task be performed at regular intervals;Event task is to touch in real time Send out performing for task.Task AM access module is submitted to distributed task scheduling Queue module by task requests message is asynchronous, and is submitting Execution status of task is begun listening for during completion.
Distributed task scheduling Queue module is appointed using the distributed message framework storage access of MS master-slave (Master-Slave) Business request message, can support to access while more than million grades tasks, and please to task in the message queue mode of first in first out Message is asked to be managed.Overall tasks scheduling architecture based on distributed task scheduling Queue module, can lift the height of server simultaneously The task scheduling ability of hair.
Task scheduling engine module is monitored distributed task scheduling Queue module, is finding distributed task scheduling Queue module In just obtain at once when having new task requests message.
Step S140, receives the task requests message that distributed task scheduling Queue module is sent, and task requests message is entered Row parsing, obtains task registration message;
Whether step S150, the execution time and dependence for judging task registration message meets preset trigger condition;Such as Fruit meets, then performs step S160;If do not met, step S170 is performed;
Step S160, tasks carrying message is obtained according to the task type of task registration message;
According to the exemplary embodiment of the present invention, task type includes timed task, and step S160 includes:
It is timed task according to the task type of task registration message, tasks carrying message is obtained according to timing mode.
Task type also includes event task, and step S160 also includes:
It is event task according to the task type of task registration message, tasks carrying message is obtained according to event schema.
Step S170, is scanned to dependence, until meeting preset trigger condition;
Specifically, task scheduling engine module is after the completion of job metadata initialization, to the task requests message of acquisition Attribute parsed, by the task registration of successfully resolved into task trigger, obtain task registration message.Triggered by task Perform analysis that time and dependence carry out rule condition of the device to task, judges whether to meet preset trigger condition, and In the case of meeting, according to task type, perform proxy module triggering according to timing mode or event schema calling task and appoint The execution of business registration message, obtains tasks carrying message.Do not met in the execution time of task registration message and dependence pre- If in the case of trigger condition, the dependence of the further scan event task of task scheduling engine module, until relation condition Untill satisfaction.Task scheduling engine module can clustered deploy(ment), including monitor node and perform node, some performs node and delayed machine Afterwards, task can be taken over by other nodes and be continued executing with.Task scheduling engine module is functionally broadly divided into tasks carrying Class function and task management class function, tasks carrying class function include task reception, task parsing, task dependence, task registration With task trigger etc., task management class function is run again including task, task suspension, task stop, task addition and task note Pin etc..
In addition, execution shape of the modules in task processes all in synchronized update distributed task scheduling Queue module State.Such as, after task scheduling engine module is parsed to task requests message, distributed task scheduling Queue module can be updated Execution state;After the completion of task scheduling engine module is judged task registration message, distributed task scheduling queue can be updated The execution state of module.
Step S180, performing module is sent to by tasks carrying message, so that performing module is carried out to tasks carrying message Perform, and obtain execution state and execution journal.
According to the exemplary embodiment of the present invention, step S180 includes:
So that performing module is called according to tasks carrying message to configuration processor, instruction message is obtained, and according to finger Make message perform tasks carrying message, obtain execution state and execution journal.
Specifically, performing module includes tasks carrying proxy module and tasks carrying program module.Tasks carrying acts on behalf of mould Block is responsible for receiving the tasks carrying message that task trigger is submitted in task scheduling engine, and the specific tasks carrying of asynchronous call Program module, specific tasks carrying program is transmitted to by tasks carrying message.Poll obtains tasks carrying journey after calling successfully The execution journal and execution state of sequence module, the state until returning to tasks carrying success or failure.Tasks carrying acts on behalf of mould Block acquiescence support java applet, storing process, Perl (Practical Extraction and Reporting Language, It is actual to extract with reporting language) script, Shell scripts, (Hyper Text Transfer Protocol surpass by Hadoop, Http Text transfer protocol), the configuration processor of the main flow such as Web Service, while providing second development interface, can customize as needed The task agent of other configuration processors.
Tasks carrying program module includes the program for being responsible for performing specific tasks, such as java applet, storing process, Perl pin The configuration processor such as sheet, Shell scripts, Hadoop, Http, WebService.Tasks carrying program module is acted on behalf of by tasks carrying Module calls triggering to perform.Namely tasks carrying proxy module is called according to tasks carrying message to configuration processor, is obtained To instruction message;Configuration processor is performed according to instruction message to specific tasks carrying message, is obtained execution state and is held Row daily record, and execution state and execution journal are returned to according to proxy interface.
According to the exemplary embodiment of the present invention, reference picture 2, this method also comprises the following steps:
Step S210, receives the execution journal that performing module is sent and the execution knot for carrying out judging to obtain to execution state Really;
Step S220, is recorded to implementing result and execution journal;
Step S230, is sent to task AM access module, so that task by implementing result by distributed task scheduling Queue module AM access module judges whether tasks carrying completes according to implementing result, and implementing result is fed back into task in the case of completion Requesting party.
Specifically, tasks carrying proxy module to execution state judge obtained implementing result, and by implementing result Task scheduling engine module is fed back to execution journal.Task scheduling engine module records implementing result and execution journal, and more Execution state in new distributed task scheduling Queue module;If tasks carrying success, calling task AM access module triggering subordinate Event task requests.Task AM access module obtains task final according to the execution state updated in distributed task scheduling Queue module Execution state, and judge whether tasks carrying completes, and implementing result is fed back into task requests side in the case of completion.
To the processing procedure completed here to task requests, task AM access module receives new task requests again Afterwards, the flow of new task processing is started.
Embodiment two:
Fig. 3 is the distributed task dispatching method signaling diagram triggered based on event that the embodiment of the present invention two is provided.
Reference picture 3, the distributed task dispatching method triggered based on event is comprised the following steps:
Step S01, task AM access module is submitted to distributed task scheduling Queue module by task requests message is asynchronous;
Step S02, distributed task scheduling Queue module using master-salve distributed message structure deposit task requests message, and with The message queue mode of first in first out sends task requests message;
The execution status of task of distributed task scheduling Queue module is monitored in step S03, task AM access module;
Task requests message is sent to task scheduling engine module by step S04, distributed task scheduling Queue module;
Task scheduling engine module is monitored distributed task scheduling Queue module, is finding distributed task scheduling Queue module In just obtain at once when having new task requests message.
Step S05, is parsed to task requests message, obtains task registration message;
Step S06, task scheduling engine module updates the execution state in distributed task scheduling Queue module;
Whether step S07, the execution time and dependence for judging task registration message meets preset trigger condition;
Step S08, task scheduling engine module updates the execution state in distributed task scheduling Queue module;
Step S09, in the case where meeting, tasks carrying message is obtained according to the task type of task registration message;
Step S10, task scheduling engine module updates the execution state in distributed task scheduling Queue module;
Tasks carrying message is sent to tasks carrying proxy module by step S11, task scheduling engine module;
Step S12, tasks carrying proxy module asynchronous call tasks carrying program;
Tasks carrying program module is issued in tasks carrying message station by step S13, tasks carrying proxy module;
Step S14, configuration processor module is performed to tasks carrying message, and obtains execution state and execution journal;
Execution state and execution journal are returned to tasks carrying proxy module by step S15, configuration processor module;
Step S16, tasks carrying proxy module is judged execution state, obtains implementing result;
Implementing result and execution journal are returned to task scheduling engine module by step S17, tasks carrying proxy module;
Step S18, task scheduling engine module is recorded to implementing result and execution journal;
Step S19, task scheduling engine module updates the execution state in distributed task scheduling Queue module;
Tasks carrying success message is sent to task AM access module by step S20, task scheduling engine module;
Step S21, task AM access module triggering subordinate event task requests message;
Step S22, task AM access module obtains the execution state from distributed task scheduling Queue module;
Whether step S23, task AM access module completes according to execution condition adjudgement tasks carrying;
Step S24, task requests side is fed back in the case of completion by implementing result.
Embodiment three:
Fig. 4 is a kind of distributed task dispatching schematic device triggered based on event that the embodiment of the present invention three is provided.
Reference picture 4, includes task AM access module 100, distribution based on the distributed task dispatching device that event is triggered and appoints Business Queue module 200, task scheduling engine module 300, performing module 400, wherein performing module 400 include tasks carrying generation Manage module 410 and tasks carrying program module 420.
Task scheduling engine module 300, for monitoring distributed task scheduling Queue module 200, obtains monitoring information, Judge that distributed task scheduling Queue module 200 please with the presence or absence of task under using master-salve distributed message structure according to monitoring information Message is sought, and in case of presence, message call is sent to distributed task scheduling Queue module 200, distributed task scheduling team is received The task requests message that row module 200 is sent, and task requests message is parsed, task registration message is obtained, judges to appoint Whether the execution time of business registration message and dependence meet preset trigger condition, and in the case where meeting, according to task The task type of registration message obtains tasks carrying message, and tasks carrying message is sent into performing module 400;
Distributed task scheduling Queue module 200, for there is task requests message using master-salve distributed message structure, and is adopted Task requests message is sent with the message queue mode of first in first out;
Performing module 400, for being performed to tasks carrying message, and obtains execution state and execution journal.
Specifically, distributed task scheduling Queue module 200, task scheduling engine module 300, tasks carrying proxy module 410 It is server with tasks carrying program module 420, the device can also include task requests side, and task requests side is computer.
According to the exemplary embodiment of the present invention, task scheduling engine module 300 is additionally operable to:
Receive the execution journal that performing module 400 is sent and the implementing result for carrying out judging to obtain to execution state;
Implementing result and execution journal are recorded;
Implementing result is sent to task AM access module 100 by distributed task scheduling Queue module 200, so that task is accessed Module 100 judges whether tasks carrying completes according to implementing result, and implementing result is fed back into task in the case of completion Requesting party.
According to the exemplary embodiment of the present invention, task scheduling engine module 300 is additionally operable to:
So that performing module 400 is called according to tasks carrying message to configuration processor, instruction message is obtained, and according to Instruction message is performed to tasks carrying message, obtains execution state and execution journal.
According to the exemplary embodiment of the present invention, task type includes timed task, and task scheduling engine module 300 is also used In:
It is timed task according to the task type of task registration message, tasks carrying message is obtained according to timing mode.
According to the exemplary embodiment of the present invention, task type also includes event task, and task scheduling engine module 300 is also For:
It is event task according to the task type of task registration message, tasks carrying message is obtained according to event schema.
Example IV:
Fig. 5 is another distributed task dispatching device signal triggered based on event that the embodiment of the present invention four is provided Figure.
All task requests message of access is responsible in reference picture 5, task AM access module 100, and task requests message includes fixed When task requests and event task requests, and task requests message is submitted to distributed task scheduling Queue module 200.Distribution is appointed The task requests message that business Queue module 200 is accessed using Master-Slave distributed message framework storage, and with advanced The message queue mode first gone out is managed to task requests message.Task scheduling engine module 300 is received using listening mode Task requests message in distributed task scheduling Queue module 200, and task requests message is parsed, by appointing for successfully resolved Business is registered in task trigger, and sentencing for rule condition is carried out to the execution time of task and dependence by task trigger It is disconnected, after judging successfully, further according to task type, touched according to the mode invocation tasks carrying proxy module 410 of timing or event Send out the execution of registered task.Tasks carrying proxy module 410 receives what task trigger in task scheduling engine module 300 was submitted Tasks carrying message, is transmitted to specific tasks carrying program, and collect the execution of tasks carrying program by tasks carrying message Daily record and implementing result.Tasks carrying program module 420 is responsible for performing specific task.
The distributed task dispatching method and device triggered based on event that the present invention is provided, passes through task scheduling engine mould Block is monitored distributed task scheduling Queue module, obtains monitoring information, and distributed task scheduling queue mould is judged according to monitoring information Block whether there is task requests message under using master-salve distributed message structure, and in case of presence, appoint to distribution Business Queue module sends message call, is appointed with making distributed task scheduling Queue module be sent using the message queue mode of first in first out Business request message, receives the task requests message that distributed task scheduling Queue module is sent, and task requests message is parsed, Task registration message is obtained, whether the execution time and dependence for judging task registration message meet preset trigger condition, and In the case where meeting, tasks carrying message is obtained according to the task type of task registration message, tasks carrying message is sent To performing module, so that performing module is performed to tasks carrying message, and execution state and execution journal are obtained.The present invention The task scheduling ability of high concurrent can be lifted, server resource is effectively saved, big data real-time task scheduling requirement is met.
The computer journey for the distributed task dispatching method and device triggered based on event that the embodiment of the present invention is provided Sequence product, including the computer-readable recording medium of program code is stored, the instruction that described program code includes can be used for holding Method described in row previous methods embodiment, implements and can be found in embodiment of the method, will not be repeated here.
It is apparent to those skilled in the art that, for convenience and simplicity of description, the system of foregoing description With the specific work process of device, the corresponding process in preceding method embodiment is may be referred to, be will not be repeated here.
In addition, in the description of the embodiment of the present invention, unless otherwise clearly defined and limited, term " installation ", " phase Even ", " connection " should be interpreted broadly, for example, it may be being fixedly connected or being detachably connected, or be integrally connected;Can To be mechanical connection or electrical connection;Can be joined directly together, can also be indirectly connected to by intermediary, Ke Yishi The connection of two element internals.For the ordinary skill in the art, with concrete condition above-mentioned term can be understood at this Concrete meaning in invention.
If the function is realized using in the form of SFU software functional unit and is used as independent production marketing or in use, can be with It is stored in a computer read/write memory medium.Understood based on such, technical scheme is substantially in other words The part contributed to prior art or the part of the technical scheme can be embodied in the form of software product, the meter Calculation machine software product is stored in a storage medium, including some instructions are to cause a computer equipment (can be individual People's computer, server, or network equipment etc.) perform all or part of step of each of the invention embodiment methods described. And foregoing storage medium includes:USB flash disk, mobile hard disk, read-only storage (ROM, Read-Only Memory), arbitrary access are deposited Reservoir (RAM, Random Access Memory), magnetic disc or CD etc. are various can be with the medium of store program codes.
In the description of the invention, it is necessary to explanation, term " " center ", " on ", " under ", "left", "right", " vertical ", The orientation or position relationship of the instruction such as " level ", " interior ", " outer " be based on orientation shown in the drawings or position relationship, merely to Be easy to the description present invention and simplify description, rather than indicate or imply signified device or element must have specific orientation, With specific azimuth configuration and operation, therefore it is not considered as limiting the invention.In addition, term " first ", " second ", " the 3rd " is only used for describing purpose, and it is not intended that indicating or implying relative importance.
Finally it should be noted that:Embodiment described above, is only the embodiment of the present invention, to illustrate the present invention Technical scheme, rather than its limitations, protection scope of the present invention is not limited thereto, although with reference to the foregoing embodiments to this hair It is bright to be described in detail, it will be understood by those within the art that:Any one skilled in the art The invention discloses technical scope in, it can still modify to the technical scheme described in previous embodiment or can be light Change is readily conceivable that, or equivalent substitution is carried out to which part technical characteristic;And these modifications, change or replacement, do not make The essence of appropriate technical solution departs from the spirit and scope of technical scheme of the embodiment of the present invention, should all cover the protection in the present invention Within the scope of.Therefore, protection scope of the present invention described should be defined by scope of the claims.

Claims (10)

1. a kind of distributed task dispatching method triggered based on event, it is characterised in that applied to task scheduling engine module, Methods described includes:
Distributed task scheduling Queue module is monitored, monitoring information is obtained;
According to the monitoring information judge the distributed task scheduling Queue module using under master-salve distributed message structure whether There is task requests message;
If it is present message call is sent to the distributed task scheduling Queue module, so that the distributed task scheduling queue mould Block sends the task requests message using the message queue mode of first in first out;
The task requests message that the distributed task scheduling Queue module is sent is received, and the task requests message is carried out Parsing, obtains task registration message;
Whether the execution time and dependence for judging the task registration message meet preset trigger condition;
If met, tasks carrying message is obtained according to the task type of the task registration message;
The tasks carrying message is sent to performing module, so that the performing module is held to the tasks carrying message OK, and execution state and execution journal are obtained.
2. the distributed task dispatching method according to claim 1 triggered based on event, it is characterised in that methods described Also include:
Receive the execution journal that the performing module is sent and the implementing result for carrying out judging to obtain to the execution state;
The implementing result and the execution journal are recorded;
The implementing result is sent to task AM access module by the distributed task scheduling Queue module, so that the task connects Enter module and judge whether tasks carrying completes according to the implementing result, and feed back the implementing result in the case of completion To task requests side.
3. it is according to claim 1 based on event trigger distributed task dispatching method, it is characterised in that it is described so that The performing module is performed to the tasks carrying message, and is obtained execution state and execution journal and included:
So that the performing module is called according to the tasks carrying message to configuration processor, instruction message, and root are obtained The tasks carrying message is performed according to the instruction message, the execution state and the execution journal is obtained.
4. the distributed task dispatching method according to claim 1 triggered based on event, it is characterised in that the task Type includes timed task, and the task type according to the task registration message, which obtains tasks carrying message, to be included:
It is the timed task according to the task type of the task registration message, the tasks carrying is obtained according to timing mode Message.
5. the distributed task dispatching method according to claim 1 triggered based on event, it is characterised in that the task Type also includes event task, and the task type according to the task registration message, which obtains tasks carrying message, to be included:
It is the event task according to the task type of the task registration message, the tasks carrying is obtained according to event schema Message.
6. a kind of distributed task dispatching device triggered based on event, it is characterised in that described device includes:
Task scheduling engine module, for monitoring distributed task scheduling Queue module, obtains monitoring information, according to the prison Message is listened to judge that the distributed task scheduling Queue module is disappeared under using master-salve distributed message structure with the presence or absence of task requests Breath, and in case of presence, message call is sent to the distributed task scheduling Queue module, receive the distributed task scheduling team The task requests message that row module is sent, and the task requests message is parsed, task registration message is obtained, is sentenced Whether the execution time for the task registration message of breaking and dependence meet preset trigger condition, and in the case where meeting, Tasks carrying message is obtained according to the task type of the task registration message, the tasks carrying message is sent to execution mould Block;
The distributed task scheduling Queue module, for there is the task requests message using master-salve distributed message structure, and The task requests message is sent using the message queue mode of first in first out;
The performing module, for being performed to the tasks carrying message, and obtains execution state and execution journal.
7. the distributed task dispatching device according to claim 6 triggered based on event, it is characterised in that the task Scheduling engine module is additionally operable to:
Receive the execution journal that the performing module is sent and the implementing result for carrying out judging to obtain to the execution state;
The implementing result and the execution journal are recorded;
The implementing result is sent to task AM access module by the distributed task scheduling Queue module, so that the task connects Enter module and judge whether tasks carrying completes according to the implementing result, and feed back the implementing result in the case of completion To task requests side.
8. the distributed task dispatching device according to claim 6 triggered based on event, it is characterised in that the task Scheduling engine module is additionally operable to:
So that the performing module is called according to the tasks carrying message to configuration processor, instruction message, and root are obtained The tasks carrying message is performed according to the instruction message, the execution state and the execution journal is obtained.
9. the distributed task dispatching device according to claim 6 triggered based on event, it is characterised in that the task Type includes timed task, and the task scheduling engine module is additionally operable to:
It is the timed task according to the task type of the task registration message, the tasks carrying is obtained according to timing mode Message.
10. the distributed task dispatching device according to claim 6 triggered based on event, it is characterised in that described Service type also includes event task, and the task scheduling engine module is additionally operable to:
It is the event task according to the task type of the task registration message, the tasks carrying is obtained according to event schema Message.
CN201710217652.4A 2017-04-05 2017-04-05 Distributed task scheduling method and device based on event triggering Active CN106980543B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710217652.4A CN106980543B (en) 2017-04-05 2017-04-05 Distributed task scheduling method and device based on event triggering

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710217652.4A CN106980543B (en) 2017-04-05 2017-04-05 Distributed task scheduling method and device based on event triggering

Publications (2)

Publication Number Publication Date
CN106980543A true CN106980543A (en) 2017-07-25
CN106980543B CN106980543B (en) 2020-02-07

Family

ID=59345483

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710217652.4A Active CN106980543B (en) 2017-04-05 2017-04-05 Distributed task scheduling method and device based on event triggering

Country Status (1)

Country Link
CN (1) CN106980543B (en)

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107704323A (en) * 2017-11-07 2018-02-16 广州探迹科技有限公司 A kind of web crawlers method for scheduling task and device
CN108184141A (en) * 2017-11-22 2018-06-19 链家网(北京)科技有限公司 A kind of processing method and server of monitor video task
CN108427600A (en) * 2018-01-24 2018-08-21 平安科技(深圳)有限公司 Data task processing method, application server and computer readable storage medium
CN108762910A (en) * 2018-06-06 2018-11-06 亚信科技(中国)有限公司 A kind of distributed task dispatching method and system
CN109426572A (en) * 2017-08-29 2019-03-05 杭州海康威视数字技术股份有限公司 Task processing method, device and electronic equipment
CN110210763A (en) * 2019-06-05 2019-09-06 深圳前海微众银行股份有限公司 Event-driven method, device, equipment and storage medium based on ITSM
CN110244991A (en) * 2019-05-20 2019-09-17 平安科技(深圳)有限公司 A kind of micro services dependency analysis method and device
CN110290215A (en) * 2019-06-28 2019-09-27 深圳前海微众银行股份有限公司 A kind of method for transmitting signals and device
CN110347505A (en) * 2019-06-28 2019-10-18 北京淇瑀信息科技有限公司 A kind of access method, device and electronic equipment for supporting cross-network segment to dispatch
CN110704173A (en) * 2019-09-29 2020-01-17 深圳前海环融联易信息科技服务有限公司 Task scheduling method, scheduling system, electronic device and computer storage medium
CN110737516A (en) * 2018-07-20 2020-01-31 网宿科技股份有限公司 method and device for executing data processing task
CN110895487A (en) * 2018-09-12 2020-03-20 北京奇虎科技有限公司 Distributed task scheduling system
CN110895486A (en) * 2018-09-12 2020-03-20 北京奇虎科技有限公司 Distributed task scheduling system
CN111026513A (en) * 2018-10-10 2020-04-17 上海寒武纪信息科技有限公司 Task assigning device, task scheduler, and task processing method
CN111314241A (en) * 2020-01-17 2020-06-19 新华智云科技有限公司 Task scheduling method and scheduling system
CN111400017A (en) * 2020-03-26 2020-07-10 华泰证券股份有限公司 Distributed complex task scheduling method
CN111782351A (en) * 2019-07-18 2020-10-16 北京京东尚科信息技术有限公司 Distributed task scheduling method and device
CN112380183A (en) * 2020-11-13 2021-02-19 深圳市和讯华谷信息技术有限公司 Distributed file synchronization method and device, computer equipment and storage medium
CN112631738A (en) * 2021-01-26 2021-04-09 江苏信数科技有限公司 Distributed task processing system and method based on library container
CN113055126A (en) * 2021-03-09 2021-06-29 华夏云融航空科技有限公司 Flight data decoding method and device and terminal equipment
CN113119127A (en) * 2021-04-20 2021-07-16 大连宏燊软件技术有限公司 Task scheduling method and system for RPA robot
CN113268318A (en) * 2021-04-07 2021-08-17 北京思特奇信息技术股份有限公司 Task scheduling method and distributed system
CN113641514A (en) * 2021-07-23 2021-11-12 浙江吉利控股集团有限公司 Controllable real-time task remote execution system
CN113919878A (en) * 2021-10-16 2022-01-11 云南白药集团医药电子商务有限公司 Automatic marketing method and device capable of being configured in full flow

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102831012A (en) * 2011-06-16 2012-12-19 日立(中国)研究开发有限公司 Task scheduling device and task scheduling method in multimode distributive system
CN105786611A (en) * 2014-12-16 2016-07-20 中国移动通信集团公司 Method and device for task scheduling of distributed cluster

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102831012A (en) * 2011-06-16 2012-12-19 日立(中国)研究开发有限公司 Task scheduling device and task scheduling method in multimode distributive system
CN105786611A (en) * 2014-12-16 2016-07-20 中国移动通信集团公司 Method and device for task scheduling of distributed cluster

Cited By (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109426572B (en) * 2017-08-29 2021-07-02 杭州海康威视数字技术股份有限公司 Task processing method and device and electronic equipment
CN109426572A (en) * 2017-08-29 2019-03-05 杭州海康威视数字技术股份有限公司 Task processing method, device and electronic equipment
CN107704323A (en) * 2017-11-07 2018-02-16 广州探迹科技有限公司 A kind of web crawlers method for scheduling task and device
CN108184141A (en) * 2017-11-22 2018-06-19 链家网(北京)科技有限公司 A kind of processing method and server of monitor video task
CN108184141B (en) * 2017-11-22 2020-07-24 贝壳找房(北京)科技有限公司 Processing method of monitoring video task and server
CN108427600A (en) * 2018-01-24 2018-08-21 平安科技(深圳)有限公司 Data task processing method, application server and computer readable storage medium
WO2019144552A1 (en) * 2018-01-24 2019-08-01 平安科技(深圳)有限公司 Data task processing method, application server and computer-readable storage medium
CN108427600B (en) * 2018-01-24 2021-03-16 平安科技(深圳)有限公司 Data task processing method, application server and computer readable storage medium
CN108762910A (en) * 2018-06-06 2018-11-06 亚信科技(中国)有限公司 A kind of distributed task dispatching method and system
CN110737516A (en) * 2018-07-20 2020-01-31 网宿科技股份有限公司 method and device for executing data processing task
CN110895487B (en) * 2018-09-12 2023-03-10 北京奇虎科技有限公司 Distributed task scheduling system
CN110895487A (en) * 2018-09-12 2020-03-20 北京奇虎科技有限公司 Distributed task scheduling system
CN110895486A (en) * 2018-09-12 2020-03-20 北京奇虎科技有限公司 Distributed task scheduling system
CN110895486B (en) * 2018-09-12 2022-08-12 北京奇虎科技有限公司 Distributed task scheduling system
CN111026513A (en) * 2018-10-10 2020-04-17 上海寒武纪信息科技有限公司 Task assigning device, task scheduler, and task processing method
CN111026513B (en) * 2018-10-10 2022-12-02 上海寒武纪信息科技有限公司 Task assigning device, task scheduler, and task processing method
CN110244991A (en) * 2019-05-20 2019-09-17 平安科技(深圳)有限公司 A kind of micro services dependency analysis method and device
CN110210763A (en) * 2019-06-05 2019-09-06 深圳前海微众银行股份有限公司 Event-driven method, device, equipment and storage medium based on ITSM
CN110347505A (en) * 2019-06-28 2019-10-18 北京淇瑀信息科技有限公司 A kind of access method, device and electronic equipment for supporting cross-network segment to dispatch
CN110290215A (en) * 2019-06-28 2019-09-27 深圳前海微众银行股份有限公司 A kind of method for transmitting signals and device
CN110347505B (en) * 2019-06-28 2022-03-08 北京淇瑀信息科技有限公司 Access method and device supporting cross-network-segment scheduling and electronic equipment
CN110290215B (en) * 2019-06-28 2021-09-28 深圳前海微众银行股份有限公司 Signal transmission method and device
CN111782351A (en) * 2019-07-18 2020-10-16 北京京东尚科信息技术有限公司 Distributed task scheduling method and device
CN110704173A (en) * 2019-09-29 2020-01-17 深圳前海环融联易信息科技服务有限公司 Task scheduling method, scheduling system, electronic device and computer storage medium
CN111314241B (en) * 2020-01-17 2024-03-29 新华智云科技有限公司 Task scheduling method and scheduling system
CN111314241A (en) * 2020-01-17 2020-06-19 新华智云科技有限公司 Task scheduling method and scheduling system
CN111400017A (en) * 2020-03-26 2020-07-10 华泰证券股份有限公司 Distributed complex task scheduling method
CN112380183A (en) * 2020-11-13 2021-02-19 深圳市和讯华谷信息技术有限公司 Distributed file synchronization method and device, computer equipment and storage medium
CN112380183B (en) * 2020-11-13 2024-05-24 深圳市和讯华谷信息技术有限公司 Method and device for synchronizing distributed files, computer equipment and storage medium
CN112631738A (en) * 2021-01-26 2021-04-09 江苏信数科技有限公司 Distributed task processing system and method based on library container
CN113055126A (en) * 2021-03-09 2021-06-29 华夏云融航空科技有限公司 Flight data decoding method and device and terminal equipment
CN113268318A (en) * 2021-04-07 2021-08-17 北京思特奇信息技术股份有限公司 Task scheduling method and distributed system
CN113119127A (en) * 2021-04-20 2021-07-16 大连宏燊软件技术有限公司 Task scheduling method and system for RPA robot
CN113641514A (en) * 2021-07-23 2021-11-12 浙江吉利控股集团有限公司 Controllable real-time task remote execution system
CN113919878A (en) * 2021-10-16 2022-01-11 云南白药集团医药电子商务有限公司 Automatic marketing method and device capable of being configured in full flow
CN113919878B (en) * 2021-10-16 2024-05-28 云南白药集团医药电子商务有限公司 Automatic marketing method and device capable of realizing full-flow configuration

Also Published As

Publication number Publication date
CN106980543B (en) 2020-02-07

Similar Documents

Publication Publication Date Title
CN106980543A (en) The distributed task dispatching method and device triggered based on event
CN112000448A (en) Micro-service architecture-based application management method
CN106095775A (en) A kind of method and system realizing data query or derivation
CN108509523A (en) Structuring processing method, equipment and the readable storage medium storing program for executing of block chain data
CN111711697A (en) Message pushing method, device, equipment and storage medium
CN109783151B (en) Method and device for rule change
CN110995511A (en) Cloud computing operation and maintenance management method and device based on micro-service architecture and terminal equipment
CN102438042B (en) Dynamic parameter synchronizing method and system of multipoint access device
CN111240812A (en) Task execution method and device
CN106331081B (en) Information synchronization method and device
CN111200606A (en) Deep learning model task processing method, system, server and storage medium
CN108123938A (en) A kind of AAA service implementing methods and system
CN113342503B (en) Real-time progress feedback method, device, equipment and storage medium
CN112799970B (en) Test data processing method, device, electronic equipment and medium
CN110661780A (en) Wireless city data sharing method and system based on SAAS application
CN107819855B (en) Message distribution method and device
CN101562622B (en) Method for executing user request and corresponding server thereof
CN112631756A (en) Distributed regulation and control method and device applied to space flight measurement and control software
CN115065686A (en) Configuration method, device and system of distributed load balancing system
CN110071840A (en) Upgrade method, device and client
CN112380040B (en) Message processing method and device, electronic equipment and storage medium
CN115499432A (en) Family terminal computing resource management system and computing resource scheduling method
CN115357395A (en) Fault equipment task transfer method and system, electronic equipment and storage medium
CN111381921B (en) Front-end and back-end separation system and method based on Ambari
CN113067878A (en) Information acquisition method, information acquisition device, information acquisition apparatus, information acquisition medium, and program product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20200331

Address after: 350100 Gaoqi village, Nanyu Town, Minhou County, Fuzhou City, Fujian Province

Co-patentee after: ZHIHENG TECHNOLOGY Co.,Ltd.

Patentee after: FUJIAN ZHIHENG SOFTWARE TECHNOLOGY Co.,Ltd.

Address before: The South Island town of Minhou County of Fuzhou City, Fujian province 350000 high Qi Cun

Patentee before: FUJIAN ZHIHENG SOFTWARE TECHNOLOGY Co.,Ltd.

TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20210115

Address after: 350109 Zhiheng science and Technology Park, 66 Zhiheng science and technology road, Gaoqi village, Nanyu Town, Minhou County, Fuzhou City, Fujian Province

Patentee after: ZHIHENG TECHNOLOGY Co.,Ltd.

Address before: 350100 Gaoqi village, Nanyu Town, Minhou County, Fuzhou City, Fujian Province

Patentee before: FUJIAN ZHIHENG SOFTWARE TECHNOLOGY Co.,Ltd.

Patentee before: ZHIHENG TECHNOLOGY Co.,Ltd.

TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20240218

Address after: 4th Floor, West Side, Zhiheng Science and Technology Park, No. 66 Bianzhou, Gaoqi Village, Nanyu Town, Minhou County, Fuzhou City, Fujian Province, 350109

Patentee after: Fujian Zhiheng Environmental Engineering Technology Co.,Ltd.

Country or region after: China

Address before: 350109 Zhiheng science and Technology Park, 66 Zhiheng science and technology road, Gaoqi village, Nanyu Town, Minhou County, Fuzhou City, Fujian Province

Patentee before: ZHIHENG TECHNOLOGY Co.,Ltd.

Country or region before: China