CN107066339A - Distributed job manager and distributed job management method - Google Patents
Distributed job manager and distributed job management method Download PDFInfo
- Publication number
- CN107066339A CN107066339A CN201611154942.0A CN201611154942A CN107066339A CN 107066339 A CN107066339 A CN 107066339A CN 201611154942 A CN201611154942 A CN 201611154942A CN 107066339 A CN107066339 A CN 107066339A
- Authority
- CN
- China
- Prior art keywords
- job
- message
- queue
- execution
- operations
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Abstract
The invention provides a kind of distributed job manager and distributed job management method.Wherein, this method includes：The User Interface provided by the job scheduling page, the Job Operations message of the job message of operation, operation is sent to queue, and from queue obtain operation execution status message；Job execution container on multiple nodes obtains job message and Job Operations message from queue, and performs corresponding operation according to job message and Job Operations message, and the execution status message of operation is sent to queue.By the present invention, operation processing and the decoupling of database are realized, the job processing system solved in correlation technique is coupled tightly with database, it is impossible to the problem of realizing distributed treatment；And because Job execution container is distributed on multiple nodes, only need to add the new node containing Job execution container when needing and increasing job throughput, system easily extends, and improves job processing efficiency and the scalability of job platform.
Description
Technical field
The present invention relates to software processing field, in particular to a kind of distributed job manager and distribution
Formula job management method.
Background technology
In history, batch processing job is widely used in mainframe computer.Because the computer of this rank is very expensive,
Based on avoiding computing resource idle etc., reason needs to use batch processing.Another reason is, on the electronic computer of early stage,
The interactive program at terminal device interface (developing into graphic user interface later) is not yet promoted.The development of batch processing is remote to be won originally
Application on giant brain, is also commonly used for unix environment now, and the working procedure of complexity is arranged with CRON and at mechanism.Microsoft
Windows systems provide task scheduling instrument, and user can be with the frequency of self-defined job run, it is possible to which what selection to be run should
Use program.
In contemporary information systems, due to operation system processing requirement, such as batch processing job is needed daily, reported
Table data calculate, print series documentation etc..For mass data processing simultaneously, it is desirable to handled in substantial amounts of open computing resource
Multiple operations, and processing procedure can be rapidly completed.
At present, Windows task manager is there is provided various temporal frequencies are customized, in specified execution operation task class
Type can only be limitedly applied to program, send Email and display message.Unix CRON and AT mechanism can equally be customized respectively
Temporal frequency is planted, and using application program as run unit.
Above-mentioned technology can only be in units of process in locally initiating application program, towards numerous high-frequency small operations of batch
When, system resources consumption is high；Nonusable routine, such as the homework type such as component, database application can not be supported；Can not be in distribution
The integration of computing resource is completed in the case of formula.Majority realizes that calculating is all based on oracle for storing process, causes occupancy a large amount of
Oracle cpu resources and memory source, are coupled tightly with oracle database.Due to being realized with storing process, performance boost mode
Single, hardware resource that can only be by increasing database causes oracle costs to improve come improving performance.
The content of the invention
The invention provides a kind of distributed job manager and distributed job management method, at least to solve related skill
Job processing system in art is coupled tightly with database, it is impossible to the problem of realizing distributed treatment.
According to an aspect of the invention, there is provided a kind of distributed job manager, including：The job scheduling page and many
Individual Job execution container, the multiple Job execution container is distributed on multiple nodes, the job scheduling page and described many
Individual Job execution container is interacted by queue；Wherein,
The job scheduling page, for providing User Interface, to realize operation customization, the scheduling mode of operation
Customization, and the execution state of operation monitoring；
The Job execution container, for performing corresponding operation according to the scheduling mode of the job scheduling page customization,
And give the job scheduling page by the execution feedback of status of operation.
According to another aspect of the present invention, a kind of distributed job management method is additionally provided, including：
The User Interface provided by the job scheduling page, the Job Operations of the job message of operation, operation are disappeared
Breath send to queue, and from queue obtain operation execution status message；
Job execution container on multiple nodes obtains job message and Job Operations message from queue, and according to operation
Message and Job Operations message perform corresponding operation, and the execution status message of operation is sent to queue.
By the present invention, using the User Interface provided by the job scheduling page, by the job message of operation, make
The Job Operations message of industry is sent to queue, and from queue obtain operation execution status message；Work on multiple nodes
Industry performs container and job message and Job Operations message is obtained from queue, and is performed according to job message and Job Operations message
Corresponding operation, and the execution status message of operation is sent to the mode of queue, realizes operation processing and database
Decoupling, the job processing system solved in correlation technique is coupled tightly with database, it is impossible to the problem of realizing distributed treatment；And
Because Job execution container is distributed on multiple nodes, new the containing of addition is only needed to when needing and increasing job throughput and is made
Industry performs the node of container, and system easily extends, and improves job processing efficiency and the scalability of job platform.
Brief description of the drawings
Accompanying drawing described herein is used for providing a further understanding of the present invention, constitutes the part of the application, this hair
Bright schematic description and description is used to explain the present invention, does not constitute inappropriate limitation of the present invention.In the accompanying drawings：
Fig. 1 is the structural representation of distributed job manager according to embodiments of the present invention；
Fig. 2 is the preferred structure schematic diagram of distributed job manager according to embodiments of the present invention；
Fig. 3 is the flow chart of distributed job management method according to embodiments of the present invention；
Fig. 4 is the structural representation of distributed job manager according to the preferred embodiment of the invention；
Fig. 5 is execution state transition diagram according to the preferred embodiment of the invention.
Embodiment
Describe the present invention in detail below with reference to accompanying drawing and in conjunction with the embodiments.It should be noted that not conflicting
In the case of, the feature in embodiment and embodiment in the application can be mutually combined.
Embodiment 1
Present embodiments provide a kind of distributed job manager.Fig. 1 is distributed job according to embodiments of the present invention
The structural representation of manager, as shown in figure 1, the distributed job manager includes：The job scheduling page 1 and multiple operations are held
Row container 2, multiple Job execution containers 2 are distributed on multiple nodes, and the job scheduling page 1 and multiple Job execution containers 2 are logical
Queue 4 is crossed to interact；Wherein,
The job scheduling page 1, for providing User Interface, to realize that operation customization, the scheduling mode of operation are determined
System, and the execution state of operation monitoring；
Job execution container 2, the scheduling mode for being customized according to the job scheduling page 1 performs corresponding operation, and will make
The execution feedback of status of industry is to the job scheduling page 1.
The above-mentioned job scheduling page 1 can be deployed on one or more node, but the same time typically only allows for
The job scheduling page 1 on one node is active, to avoid operation or the tune of multiple customizations of the job scheduling pages 1
Degree mode is collided with each other.Or, it is allowed to the job scheduling page 1 on multiple nodes is in active state, certain by customization
Conflict-solving strategy avoid the operation that the multiple job scheduling pages 1 are customized or scheduling mode from colliding with each other.
Above-mentioned operation customization, the customization of the scheduling mode of operation are to Job execution container 2 to send operation by queue 4
Description information (such as contextual information) come what is realized.
Alternatively, divided according to the function of queue, queue 4 can include：Job queue 41, operation queue 42 and monitoring team
Row 43, wherein, job queue 41 is used for the discrepancy queue operation for handling job message, wherein, job message is used to indicate operation
Type and operation scheduling mode；Operation queue 42 is used for the discrepancy queue operation for handling Job Operations message, wherein, operation
Operation information is used to indicate to perform operation, termination operation or pausing operation to operation；Monitoring queue 43 is used to handle operation
Perform the discrepancy queue operation of status message.Pass through one operation of unique mark mark of operation in different queue.
Fig. 2 is the preferred structure schematic diagram of distributed job manager according to embodiments of the present invention.
With reference to Fig. 2, alternatively, the job scheduling page 1 includes：Job message sending module 11, Job Operations message are sent
Module 12 and Job execution status message receiving module 13, wherein, job message sending module 11, for sending job message extremely
Job queue 41；Job Operations message transmission module 12, for sending Job Operations message to operation queue 42；Job execution shape
State message reception module 13, for receiving Job execution status message from monitoring queue 43.
Alternatively, Job Operations message transmission module 12 includes：Job's context resolution unit 121, Job execution state
Message reception units 122, Job Operations message generation unit 123, and Job Operations message sending unit 124, wherein, operation
Context resolution unit 121, for parsing the corresponding job message of operation, to obtain the job relations between multiple operations, its
In, job relations include：Dependence, mutex relation；Job execution status message receiving unit 122, for from monitoring queue
43 obtain the Job execution status message of operation, and Job execution status message includes：Job execution completion message, Job execution are different
Normal message；Job Operations message generation unit 123, for according to Job execution status message, making it is determined that each operation is corresponding
Industry operation information；Wherein, in the case where the operation of an operation mutual exclusion is carrying out, the corresponding operation of this operation is determined
Operation information is pausing operation message；In the case of the operation of a job dependence has not carried out successfully, determine that this is made
The corresponding Job Operations message of industry is pausing operation message or terminates operation information；Job Operations message sending unit 124, is used
In transmission Job Operations message to operation queue 42.
Alternatively, Job execution container 2 includes：Job message receiving module 21, Job Operations message reception module 22, work
Industry performing module 23 and Job execution status message sending module 24, wherein, job message receiving module 21, for from operation team
Row 41 receive job message；Job Operations message reception module 22, for receiving Job Operations message from operation queue 42；Operation
Performing module 23, for the operation according to described by the corresponding job message of Job Operations message execution；Job execution state disappears
Sending module 24 is ceased, the execution status message for sending operation extremely monitors queue 43.
With reference to Fig. 2, alternatively, Job execution mould 23 includes：Thread-management unit 231, and by thread-management unit 231
The single thread or multithreading 232 of offer, wherein, thread-management unit 231 corresponding performs work for being provided for each operation
The single thread or multithreading 232 of industry；Single thread or multithreading 232, for performing corresponding operation.
It should be noted that above-mentioned term " module " or " unit " refer to the software that can realize predetermined function and/or
The combination of hardware.Although the device described by following examples is preferably realized with software, hardware, or software and hard
The realization of the combination of part is also that may and be contemplated.
Above-mentioned modules or unit can be by software or hardware to realize, for the latter, can by with
Under type is realized, but not limited to this：Above-mentioned module is respectively positioned in same processor；Or, above-mentioned module is located at multiple places respectively
Manage in device.
Embodiment 2
Present embodiments provide a kind of distributed job management method.Fig. 3 is distributed work according to embodiments of the present invention
The flow chart of industry management method, as shown in figure 3, the flow comprises the following steps：
Step S301, the User Interface provided by the job scheduling page, by the job message of operation, the work of operation
Industry operation information is sent to queue, and from queue obtain operation execution status message；
Job execution container on step S302, multiple nodes obtains job message and Job Operations message from queue,
And corresponding operation is performed according to job message and Job Operations message, and the execution status message of operation is sent to team
Row.
Alternatively, above-mentioned queue includes：Job queue, operation queue and monitoring queue, wherein, job queue is used for
The discrepancy queue operation of job message is handled, wherein, job message is used to indicate the type of operation and the scheduling mode of operation；Behaviour
Make queue, the discrepancy queue operation for handling Job Operations message, wherein, Job Operations message is used to indicate to hold operation
Row operation, termination operation or pausing operation；Monitor queue, the discrepancy queue operation for handling Job execution status message；
Alternatively, above-mentioned steps S301 includes：Job message is sent to job queue；Job Operations message is sent to operation
Queue；Job execution status message is received from monitoring queue.
Alternatively, the present embodiment can be determined according to the job relations between operation business execution sequencing or
Determine whether task performs.Above-mentioned transmission Job Operations message to operation queue includes：The corresponding job message of operation is parsed,
To obtain the job relations between multiple operations, wherein, job relations include：Dependence, mutex relation；Obtained from monitoring queue
The Job execution status message of industry is taken as, Job execution status message includes：Job execution completion message, Job execution disappear extremely
Breath；According to Job execution status message, it is determined that the corresponding Job Operations message of each operation；Wherein, in operation mutual exclusion
In the case that operation is carrying out, it is pausing operation message to determine the corresponding Job Operations message of this operation；In a work
In the case of the operation that industry is relied on has not carried out successfully, determine that the corresponding Job Operations message of this operation disappears for pausing operation
Breath terminates operation information；Job Operations message is sent to operation queue.
Alternatively, in step s 302, performing corresponding operation according to job message and Job Operations message includes：According to
Job message and Job Operations message provide the single thread or multithreading of corresponding execution operation for each operation；Use single line
Journey or multithreading perform corresponding operation.
Through the above description of the embodiments, those skilled in the art can be understood that according to above-mentioned implementation
The method of example can add the mode of required general hardware platform to realize by software, naturally it is also possible to by hardware, but a lot
In the case of the former be more preferably embodiment.Understood based on such, technical scheme is substantially in other words to existing
The part that technology contributes can be embodied in the form of software product, and the computer software product is stored in a storage
In medium (such as ROM/RAM, magnetic disc, CD), including some instructions are to cause a station terminal equipment (can be mobile phone, calculate
Machine, server, or network equipment etc.) perform method described in each of the invention embodiment.
Embodiments of the invention additionally provide a kind of software, and the software is used to perform above-described embodiment and preferred embodiment
Described in technical scheme.
Embodiments of the invention additionally provide a kind of storage medium.In the present embodiment, above-mentioned storage medium can be set
It is set to the program code that storage is used to perform following steps：
Step S301, the User Interface provided by the job scheduling page, by the job message of operation, the work of operation
Industry operation information is sent to queue, and from queue obtain operation execution status message；
Job execution container on step S302, multiple nodes obtains job message and Job Operations message from queue,
And corresponding operation is performed according to job message and Job Operations message, and the execution status message of operation is sent to team
Row.
Alternatively, in the present embodiment, above-mentioned storage medium can include but is not limited to：USB flash disk, read-only storage (Read-
Only Memory, referred to as ROM), it is random access memory (Random Access Memory, referred to as RAM), mobile hard
Disk, magnetic disc or CD etc. are various can be with the medium of store program codes.
Alternatively, the specific example in the present embodiment may be referred to described in above-described embodiment and optional embodiment
Example, the present embodiment will not be repeated here.
In order that the description of the embodiment of the present invention is clearer, it is described and illustrates with reference to preferred embodiment.
Embodiment 3
This preferred embodiment provides a kind of distributed job manager.Fig. 4 is according to the preferred embodiment of the invention point
The structural representation of cloth job manager, distributed job manager as shown in Figure 4 be for customizing Job (i.e. operation),
And Job scheduling mode, and the system for monitoring Job execution state.Distributed job manager is divided into JobSchedule
Web (equivalent to the above-mentioned job scheduling page), and JobExecution Container are (equivalent to above-mentioned Job execution
Container) two large divisions.Wherein JobSchedule Web are foreground web systems, for customizing Job, Job scheduling, monitoring etc. with
The part of user mutual.JobExecution Container are then a background systems, for performing Job service logic.
JobSchedule Web and JobExecution Container are interacted by queue.
With reference to Fig. 4, JobSchedule web include web front-end (showing as a web page), and Schedule
Service (dispatch service)；Queue (queue) then include operation Q (operation queue), Job Q (job queue) and
Monitor Q (monitoring queue)；JobExecution Container include：Envmonitor (environmental variance monitor),
Core (processing core), process (process pool).
Alternatively, web front-end is communicated by remote procedure call protocol (RPC) and Schedule Service.
ScheduleSevice includes：Executionmonitor (execution Status Monitor), config (configurator),
Jobmonitor (operation monitor) and schedule (scheduler).Config and Jobmonitor from Job RDBS, (close by operation
Be database) and Quartz RDBS (timed task frame relations database) read and write Job and Job timing task information；
Schedule is used to be separately added into Job Q by Job, to Job operation (such as terminating operation kill, pausing operation suspend)
With operation Q, and from monitor Q obtain Job implementing result；JobExecution Container execution
Cheng Zhong, the write-in such as environmental variance information, log information Redis, Schedule Service by executionmonitor from
Environmental variance information, log information are obtained in Redis.
Preferably, JobExecution Container can have multiple, be respectively distributed on multiple nodes；At this preferably
In embodiment, the JobExecution Container newly added are registered by registry (Register)；JobSchedule
Web then obtained by executionmonitor from registry whole JobExecution Container it is current whether
Can use or resources left situation status information, with realize to can use JobExecution Container scheduling.
Registry is additionally operable to carry out heartbeat monitor to registered JobExecution Container.
JobSchedule Web are the foreground partitions in distributed job manager.It is responsible for interacting with user, customization
Job Type, Job Schedule, and the state that monitoring Job is performed.Job execution state includes：It is scheduled
(scheduled) (waiting), is waited, joined the team (queued), failed (failed), having suspended in (suspend), pause
(suspending) (processing) in, performing, terminate in (killing), terminated (killed), and completed
(completed).Figure 5 illustrates the change over condition between each execution state, operation therein includes forced service
(forcerun) (resume), again operation (rerun), pause (suspend), and terminate (kill), are restarted.
In the preferred embodiment, Job scheduling is performed, and three steps have been classified as in JobSchedule web：
First, safeguard Job types.The execution of Job in JobSchedule subsystems, being will by JobSchedule Web
Job information is sent to Job Q, then captures Message by JobExecution Container, calls real Job business generation
Code goes what is performed.Here the Job Type safeguarded are the mark that JobExecution Container search service code.
Second, safeguard Job Schedule.That safeguards here is Job is scheduled execution in which type of mode.Wherein
Dependence is contained, mutex relation performs parameter etc..Job can Ad Hoc's (interim) or be moved in circles
It is performing or Timer (timing) type.JobSchedule is realized by Quartz and dispatched.
3rd, JobInstance (job instances).This similar Job Log concept, represents the Job of a logic, presses
According to schedule, the log recording once produced is performed.
In above-mentioned distributed job manager, Job is performed in shcedule web systems, and phase is simply sent in fact
The message answered is into queue.The execution of specific service logic, in JobExecution Container.
When performing Job, relied between multiple Job or mutex relation is meets, be then changed to Waiting states, wait
After the completion of other corresponding Job, trigger again.
For example, when the information monitored is Complete, JobInstance first is changed into Complete states.Divide again
Not Cha Xun mutual exclusion and the Waiting states dependent on this JobInstance JobInstance, then trigger these
JobInstance。
The judgement of dependence.Use and judged between the JobInstance generated at present.For example define
Job A, daily 11 points of execution define the daily 12 points of execution of Job B, and Job B depend on Job A.If that the A on the same day
Succeed, then B to 12 points normally performs；If A have failed, at B to 12, perform, but judge that dependence is not full
Foot, automatically becomes Waiting states.In turn, if defining A depends on B, then when A is performed, can go to judge whether B performs
Success, what now B judged is the result of 12 points of implementations of the previous day.
In the operation to operation, Job Kill, Suspend is completed eventually through thread pool.JobExecution
Container can be performed for each Job and be created a thread pool.The Kill of thread, Suspend can be completed by thread pool.
JobExecution Container are execution Job containers.Pass through FetchJobTimer (crawl operations
Timer) message is regularly scanned from Job Q, capture message.Create Job perform need Spring environment (using Java as
Example), then call Job service codes to be performed.
SimpleJob (simple operation) and ParallelJob is provided in JobExecution Container (parallel to make
Industry) two kinds of basic Job types.Business Job can selecting inheritance according to their needs.SimpleJob is simple Job, single line
Cheng Shunxu is performed；ParallelJob is performed using producer consumer modes in parallel.
Container can capture the exception dished out during Job is performed, and then send a message to MonitorQueue, notify
JobSchedule Web systems change JobInstance state.Container does not connect Jobschedule data directly
Storehouse.
In summary, the distributed job manager that the present invention is provided uses application multithreading to realize, can pass through increase
Node, improves the performance of Job execution.The operation for supporting heat deployment new, it is not necessary to restart original node.Homework type is more
It is abundant.Monitoring and configuration can also be patterned by the job scheduling page.
Distributed job manager provided in an embodiment of the present invention has further the advantage that：Operation uses application multithreading real
It is existing, it may operate in App Server.Operation can share performance requirement jointly using multinode, extending transversely easy.Number
Only it is used for data storage according to storehouse, independent of certain database.Performance issue concentrates on the I/O capability of storage and network
On, it is easy to solve performance issue.Using redis, increase cache policy, for reducing the performance issue that input and output are brought.Make
Industry performs node and supports Dynamical Deployment to unload new operation, it is not necessary to restart whole Job execution node.Operation is supported more
Action and state.Operation maintenance support is more configured：Regularly, dependence, mutual exclusion.Monitoring function is more comprehensive, the state of node,
Cpu, internal memory such as uses at the overall monitor.Monitoring operation experience is more preferable, increase job calendar, graphical, mobilism monitoring.
The preferred embodiments of the present invention are the foregoing is only, are not intended to limit the invention, for the skill of this area
For art personnel, the present invention can have various modifications and variations.Within the spirit and principles of the invention, that is made any repaiies
Change, equivalent substitution, improvement etc., should be included in the scope of the protection.
Claims (10)
1. a kind of distributed job manager, it is characterised in that including：The job scheduling page and multiple Job execution containers, it is described
Multiple Job execution containers are distributed on multiple nodes, and the job scheduling page and the multiple Job execution container pass through team
Row are interacted；Wherein,
The job scheduling page, for providing User Interface, to realize that operation customization, the scheduling mode of operation are determined
System, and the execution state of operation monitoring；
The Job execution container, for performing corresponding operation according to the scheduling mode of the job scheduling page customization, and will
The execution feedback of status of operation gives the job scheduling page.
2. distributed job manager according to claim 1, it is characterised in that the queue includes：Job queue, behaviour
Make queue and monitoring queue, wherein,
The job queue, the discrepancy queue operation for handling job message, wherein, the job message is used to indicate operation
Type and operation scheduling mode；
The operation queue, the discrepancy queue operation for handling Job Operations message, wherein, the Job Operations message is used for
Indicate to perform operation, termination operation or pausing operation to operation；
The monitoring queue, the discrepancy queue operation for handling Job execution status message.
3. distributed job manager according to claim 2, it is characterised in that the job scheduling page includes：Make
Industry message transmission module, Job Operations message transmission module and Job execution status message receiving module, wherein,
The job message sending module, for sending job message to the job queue；
The Job Operations message transmission module, for sending Job Operations message to the operation queue；
The Job execution status message receiving module, for receiving Job execution status message from the monitoring queue.
4. distributed job manager according to claim 3, it is characterised in that the Job Operations message transmission module
Including：Job's context resolution unit, Job execution status message receiving unit, Job Operations message generation unit, Yi Jizuo
Industry operation information transmitting element, wherein,
The job's context resolution unit, for parsing the corresponding job message of operation, to obtain the work between multiple operations
Industry relation, wherein, the job relations include：Dependence, mutex relation；
The Job execution status message receiving unit, the Job execution state for obtaining operation from the monitoring queue disappears
Breath, the Job execution status message includes：Job execution completion message, Job execution unexpected message；
The Job Operations message generation unit, for according to Job execution status message, it is determined that the corresponding operation of each operation
Operation information；Wherein, in the case where the operation of an operation mutual exclusion is carrying out, the corresponding operation behaviour of this operation is determined
Make message for pausing operation message；In the case of the operation of a job dependence has not carried out successfully, this operation is determined
Corresponding Job Operations message is pausing operation message or terminates operation information；
The Job Operations message sending unit, for sending Job Operations message to the operation queue.
5. distributed job manager according to claim 3, it is characterised in that the Job execution container includes：Make
Industry message reception module, Job Operations message reception module, operation execution module and Job execution status message sending module, its
In,
The job message receiving module, for receiving job message from the job queue；
The Job Operations message reception module, for receiving Job Operations message from the operation queue；
The operation execution module, for the operation according to described by the corresponding job message of Job Operations message execution；
The Job execution status message sending module, for sending the execution status message of operation to the monitoring queue.
6. distributed job manager according to claim 5, it is characterised in that the operation execution module includes：Line
Thread management unit, and the single thread or multithreading provided by the thread-management unit, wherein,
The thread-management unit, single thread or multithreading for providing corresponding execution operation for each operation；
The single thread or multithreading, for performing corresponding operation.
7. a kind of distributed job management method, it is characterised in that including：
The User Interface provided by the job scheduling page, the Job Operations message of the job message of operation, operation is sent out
Deliver to queue, and obtain from queue the execution status message of operation；
Job execution container on multiple nodes obtains job message and Job Operations message from queue, and according to job message
Corresponding operation is performed with Job Operations message, and the execution status message of operation is sent to queue.
8. distributed job management method according to claim 7, it is characterised in that the queue includes：Job queue,
Operation queue and monitoring queue, wherein, the job queue, the discrepancy queue operation for handling job message, wherein, it is described
Job message is used to indicate the type of operation and the scheduling mode of operation；The operation queue, for handling Job Operations message
Discrepancy queue operation, wherein, the Job Operations message be used for indicate to operation perform operation, terminate operation or pause
Operation；The monitoring queue, the discrepancy queue operation for handling Job execution status message；
The User Interface provided by the job scheduling page, the Job Operations message of the job message of operation, operation is sent out
Queue is delivered to, and the execution status message of acquisition operation includes from queue：Job message is sent to the job queue；Hair
Job Operations message is sent to the operation queue；Job execution status message is received from the monitoring queue.
9. distributed job management method according to claim 8, it is characterised in that send Job Operations message to described
Operation queue includes：
The corresponding job message of operation is parsed, to obtain the job relations between multiple operations, wherein, the job relations bag
Include：Dependence, mutex relation；
The Job execution status message of operation is obtained from the monitoring queue, the Job execution status message includes：Operation is held
Row completion message, Job execution unexpected message；
According to Job execution status message, it is determined that the corresponding Job Operations message of each operation；Wherein, in operation mutual exclusion
In the case that operation is carrying out, it is pausing operation message to determine the corresponding Job Operations message of this operation；In a work
In the case of the operation that industry is relied on has not carried out successfully, determine that the corresponding Job Operations message of this operation disappears for pausing operation
Breath terminates operation information；
Job Operations message is sent to the operation queue.
10. distributed job management method according to claim 7, it is characterised in that grasped according to job message and operation
Making the corresponding operation of message execution includes：
The single thread or multithreading of corresponding execution operation are provided according to job message and Job Operations message for each operation；
Corresponding operation is performed using the single thread or multithreading.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611154942.0A CN107066339A (en) | 2016-12-14 | 2016-12-14 | Distributed job manager and distributed job management method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611154942.0A CN107066339A (en) | 2016-12-14 | 2016-12-14 | Distributed job manager and distributed job management method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107066339A true CN107066339A (en) | 2017-08-18 |
Family
ID=59619749
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611154942.0A Pending CN107066339A (en) | 2016-12-14 | 2016-12-14 | Distributed job manager and distributed job management method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107066339A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109582446A (en) * | 2018-10-11 | 2019-04-05 | 中国建设银行股份有限公司 | Quasi real time asynchronous batch processing system, method, apparatus and storage medium |
| CN111221662A (en) * | 2019-10-16 | 2020-06-02 | 贝壳技术有限公司 | Task scheduling method, system and device |
| CN111221645A (en) * | 2019-11-14 | 2020-06-02 | 中国民航信息网络股份有限公司 | Aviation operation processing method and system |
| CN111245900A (en) * | 2019-12-31 | 2020-06-05 | 北京健康之家科技有限公司 | Distributed message sending processing system and processing method thereof |
| CN111340613A (en) * | 2020-02-26 | 2020-06-26 | 中国邮政储蓄银行股份有限公司 | Job processing method, system and storage medium |
| CN112200534A (en) * | 2020-09-24 | 2021-01-08 | 中国建设银行股份有限公司 | Method and device for managing time events |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101645022B (en) * | 2009-08-28 | 2013-07-10 | 曙光信息产业(北京)有限公司 | Work scheduling management system and method for a plurality of colonies |
| CN104657214A (en) * | 2015-03-13 | 2015-05-27 | 华存数据信息技术有限公司 | Multi-queue multi-priority big data task management system and method for achieving big data task management by utilizing system |
| CN105630588A (en) * | 2014-11-06 | 2016-06-01 | 卓望数码技术（深圳）有限公司 | Distributed job scheduling method and system |
| CN105760221A (en) * | 2016-02-02 | 2016-07-13 | 中博信息技术研究院有限公司 | Task dispatching system with distributed calculating frame |
-
2016
- 2016-12-14 CN CN201611154942.0A patent/CN107066339A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101645022B (en) * | 2009-08-28 | 2013-07-10 | 曙光信息产业(北京)有限公司 | Work scheduling management system and method for a plurality of colonies |
| CN105630588A (en) * | 2014-11-06 | 2016-06-01 | 卓望数码技术（深圳）有限公司 | Distributed job scheduling method and system |
| CN104657214A (en) * | 2015-03-13 | 2015-05-27 | 华存数据信息技术有限公司 | Multi-queue multi-priority big data task management system and method for achieving big data task management by utilizing system |
| CN105760221A (en) * | 2016-02-02 | 2016-07-13 | 中博信息技术研究院有限公司 | Task dispatching system with distributed calculating frame |
Non-Patent Citations (1)
| Title |
|---|
| 包振强,等；: "分布式混合优化调度方法研究", 《中国机械工程》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109582446A (en) * | 2018-10-11 | 2019-04-05 | 中国建设银行股份有限公司 | Quasi real time asynchronous batch processing system, method, apparatus and storage medium |
| CN111221662A (en) * | 2019-10-16 | 2020-06-02 | 贝壳技术有限公司 | Task scheduling method, system and device |
| CN111221645A (en) * | 2019-11-14 | 2020-06-02 | 中国民航信息网络股份有限公司 | Aviation operation processing method and system |
| CN111245900A (en) * | 2019-12-31 | 2020-06-05 | 北京健康之家科技有限公司 | Distributed message sending processing system and processing method thereof |
| CN111245900B (en) * | 2019-12-31 | 2021-09-14 | 北京健康之家科技有限公司 | Distributed message sending processing system and processing method thereof |
| CN111340613A (en) * | 2020-02-26 | 2020-06-26 | 中国邮政储蓄银行股份有限公司 | Job processing method, system and storage medium |
| CN111340613B (en) * | 2020-02-26 | 2023-10-03 | 中国邮政储蓄银行股份有限公司 | Job processing method, job processing system and storage medium |
| CN112200534A (en) * | 2020-09-24 | 2021-01-08 | 中国建设银行股份有限公司 | Method and device for managing time events |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107066339A (en) | Distributed job manager and distributed job management method | |
| US10565034B2 (en) | Event-driven serverless function orchestration | |
| CN107729139B (en) | Method and device for concurrently acquiring resources | |
| US8713163B2 (en) | Monitoring cloud-runtime operations | |
| CN108733461B (en) | Distributed task scheduling method and device | |
| US20140101665A1 (en) | Operation control for deploying and managing software service in a virtual environment | |
| CN107918556A (en) | A kind of timed task performs method and apparatus in the parallel of multiple servers | |
| US8538793B2 (en) | System and method for managing real-time batch workflows | |
| CN109656782A (en) | Visual scheduling monitoring method, device and server | |
| CN108243012B (en) | Charging application processing system, method and device in OCS (online charging System) | |
| US11704616B2 (en) | Systems and methods for distributed business processmanagement | |
| WO2019200984A1 (en) | Life cycle management method for distributed application, managers, device and medium | |
| CN109783255B (en) | Data analysis and distribution device and high-concurrency data processing method | |
| US20230029198A1 (en) | Scheduling complex jobs in a distributed network | |
| CN110177146A (en) | A kind of non-obstruction Restful communication means, device and equipment based on asynchronous event driven | |
| US20220182851A1 (en) | Communication Method and Apparatus for Plurality of Administrative Domains | |
| CN112162852A (en) | Multi-architecture CPU node management method, device and related components | |
| CN102420709A (en) | Method and equipment for managing scheduling task based on task frame | |
| CN111190732A (en) | Timed task processing system and method, storage medium and electronic device | |
| CN112667383A (en) | Task execution and scheduling method, system, device, computing equipment and medium | |
| CN108399095B (en) | Method, system, device and storage medium for supporting dynamic management of timed tasks | |
| CN114090198A (en) | Distributed task scheduling method and device, electronic equipment and storage medium | |
| CN108984290A (en) | Method for scheduling task and system | |
| CN117076096A (en) | Task flow execution method and device, computer readable medium and electronic equipment | |
| CN113934517A (en) | Timing task scheduling method and device, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170818 |