CN116719672A - Job scheduling control method, system, equipment and storage medium - Google Patents

Job scheduling control method, system, equipment and storage medium Download PDF

Info

Publication number
CN116719672A
CN116719672A CN202310686348.XA CN202310686348A CN116719672A CN 116719672 A CN116719672 A CN 116719672A CN 202310686348 A CN202310686348 A CN 202310686348A CN 116719672 A CN116719672 A CN 116719672A
Authority
CN
China
Prior art keywords
job
scheduling
task
dispatching
executable
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310686348.XA
Other languages
Chinese (zh)
Inventor
田琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jinan Inspur Data Technology Co Ltd
Original Assignee
Jinan Inspur Data Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jinan Inspur Data Technology Co Ltd filed Critical Jinan Inspur Data Technology Co Ltd
Priority to CN202310686348.XA priority Critical patent/CN116719672A/en
Publication of CN116719672A publication Critical patent/CN116719672A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1415Saving, restoring, recovering or retrying at system level
    • G06F11/1438Restarting or rejuvenating
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1474Saving, restoring, recovering or retrying in transactions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3051Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G06F9/4887Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues involving deadlines, e.g. rate based, periodic

Abstract

The embodiment of the application provides a job scheduling management and control method, a system, equipment and a storage medium, which are used for solving the technical problem that the existing job scheduling scheme cannot meet the actual application requirements. Comprising the following steps: receiving a scheduling job request instruction constructed by a user, and storing the content of a scheduling job contained in the scheduling job request instruction into a job warehouse; registering an executable job scope to a registration center, and scheduling the scheduling job at regular time based on the executable job scope; and monitoring the progress condition of the main service, and under the condition that the main service is down, synchronizing the content of the dispatching job from the job warehouse, and restarting the dispatching process of the dispatching job. The scheme supports various timing trigger strategies, task failure retry mechanisms, operation dynamic fragmentation mechanisms, executor thread pool isolation, queue protection and the like. The method can reduce the service development cost, the operation and maintenance cost and the deployment cost, and can effectively promote the sustainable integration of related services.

Description

Job scheduling control method, system, equipment and storage medium
Technical Field
The present application relates to the field of job scheduling technologies, and in particular, to a job scheduling management and control method, system, device, and storage medium.
Background
At present, a plurality of solutions exist for a job scheduling framework, but the framework is taken out as a middleware, and supports the characteristics of full link tracking, task overflow prevention and the like, and is not common. In the field of micro-services, the introduction of a good job scheduling middleware generally reduces a great deal of research, development and operation and maintenance work.
The common job scheduling framework makes great efforts on a distributed level, expands the requirements on micro-service and remote procedure calling directions, but is too centralized for the three ends of the task, and easily causes the service quality problem of a self-scheduling center caused by client realization, thereby causing the chain reaction of downtime of other micro-service jobs. Therefore, a scientific and reasonable job scheduling management and control method is not available at present.
Disclosure of Invention
The embodiment of the application provides a job scheduling management and control method, a system, equipment and a storage medium, which are used for solving the technical problem that the existing job scheduling scheme cannot meet the actual application requirements.
In one aspect, an embodiment of the present application provides a job scheduling management and control method, where the method includes: receiving a scheduling job request instruction constructed by a user, and storing the content of a scheduling job contained in the scheduling job request instruction into a job warehouse; registering an executable job scope to a registration center, and scheduling the scheduling job at regular time based on the executable job scope; and monitoring the progress condition of the main service, and under the condition that the main service is down, synchronizing the content of the dispatching job from the job warehouse, and restarting the dispatching process of the dispatching job.
In an implementation manner of the present application, according to the foregoing method for managing and controlling job scheduling, before the receiving a scheduling job request instruction constructed by a user, storing the content of a scheduling job included in the scheduling job request instruction in a job repository, the method further includes: embedding the information of the user into a client; submitting attributes required by the operation of the scheduling job at the client; wherein, the attribute required by the operation of the scheduling job comprises: job ID attribute, trigger attribute, and sharer attribute.
In an implementation manner of the present application, according to the foregoing method for managing and controlling job scheduling, the operation of the scheduled job further needs to submit a job policy; wherein the job policies include, but are not limited to: log recording strategy, task backlog prevention strategy, job execution recording strategy, failure retry strategy and current limiting strategy.
In an implementation manner of the present application, according to the foregoing method for managing and controlling job scheduling, the timing job scheduling specifically includes: splitting the scheduling job into a plurality of job tasks; distributing the job task to an executor; and recording the expected timeout time of the job task during distribution, and forcibly ending the job task after the expected timeout time is reached.
In an implementation manner of the present application, according to the foregoing method for managing and controlling job scheduling, the method further includes: tracking the execution state of the operation task based on a task tracker; monitoring whether the execution of the job task fails, and retrying after the failure; and reporting the execution state of the job task and the running log in real time.
In an implementation manner of the present application, according to the foregoing method for managing and controlling job scheduling, after the distributing the job task to the executor, the method further includes: recording the job task to be executed into a task queue; recording the execution time of the job task, if the execution time is not over, forcibly ending the corresponding thread, and removing the job task from the task queue.
In an implementation manner of the present application, according to the foregoing method for managing and controlling job scheduling, the method further includes: monitoring the operation of the job task; and limiting the current of all the job tasks based on a preset current limiting strategy.
Secondly, the embodiment of the application also provides a job scheduling management and control system, which comprises: the job storage unit is used for receiving a dispatching job request instruction constructed by a user and storing the content of a dispatching job contained in the dispatching job request instruction into a job warehouse; a job scheduling unit configured to register an executable job range to a registry, and schedule the scheduled job at regular time based on the executable job range; and the downtime protection unit is used for monitoring the progress condition of the main service, synchronizing the content of the dispatching job from the job warehouse and restarting the dispatching process of the dispatching job under the condition that the main service is downtime.
Still further, an embodiment of the present application further provides a job scheduling management and control apparatus, where the apparatus includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to: receiving a scheduling job request instruction constructed by a user, and storing the content of a scheduling job contained in the scheduling job request instruction into a job warehouse; registering an executable job scope to a registration center, and scheduling the scheduling job at regular time based on the executable job scope; and monitoring the progress condition of the main service, and under the condition that the main service is down, synchronizing the content of the dispatching job from the job warehouse, and restarting the dispatching process of the dispatching job.
Finally, an embodiment of the present application further provides a job scheduling management non-volatile computer storage medium, storing computer executable instructions, where the computer executable instructions are configured to: receiving a scheduling job request instruction constructed by a user, and storing the content of a scheduling job contained in the scheduling job request instruction into a job warehouse; registering an executable job scope to a registration center, and scheduling the scheduling job at regular time based on the executable job scope; and monitoring the progress condition of the main service, and under the condition that the main service is down, synchronizing the content of the dispatching job from the job warehouse, and restarting the dispatching process of the dispatching job.
The job scheduling management and control method, the system, the equipment and the storage medium provided by the embodiment of the application can realize a simple and easy-to-use scheduling center, can dynamically modify the job without paying attention to HA scenes, and is additionally provided with an HA scheme of an actuator, support elastic capacity expansion and contraction, support various timing trigger strategies, a task failure retry mechanism, a job dynamic fragmentation mechanism, a task routing fault transfer mechanism, real-time operation tracking and monitoring, full-link asynchronous scheduling, task overflow prevention, and isolation and queue protection of an actuator thread pool. Meanwhile, the technical scheme of the application separates the points of interest from the three ends (the client, the scheduling end and the executing end), so that the service development cost, the operation and maintenance cost and the deployment cost can be reduced, and meanwhile, the sustainable integration of related services can be effectively improved due to the separation of the points of interest.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a flow chart of a job scheduling control method according to an embodiment of the present application;
FIG. 2 is a diagram of a job scheduling management and control strategy architecture according to an embodiment of the present application;
FIG. 3 is a diagram illustrating a system for managing and controlling job scheduling according to an embodiment of the present application;
fig. 4 is a schematic diagram of a job scheduling control device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
At present, a plurality of solutions exist for a job scheduling framework, but the framework is taken out as a middleware, and supports the characteristics of full link tracking, task overflow prevention and the like, and is not common. In the field of micro-services, we are concerned about the separation of the points of interest, and the introduction of a good job scheduling middleware generally reduces a great deal of research, development and operation and maintenance work. Commonly seen job scheduling frameworks are Quartz, xx-job, and the like. The Quartz makes a lot of important contributions and guides in the time round scheduling direction, can be used as a development framework, is static for job submission, namely, the operation method of each job is consistent, and cannot be directly used as a middleware, and needs secondary development. The xx-job makes great efforts on a distributed level, expands the requirements on micro service and RPC directions, but is too centralized on three ends (a client, a scheduling end and an executing end) of a task, and easily causes the service quality problem of a self-scheduling center caused by client realization, thereby causing chain reactions of downtime of other micro service operations.
Therefore, development of a job scheduling framework is needed, which can be completely separated from a service, can be selectively and directly deployed as a middleware, flexibly meets the requirements of single and distributed assembly, supports flexible capacity expansion and contraction, supports various timing trigger strategies, task failure retry mechanisms, job dynamic fragmentation mechanisms, task routing fault transfer mechanisms, real-time operation tracking and monitoring, full-link asynchronous scheduling, task overflow prevention, executor thread pool isolation, queue protection and other characteristics.
The embodiment of the application is mainly applied to a large-scale distributed or single job scheduling scene, and aims to improve the scheduling efficiency and stability of threads and realize the accurate control of tasks.
The method and the device are used for job scheduling control, so that a simple and easy-to-use scheduling center can be realized, a user does not need to pay attention to an HA scene, the job can be dynamically modified, an HA scheme with an actuator is additionally arranged, the flexible capacity expansion and contraction are supported, various timing trigger strategies, a task failure retry mechanism, a job dynamic fragmentation mechanism, a task routing failover mechanism, real-time operation tracking and monitoring, full-link asynchronous scheduling, task overflow prevention, and actuator thread pool isolation and queue protection are supported.
Meanwhile, the technical scheme of the application separates the points of interest from the three ends (the client, the scheduling end and the executing end), so that the service development cost, the operation and maintenance cost and the deployment cost can be reduced, and meanwhile, the sustainable integration of related services can be effectively improved due to the separation of the points of interest.
The embodiment of the application provides a job scheduling management and control method, a system, equipment and a storage medium, which are used for solving the technical problem that the existing job scheduling scheme cannot meet the actual application requirements.
The following describes the technical scheme provided by the embodiment of the application in detail through the attached drawings.
Fig. 1 is a flowchart of a job scheduling control method according to an embodiment of the present application. As shown in fig. 1, the method mainly comprises the following steps:
step 101, receiving a scheduling job request instruction constructed by a user, and storing the content of the scheduling job contained in the scheduling job request instruction into a job warehouse.
In the embodiment of the application, the protocol for managing the job scheduling comprises a client, a scheduling end and an executing end, wherein the client is embedded into the business process of the client and is responsible for submitting a job object, formulating a job strategy, modifying, deleting, inquiring the job and the like.
The dispatching end device adopts a central design, supports cluster deployment, is responsible for tracking the whole life cycle of the job and the task, periodically dispatches the operation of the job, and dynamically fragments the job into the task, and supports the dynamic capacity expansion and contraction of the execution end, the execution request of the load balancing task, the task overflow protection and the task overtime limitation.
The execution end device is responsible for feeding back an execution request, an execution result, a running log and the like sent by the stateless execution scheduling end, and simultaneously supports the execution thread of the task to be controlled according to the limited timeout time, so that the real execution efficiency of the task is ensured.
The basic idea of the job scheduling control scheme provided by the embodiment of the application is as follows: the core object of the scheme is a job object, and the job object comprises timing attributes, a fragment attribute, job variables, a job execution strategy and the like. The client device is responsible for helping a client construct a job and simultaneously providing the realization of a job slicing interface, so that the dynamic slicing of the job can be realized. The dispatching end adopts a central design, regularly dispatches the operation of the job, tracks the whole life cycle of the job and the task, and controls the anti-overflow of the task. The execution end device is responsible for executing the execution request sent by the stateless execution scheduling end, feeding back the execution result, the running log and the like. Meanwhile, the utilization of the thread is controlled for the limited timeout time of the task.
In the embodiment of the application, before job scheduling is performed, the information of the user is firstly required to be embedded into the client device, and then various attributes required by job operation are submitted, wherein the attributes required by the scheduled job operation comprise: job ID attribute, trigger attribute, and sharer attribute.
Further, when the user builds the scheduling job, the user must set the attribute of the job ID attribute, the trigger attribute and the fragment attribute, and meanwhile, the operation of the scheduling job also needs to submit a job strategy; wherein the job policies include, but are not limited to: log recording strategy, task backlog prevention strategy, job execution recording strategy, failure retry strategy and current limiting strategy.
Further, the user-constructed scheduling job is submitted to the scheduling end based on the remote procedure call protocol. Meanwhile, a series of job adding, deleting and checking functions are provided for meeting the job operation focused by the client.
In the embodiment of the application, a dispatching end receives a dispatching job request instruction of a user and stores the content of a dispatching job contained in the dispatching job request instruction into a job warehouse. The job information submitted by the user needs to be persisted into the database, waiting for the scheduler to schedule the operation of the job at regular time.
In an embodiment of the application, the scheduler schedules job runs based on the time wheel.
Further, the scheme defines that the scheduling job object should be split into job tasks for execution, and the job tasks are the minimum execution units of the framework. This process requires the user to implement the shard interface at his own discretion. It should be noted that, since the job slicing operation occurs every time the job is scheduled, there is no need to make excessive worry about the problem of parameter synchronization of the task.
Step 102, registering an executable job range to a registration center, and scheduling the scheduling job at regular time based on the executable job range.
In the embodiment of the application, the method for scheduling the jobs at fixed time specifically comprises the following steps: splitting the scheduling job into a plurality of job tasks; distributing the job task to an executor; and recording the expected timeout time of the job task during distribution, and forcibly ending the job task after the expected timeout time is reached.
Further, tracking the execution state of the job task based on a task tracker; monitoring whether the execution of the job task fails, and retrying after the failure; and reporting the execution state of the job task and the running log in real time.
Further, after the job task is distributed to the executor, the job task to be executed is recorded in a task queue; recording the execution time of the job task, if the execution time is not over, forcibly ending the corresponding thread, and removing the job task from the task queue.
In the embodiment of the application, after the tasks of the user are all fragmented, the distributed tasks are processed by the executor. In practice, the portion is defined by the dispatch interface, which can be implemented by providing a default asynchronous dispatcher by means of a delay queue, and by the job execution tracker interface intervening in monitoring the running end of the job, while the task tracker interface intervening in monitoring the beginning and end of the task; during this distribution process, the distribution of tasks may be throttled according to a throttling policy.
It should be noted that, in the scheduling job and job task tracking mechanism set in the embodiment of the present application, the purpose is to record the expected execution timeout time of the task when the task is distributed, and when the timeout time is reached, if the end flag sent by the executor is not received yet, the abnormal end of the task is set. The tasks which are not finished can be removed after the job fragments are saved, and the tasks can only execute one same task at the same time, so that the tasks are particularly useful in the fields of hardware monitoring and the like.
In the embodiment of the application, a user is embedded into a business process from an execution end, the execution end registers an executable task range to a registration center, and a scheduling end routes an execution request to the execution end for execution based on the executable task range; the execution end is used for constructing a thread pool by user definition based on different tasks, and isolating the execution of the tasks.
The task tracker tracks the task execution state, retries the task after the task execution failure based on a user submitting strategy, and reports the execution state and the running log of the task.
After the task is issued to the execution end, the task to be executed is recorded into a task queue, and the task thread pool needs to be realized in a queue-free form. And recording the expected execution timeout time of the task, and forcibly ending the corresponding thread if the execution timeout time is not yet finished. If there is no thread corresponding to the current task, the task is removed from the task queue.
And distributing the job to be scheduled to a priority task queue or a timing task queue according to the trigger type of the job to be scheduled. And distributing the job according to the trigger type of the job by calling a job distributor, and adding the job into a priority task queue to wait for execution if the job is a queue task. If the job is a timed trigger task, it is added to the timed task queue. The timer of the timing task queue is mainly responsible for executing the timing tasks, and the timing tasks can be divided into system-level timing tasks and user-level timing tasks. The system-level timed tasks are added as the system is running, and tasks that have run and time out can be scanned and processed by invoking the task monitor.
And creating the job task to be executed of the job to be scheduled according to the task information sent by the priority task queue or the timing trigger information sent by the timing task queue. Scheduling and allocation of execution of tasks may be accomplished by invoking a task tracker. The task tracker may create a specific job task according to the scheduled job, and select a specific executor to execute the corresponding job task, monitor execution of the task, and notify a task execution result according to the type of the executor, e.g., serial, parallel, etc.
By setting the priority task queue, the task of the scheduling job with higher priority in the task center can be guaranteed to be executed preferentially, the adaptability of the scheme architecture can be improved on the whole, and the efficiency of job scheduling is improved.
The job scheduling architecture provided by the embodiment of the application is shown in fig. 2. The job creator gathers the job ID attributes, trigger attributes, and sharer attributes, and then transmits from the job dispatcher to the job dispatcher where it enters the job tracker and saves to the job repository, respectively. Then, the scheduling job is subjected to job slicing, task distribution and then the job tracking. Finally, the execution device route and thread pool isolation reach the execution device to execute the scheduling of the job.
And 103, monitoring the progress condition of the main service, and synchronizing the content of the dispatching job from the job warehouse and restarting the dispatching process of the dispatching job under the condition that the main service is down.
In the embodiment of the application, under a distributed scene, the dispatching end can ensure the HA of the dispatching center by means of the cluster selection main component. The cluster owner selection can be realized by means of a registration center or a distributed session lock of other components, and the scheme is not repeated. When the main service is down, the standby service can synchronize the job content from the job warehouse and restart the job scheduling.
The job scheduling control method provided by the embodiment of the application can realize a simple and easy-to-use scheduling center, a user does not need to pay attention to an HA scene, the job can be dynamically modified, an HA scheme with an embedded executor is supported, the flexible capacity expansion and contraction are supported, various timing trigger strategies, a task failure retry mechanism, a job dynamic fragmentation mechanism, a task routing fault transfer mechanism, the real-time operation tracking and monitoring of the job, the full-link asynchronous scheduling, the task overflow prevention, the executor thread pool isolation and the queue protection are supported. Meanwhile, the technical scheme of the application separates the points of interest from the three ends (the client, the scheduling end and the executing end), so that the service development cost, the operation and maintenance cost and the deployment cost can be reduced, and meanwhile, the sustainable integration of related services can be effectively improved due to the separation of the points of interest.
The above is a job scheduling control method provided by the embodiment of the present application, based on the same inventive concept, the embodiment of the present application further provides a job scheduling control system, and fig. 3 is a system composition diagram provided by the embodiment of the present application, as shown in fig. 3, where the system mainly includes: a job saving unit 301, configured to receive a scheduling job request instruction constructed by a user, and save the content of a scheduling job included in the scheduling job request instruction to a job repository; a job scheduling unit 302, configured to register an executable job range to a registry, and schedule the scheduled job at regular time based on the executable job range; and the downtime protection unit 303 is configured to monitor a progress of a main service, synchronize contents of the scheduled job from the job repository, and restart a scheduling process of the scheduled job in the event that the main service is downtime.
In the embodiment of the present application, the job scheduling unit 302 is specifically configured to perform: splitting the scheduling job into a plurality of job tasks; distributing the job task to an executor; and recording the expected timeout time of the job task during distribution, and forcibly ending the job task after the expected timeout time is reached.
Further, the job scheduling unit 302 is further configured to perform: tracking the execution state of the operation task based on a task tracker; monitoring whether the execution of the job task fails, and retrying after the failure; and reporting the execution state of the job task and the running log in real time.
Further, the job task to be executed is recorded in a task queue; recording the execution time of the job task, if the execution time is not over, forcibly ending the corresponding thread, and removing the job task from the task queue.
The job scheduling management and control system provided by the embodiment of the application can realize a simple and easy-to-use scheduling center, a user does not need to pay attention to an HA scene, the job can be dynamically modified, an HA scheme with an embedded executor is supported, the flexible capacity expansion and contraction are supported, various timing triggering strategies, a task failure retry mechanism, a job dynamic fragmentation mechanism, a task routing fault transfer mechanism, the real-time operation tracking and monitoring of the job, the full-link asynchronous scheduling, the task overflow prevention, the executor thread pool isolation and the queue protection are supported. Meanwhile, the technical scheme of the application separates the points of interest from the three ends (the client, the scheduling end and the executing end), so that the service development cost, the operation and maintenance cost and the deployment cost can be reduced, and meanwhile, the sustainable integration of related services can be effectively improved due to the separation of the points of interest.
The foregoing is a job scheduling management and control system provided by the embodiment of the present application, and based on the same inventive concept, the embodiment of the present application further provides a job scheduling management and control device, and fig. 4 is a schematic diagram of a job scheduling management and control device provided by the embodiment of the present application, where, as shown in fig. 4, the device mainly includes: at least one processor 401; and a memory 402 communicatively coupled to the at least one processor; wherein the memory 402 stores instructions executable by the at least one processor 401, the instructions being executable by the at least one processor 401 to enable the at least one processor 401 to: receiving a scheduling job request instruction constructed by a user, and storing the content of a scheduling job contained in the scheduling job request instruction into a job warehouse; registering an executable job scope to a registration center, and scheduling the scheduling job at regular time based on the executable job scope; and monitoring the progress condition of the main service, and under the condition that the main service is down, synchronizing the content of the dispatching job from the job warehouse, and restarting the dispatching process of the dispatching job.
In addition, the embodiment of the application also provides a non-volatile computer storage medium for job scheduling management, which stores computer executable instructions, wherein the computer executable instructions are configured to: receiving a scheduling job request instruction constructed by a user, and storing the content of a scheduling job contained in the scheduling job request instruction into a job warehouse; registering an executable job scope to a registration center, and scheduling the scheduling job at regular time based on the executable job scope; and monitoring the progress condition of the main service, and under the condition that the main service is down, synchronizing the content of the dispatching job from the job warehouse, and restarting the dispatching process of the dispatching job.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The embodiments of the present application are described in a progressive manner, and the same and similar parts of the embodiments are all referred to each other, and each embodiment is mainly described in the differences from the other embodiments. In particular, for the device embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of the method embodiments in part.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.

Claims (10)

1. A method for job scheduling management and control, the method comprising:
receiving a scheduling job request instruction constructed by a user, and storing the content of a scheduling job contained in the scheduling job request instruction into a job warehouse;
registering an executable job scope to a registration center, and scheduling the scheduling job at regular time based on the executable job scope;
and monitoring the progress condition of the main service, and under the condition that the main service is down, synchronizing the content of the dispatching job from the job warehouse, and restarting the dispatching process of the dispatching job.
2. The job scheduling control method according to claim 1, wherein before the receiving the user-constructed scheduling job request instruction, saving the content of the scheduling job contained in the scheduling job request instruction to a job repository, the method further comprises:
embedding the information of the user into a client;
submitting attributes required by the operation of the scheduling job at the client; wherein, the attribute required by the operation of the scheduling job comprises: job ID attribute, trigger attribute, and sharer attribute.
3. The job scheduling control method according to claim 2, wherein the operation of the scheduled job further requires submitting a job policy; wherein the job policies include, but are not limited to: log recording strategy, task backlog prevention strategy, job execution recording strategy, failure retry strategy and current limiting strategy.
4. The job scheduling control method according to claim 1, wherein the timing scheduling job specifically comprises:
splitting the scheduling job into a plurality of job tasks;
distributing the job task to an executor;
and recording the expected timeout time of the job task during distribution, and forcibly ending the job task after the expected timeout time is reached.
5. The job scheduling management method according to claim 4, wherein the method further comprises:
tracking the execution state of the operation task based on a task tracker;
monitoring whether the execution of the job task fails, and retrying after the failure;
and reporting the execution state of the job task and the running log in real time.
6. The job scheduling management method according to claim 4, wherein after the distributing the job task to an actuator, the method further comprises:
recording the job task to be executed into a task queue;
recording the execution time of the job task, if the execution time is not over, forcibly ending the corresponding thread, and removing the job task from the task queue.
7. The job scheduling management method according to claim 4, wherein the method further comprises:
monitoring the operation of the job task;
and limiting the current of all the job tasks based on a preset current limiting strategy.
8. A job scheduling management and control system, the system comprising:
the job storage unit is used for receiving a dispatching job request instruction constructed by a user and storing the content of a dispatching job contained in the dispatching job request instruction into a job warehouse;
a job scheduling unit configured to register an executable job range to a registry, and schedule the scheduled job at regular time based on the executable job range;
and the downtime protection unit is used for monitoring the progress condition of the main service, synchronizing the content of the dispatching job from the job warehouse and restarting the dispatching process of the dispatching job under the condition that the main service is downtime.
9. A job scheduling management and control apparatus, the apparatus comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores instructions executable by the at least one processor to enable the at least one processor to:
receiving a scheduling job request instruction constructed by a user, and storing the content of a scheduling job contained in the scheduling job request instruction into a job warehouse;
registering an executable job scope to a registration center, and scheduling the scheduling job at regular time based on the executable job scope;
and monitoring the progress condition of the main service, and under the condition that the main service is down, synchronizing the content of the dispatching job from the job warehouse, and restarting the dispatching process of the dispatching job.
10. A non-transitory computer storage medium storing computer-executable instructions for job scheduling management, the computer-executable instructions configured to:
receiving a scheduling job request instruction constructed by a user, and storing the content of a scheduling job contained in the scheduling job request instruction into a job warehouse;
registering an executable job scope to a registration center, and scheduling the scheduling job at regular time based on the executable job scope;
and monitoring the progress condition of the main service, and under the condition that the main service is down, synchronizing the content of the dispatching job from the job warehouse, and restarting the dispatching process of the dispatching job.
CN202310686348.XA 2023-06-09 2023-06-09 Job scheduling control method, system, equipment and storage medium Pending CN116719672A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310686348.XA CN116719672A (en) 2023-06-09 2023-06-09 Job scheduling control method, system, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310686348.XA CN116719672A (en) 2023-06-09 2023-06-09 Job scheduling control method, system, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116719672A true CN116719672A (en) 2023-09-08

Family

ID=87872889

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310686348.XA Pending CN116719672A (en) 2023-06-09 2023-06-09 Job scheduling control method, system, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116719672A (en)

Similar Documents

Publication Publication Date Title
CN112379995B (en) DAG-based unitized distributed scheduling system and method
CN106844198B (en) Distributed dispatching automation test platform and method
CN111506412B (en) Airflow-based distributed asynchronous task construction and scheduling system and method
US8171474B2 (en) System and method for managing, scheduling, controlling and monitoring execution of jobs by a job scheduler utilizing a publish/subscription interface
US8301935B2 (en) Distributed batch runner
US20010018710A1 (en) System and method for improved automation of a computer network
CN110611707B (en) Task scheduling method and device
CN107168777B (en) Method and device for scheduling resources in distributed system
CN112910937B (en) Object scheduling method and device in container cluster, server and container cluster
US20030135617A1 (en) Intelligent system control agent
JPH06243070A (en) Inter-processor communication system
CN113630438A (en) Stream processing task scheduling method and distributed stream processing system
US7424712B1 (en) System and method for controlling co-scheduling of processes of parallel program
CN113032125A (en) Job scheduling method, device, computer system and computer-readable storage medium
CN109697112A (en) Distributed intensive one-stop operating system and implementation method
CN116719672A (en) Job scheduling control method, system, equipment and storage medium
CN112269647A (en) Node scheduling, switching and coordinating method and corresponding device, equipment and medium thereof
JP2003167746A (en) Software distribution method, execution system for the same and processing program for the same
CN113419835A (en) Job scheduling method, device, equipment and medium
CN109634749B (en) Distributed unified scheduling method and device
CN113127158B (en) Method and device for executing data processing task
CN114244894A (en) Halt and recovery service processing method and system, computer storage medium and electronic equipment
JP3660376B2 (en) Distributed processing system and load balancing method in distributed processing system
CN112685155A (en) Computing task scheduling method, device, system and storage medium
KR100321408B1 (en) Real-time processing system based on data processing sequence according to processing time and its processing method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination