CN117762587A - Task scheduling method, device, terminal equipment and storage medium - Google Patents
Task scheduling method, device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN117762587A CN117762587A CN202311769303.5A CN202311769303A CN117762587A CN 117762587 A CN117762587 A CN 117762587A CN 202311769303 A CN202311769303 A CN 202311769303A CN 117762587 A CN117762587 A CN 117762587A
- Authority
- CN
- China
- Prior art keywords
- task
- atomic
- atomic tasks
- tasks
- task scheduling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 230000011218 segmentation Effects 0.000 claims abstract description 20
- 238000012544 monitoring process Methods 0.000 claims description 70
- 230000007246 mechanism Effects 0.000 claims description 54
- 238000012216 screening Methods 0.000 claims description 10
- 238000004458 analytical method Methods 0.000 claims description 7
- 238000007726 management method Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000007405 data analysis Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000012423 maintenance Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 108090000623 proteins and genes Proteins 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a task scheduling method, a device, terminal equipment and a storage medium, wherein the method comprises the following steps: based on the task scheduling request, confirming the total workload of the original task; based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks; setting runway quantity according to the plurality of atomic tasks and the system resources; and performing task scheduling according to the runway quantity based on the runway model. Based on the scheme, the method and the device can ensure the effective utilization of the computer system resources and improve the efficiency and stability of task scheduling.
Description
Technical Field
The present invention relates to the field of system operation and maintenance, and in particular, to a task scheduling method, device, terminal device, and storage medium.
Background
In the field of computers, how to improve the performance and efficiency of a computer system is a key problem, which determines the working efficiency of the computer system, and the working efficiency of the computer system affects the development of various fields of society, such as gene sequencing, climate prediction, big data analysis, artificial intelligence, virtual reality and the like. To solve this problem, task scheduling methods may be employed to efficiently manage tasks, utilize resources, balance loads, and thereby improve computer system performance and efficiency.
When a computer system executes a plurality of tasks, the task scheduling method schedules and optimizes the execution sequence of the tasks according to a certain strategy so as to reduce resource conflict and waiting time and improve the throughput and performance of the system, wherein the throughput refers to the number of the tasks or requests processed by the system in unit time. Which represents the processing capacity or throughput of the system.
Because tasks in a computer system are performed by threads, existing task scheduling methods often rely on thread scheduling by the computer system. In a multi-thread environment, a task scheduling method needs to consider the state of a thread and the resource requirement to determine the thread to be run and execute the corresponding task. However, the existing task scheduling method lacks effective management and control on thread creation, thread waiting and thread execution, is relatively easy to bring challenges to thread scheduling, management and memory management of a computer system, is easy to cause memory overflow of the computer system for a scene requiring executing a large number of tasks, and cannot ensure that task scheduling is efficiently and stably executed.
In summary, the existing task scheduling method has the problem of low task scheduling efficiency and stability.
Disclosure of Invention
The invention mainly aims to provide a task scheduling method, a device, terminal equipment and a storage medium, and aims to solve the technical problems of low efficiency and stability of the existing task scheduling method.
In order to achieve the above object, the present invention provides a task scheduling method, including the steps of:
based on the task scheduling request, confirming the total workload of the original task;
based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks;
setting runway quantity according to the plurality of atomic tasks and the system resources;
and performing task scheduling according to the runway quantity based on the runway model.
Optionally, before the confirming the total workload of the original task based on the task scheduling request, the method further includes:
acquiring a task scheduling request, identifying an execution range of an original task, and configuring relevant necessary data according to the task;
recording the current state of the original task as a preparation state, sending the preparation state to a task monitoring process, and monitoring the state of the original task by the task monitoring process.
Optionally, the step of performing task scheduling according to the number of runways based on the runway model includes:
Based on a runway model, selecting a batch of parallel atomic tasks from the plurality of atomic tasks to create according to the quantity of the runways, and obtaining a first batch of atomic tasks;
executing the atomic tasks of the first batch in parallel;
and after the task monitoring process monitors that the execution of the atomic tasks of the first batch is completed, continuously creating and executing parallel atomic tasks of the next batch according to the number of runways based on a runway model.
Optionally, the step of performing task scheduling according to the number of runways based on the runway model further includes:
after the atomic tasks of the first batch are created, the current state of the original task is updated, and the updated current state of the original task is sent to a task monitoring process;
after the atomic tasks of each batch start to be executed, recording the execution information of the atomic tasks of each batch, sending the execution information to the task monitoring process, and monitoring the atomic tasks by the task monitoring process.
Optionally, after the task scheduling is performed according to the runway number based on the runway model, the method further includes:
After the execution of the plurality of atomic tasks is completed, detecting all the atomic tasks based on a preset failure retry mechanism, and automatically performing task retry on the atomic tasks conforming to the preset failure retry mechanism.
Optionally, after the execution of the plurality of atomic tasks is completed, detecting all the atomic tasks based on a preset failure retry mechanism, and automatically performing task retry on the atomic tasks that meet the preset failure retry mechanism includes:
after the execution of the plurality of atomic tasks is completed, acquiring the execution information of all the atomic tasks through a task monitoring process;
based on the execution information of all the atomic tasks, screening out failed atomic tasks meeting a preset failure retry mechanism according to the preset failure retry mechanism;
and carrying out task scheduling again on the failed atomic task meeting the preset failed retry mechanism.
Optionally, after the execution of the plurality of atomic tasks is completed, the task monitoring process obtains the execution information of all the atomic tasks, and further includes:
based on the execution information of all the atomic tasks, screening out failed atomic tasks which do not meet a preset failure retry mechanism according to the preset failure retry mechanism;
And generating early warning information based on the failed atomic task which does not meet a preset failed retry mechanism.
In addition, in order to achieve the above object, the present invention also provides a task scheduling device, including:
the task analysis module is used for confirming the total workload of the original task based on the task scheduling request;
the task segmentation module is used for carrying out atomic task segmentation based on the total workload of the original task to obtain a plurality of atomic tasks;
the runway setting module is used for setting the number of runways according to the plurality of atomic tasks and the system resources;
and the task scheduling module is used for performing task scheduling according to the runway quantity based on the runway model.
The embodiment of the application also provides a terminal device, which comprises a memory, a processor and a task scheduler which is stored on the memory and can run on the processor, wherein the task scheduler realizes the steps of the task scheduling method when being executed by the processor.
The embodiments of the present application also provide a computer readable storage medium having a task scheduler stored thereon, which when executed by a processor, implements the steps of the task scheduling method as described above.
According to the task scheduling method, device, terminal equipment and storage medium, the runway model is adopted to reasonably divide the total atomic task amount of the task, so that the parallel number of threads of the task and the execution of the atomic task are effectively controlled, the effective utilization of computer system resources is ensured, and the task scheduling efficiency and stability are improved.
Drawings
FIG. 1 is a schematic diagram of functional modules of a terminal device to which a task scheduling device of the present application belongs;
FIG. 2 is a flowchart of a first embodiment of a task scheduling method according to the present application;
FIG. 3 is a schematic diagram of a refinement flow of step S240 in an embodiment of a task scheduling method of the present application;
FIG. 4 is a schematic flow chart of a second embodiment of a task scheduling method according to an embodiment of the present application;
FIG. 5 is a schematic flow chart of a third embodiment of a task scheduling method according to an embodiment of the present application;
FIG. 6 is a detailed flowchart of step S510 in an embodiment of the task scheduling method of the present application;
fig. 7 is a schematic diagram of another refinement flow of step S510 in an embodiment of the task scheduling method of the present application.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The main solutions of the embodiments of the present invention are: based on the task scheduling request, confirming the total workload of the original task; based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks; setting runway quantity according to the plurality of atomic tasks and the system resources; and performing task scheduling according to the runway quantity based on the runway model.
Because the existing task scheduling technology often depends on the thread scheduling of a computer system, the thread is created, the thread waits, the thread execution lacks effective management and control, and the memory overflow of the computer system is easily caused for the scene of needing to execute a large number of tasks, so that the task scheduling cannot be ensured to be executed efficiently and stably.
The invention provides a solution, which reasonably divides the total atomic task amount of the task by adopting a runway model, effectively controls the thread parallel quantity of the task and the execution of the atomic task, ensures the effective utilization of computer system resources and improves the efficiency and the stability of task scheduling.
Referring to fig. 1, fig. 1 is a schematic diagram of a task scheduling device in a hardware running environment according to an embodiment of the present invention.
As shown in fig. 1, the task scheduling device may include: a processor 101, such as a central processing unit (Central Processing Unit, CPU), a communication bus 102, a user interface 103, a network interface 104, a memory 105. Wherein the communication bus 102 is used to enable connected communication between these components. The user interface 103 may comprise a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 103 may further comprise a standard wired interface, a wireless interface. The network interface 104 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 105 may be a high-speed random access Memory (Random Access Memory, RAM) Memory or a stable Non-Volatile Memory (NVM), such as a disk Memory. The memory 105 may alternatively be a storage device separate from the aforementioned processor 101.
Those skilled in the art will appreciate that the structure shown in fig. 1 does not constitute a limitation of the task scheduling device, and may include more or fewer components than shown, or may combine certain components, or may be a different arrangement of components.
In the task scheduling device shown in fig. 1, the network interface 104 is mainly used for data communication with other devices; the user interface 103 is mainly used for data interaction with a user; the processor 101 and the memory 105 in the task scheduling device of the present invention may be disposed in the task scheduling device, and the task scheduling device invokes the task scheduling program stored in the memory 105 through the processor 101 and executes the task scheduling method provided by the embodiment of the present invention.
Specifically, the task scheduler in the memory 105, when executed by the processor, performs the steps of:
based on the task scheduling request, confirming the total workload of the original task;
based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks;
setting runway quantity according to the plurality of atomic tasks and the system resources;
and performing task scheduling according to the runway quantity based on the runway model.
Further, the task scheduler in the memory 105, when executed by the processor, also performs the following steps:
acquiring a task scheduling request, identifying an execution range of an original task, and configuring relevant necessary data according to the task;
recording the current state of the original task as a preparation state, sending the preparation state to a task monitoring process, and monitoring the state of the original task by the task monitoring process.
Further, the task scheduler in the memory 105, when executed by the processor, also performs the following steps:
based on a runway model, selecting a batch of parallel atomic tasks from the plurality of atomic tasks to create according to the quantity of the runways, and obtaining a first batch of atomic tasks;
executing the atomic tasks of the first batch in parallel;
And after the task monitoring process monitors that the execution of the atomic tasks of the first batch is completed, continuously creating and executing parallel atomic tasks of the next batch according to the number of runways based on a runway model.
Further, the task scheduler in the memory 105, when executed by the processor, also performs the following steps:
after the atomic tasks of the first batch are created, the current state of the original task is updated, and the updated current state of the original task is sent to a task monitoring process;
after the atomic tasks of each batch start to be executed, recording the execution information of the atomic tasks of each batch, sending the execution information to the task monitoring process, and monitoring the atomic tasks by the task monitoring process.
Further, the task scheduler in the memory 105, when executed by the processor, also performs the following steps:
after the execution of the plurality of atomic tasks is completed, detecting all the atomic tasks based on a preset failure retry mechanism, and automatically performing task retry on the atomic tasks conforming to the preset failure retry mechanism.
Further, the task scheduler in the memory 105, when executed by the processor, also performs the following steps:
after the execution of the plurality of atomic tasks is completed, acquiring the execution information of all the atomic tasks through a task monitoring process;
based on the execution information of all the atomic tasks, screening out failed atomic tasks meeting a preset failure retry mechanism according to the preset failure retry mechanism;
and carrying out task scheduling again on the failed atomic task meeting the preset failed retry mechanism.
Further, the task scheduler in the memory 105, when executed by the processor, also performs the following steps:
based on the execution information of all the atomic tasks, screening out failed atomic tasks which do not meet a preset failure retry mechanism according to the preset failure retry mechanism;
and generating early warning information based on the failed atomic task which does not meet a preset failed retry mechanism.
According to the scheme, the total workload of the original task is confirmed based on the task scheduling request; based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks; setting runway quantity according to the plurality of atomic tasks and the system resources; and performing task scheduling according to the runway quantity based on the runway model. By adopting the runway model, the total atomic task amount of the task is reasonably divided, the thread parallel quantity of the task and the execution of the atomic task are effectively controlled, the effective utilization of the computer system resources is ensured, and the efficiency and the stability of task scheduling are improved.
Based on the above terminal device architecture, but not limited to the above architecture, the method embodiments of the present application are presented.
Referring to fig. 2, fig. 2 is a flowchart of a first embodiment of a task scheduling method according to the present application. The task scheduling method comprises the following steps:
step S210: based on the task scheduling request, confirming the total workload of the task;
step S220: based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks;
step S230: setting runway quantity according to the plurality of atomic tasks and the system resources;
step S240: and performing task scheduling according to the runway quantity based on the runway model.
The execution main body of the method of the embodiment may be a task scheduling device, or may be a task scheduling terminal device or a server, and the task scheduling device is exemplified by the task scheduling device, and the task scheduling device may be integrated on a terminal device such as a smart phone, a tablet computer, and the like with a data processing function.
The scheme of the embodiment mainly realizes task scheduling of the computer system, improves the efficiency and stability of task scheduling, ensures the effective utilization of computer resources, and further improves the performance and efficiency of the computer system.
The steps of this embodiment are explained in detail as follows:
in particular, with the popularity of the internet in daily life and various industries, computer systems have become an indispensable tool for people to accomplish their work. However, the performance and efficiency of computer systems is certainly facing a significant challenge when dealing with complex and task-intensive work scenarios. The computer system resources are generally and effectively allocated and managed by adopting a strategy of task scheduling and resource scheduling, so that the stable operation of the system is ensured, the resource utilization is optimized and the system performance is improved.
However, the existing task scheduling method often depends on the thread scheduling of the computer system, and lacks effective management and control on thread creation, thread waiting and thread execution, which is easy to bring challenges to the thread scheduling, management and memory management of the computer system, and for the scene that needs to execute a large number of tasks, the memory overflow of the computer system is easy to cause, and the task scheduling cannot be guaranteed to be executed efficiently and stably.
Therefore, the invention designs a task scheduling method based on a runway model aiming at the problems of the prior method. The task scheduling method specifically comprises the following steps:
Step S210: based on the task scheduling request, confirming the total workload of the task;
in order to embody more excellent performance in the working scene of complex coping and huge task quantity, the computing system needs to confirm the task scope of the task to be executed at first so as to ensure the accuracy of the subsequent execution task.
Specifically, when the computer system faces a complex and huge task scenario, it is first required to receive a task scheduling request through a task scheduling device, analyze the task scheduling request, and confirm the total workload of the task required to be executed by the request. The total workload of a task refers to the total working steps of the task, which need to be executed by threads in a computer.
Further, in order to ensure efficient execution of the task, the task scheduling device needs to perform configuration of relevant data according to the task to be executed before acquiring the total workload of the task. Therefore, before the step S210, the method further includes:
step S310: acquiring a task scheduling request, identifying an execution range of an original task, and configuring relevant necessary data according to the task;
step S320: recording the current state of the original task as a preparation state, sending the preparation state to a task monitoring process, and monitoring the state of the original task by the task monitoring process.
The method comprises the steps of obtaining a task scheduling request through a task scheduling device, identifying an execution range of an original task, namely a computer system resource related to the original task, obtaining relevant information of the original task according to analysis of the original task, and configuring relevant information of system resources required by executing the task for the original task.
Wherein, the computer system resources refer to related resources such as CPU, memory, storage, network bandwidth, I/O equipment and the like; the related information of the original task refers to the type, size, priority, complexity, execution time and other attributes of the task and the dependency relationship of the task; the related information of the system resources required for executing the task refers to the information of the size, the number, the type and the like of the resources such as the CPU, the memory, the storage, the network bandwidth, the I/O equipment and the like required for executing the original task.
It can be appreciated that in order to achieve efficient task monitoring, the task scheduling device needs to record the current state of the original task, and since the current stage is only to perform the data preparation of the previous stage for the task, record the current state of the task as the ready state, and send the state record to the task monitoring process, so as to be able to know the task progress in real time.
Step S220: based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks;
step S230: setting runway quantity according to the plurality of atomic tasks and the system resources;
specifically, atomic task segmentation is performed based on the total workload of the original tasks to obtain the total atomic task amount of a plurality of original tasks, and the number of parallel atomic tasks, namely the runway number, is set according to the effective resources of the system and the resource requirements of the atomic tasks. The effective resources of the system refer to computer system resources, namely related resources such as CPU, memory, storage, network bandwidth, I/O equipment and the like; the atomic task refers to a task which cannot be interrupted in the execution process and must be continuously executed and completed, and the atomic tasks are mutually independent.
By setting the number of runways, on one hand, the resources required by the execution of the tasks are ensured, and meanwhile, the orderly execution among the multiple tasks can be ensured under the parallel of the multiple tasks.
For example, when the original task is a data analysis task, the task scheduling device needs to analyze each data as an atomic task according to the total amount of analysis required by the data analysis task, and obtains the maximum number of atomic tasks that can be supported and executed by the current system through available resources of the current system and resource requirements of the atomic task, and sets the maximum number of atomic tasks as the number of parallel atomic tasks. It can be understood that in practical application, in order to ensure stable running of the computer system, the computer system is generally not fully loaded for a long time, so the number of parallel atoms, i.e. the number of runways, can be slightly smaller than the calculated maximum number of atomic tasks that can be supported by the current system, and the specific reduction amplitude can be determined according to the experience of relevant operation and maintenance personnel.
Step S240: and performing task scheduling according to the runway quantity based on the runway model.
Specifically, the task scheduling based on the runway model is performed according to the runway number obtained in the above step S230. The task scheduling device creates a group of related threads of parallel atomic tasks according to the runway quantity, and simultaneously executes the corresponding atomic tasks in parallel. And meanwhile, determining whether the number of the residual atomic tasks is 0 or not through an independent task monitoring process, and continuously carrying out thread creation and execution of the next batch of parallel atomic tasks under the condition that the number of the residual atomic tasks is not 0.
According to the scheme, the total work amount of the original task is confirmed based on the task scheduling request; based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks; setting runway quantity according to the plurality of atomic tasks and the system resources; and performing task scheduling according to the runway quantity based on the runway model.
By adopting the runway model, the total atomic task amount of the task is reasonably divided, the thread parallel quantity of the task and the execution of the atomic task are effectively controlled, the effective utilization of the computer system resources is ensured, and the efficiency and the stability of task scheduling are improved.
Referring to fig. 3, fig. 3 is a schematic diagram of a refinement flow of step S240 in an embodiment of a task scheduling method of the present application. Based on the embodiment shown in fig. 2, in this embodiment, the step S240: based on the runway model, performing task scheduling according to the runway number includes:
step S2401: based on a runway model, selecting a batch of parallel atomic tasks from the plurality of atomic tasks to create according to the quantity of the runways, and obtaining a first batch of atomic tasks;
step S2402: executing the atomic tasks of the first batch in parallel;
specifically, batches are divided into the total number of atomic tasks of the aforementioned tasks according to the number of runways, i.e., the number of parallel atomic tasks. The thread of the atomic task is not created once, but in a thread pool of the task, parallel atomic tasks are created in batches according to runway model scheduling, and then parallel execution of the created corresponding batch of atomic tasks is started.
Step S2403: and after the task monitoring process monitors that the execution of the atomic tasks of the first batch is completed, continuously creating and executing parallel atomic tasks of the next batch according to the number of runways based on a runway model.
It can be understood that in order to ensure that the task is completely executed, the number of parallel atomic tasks created and executed in batches needs to be monitored, so that the missing of executing related atomic tasks is avoided, and the reliability of the task scheduling device is further ensured.
Specifically, after the task scheduling device divides the total atomic task amount of the original task according to the number of runways, thread creation and execution of the atomic task of the first batch are performed first. Then, if the unexecuted atomic task exists in the plurality of atomic tasks, that is, if the task is not completely executed, the task monitoring process monitors that the atomic task of the first batch is executed, and then creates the atomic task of the next batch according to the number of runways for the remaining atomic tasks in the plurality of atomic tasks, and executes the atomic task of the next batch in parallel. Wherein parallel execution means that in a computer system, multiple threads simultaneously run multiple atomic tasks.
The method can enable tasks to be sequentially executed in batches according to the number of the runways, so that the execution sequence of the tasks is ensured, and the system load is ensured to be in a stable and controllable state.
Meanwhile, in the embodiment, when task scheduling is performed for the first time, threads of atomic tasks of a first batch are created according to the number of runways, and when scheduling of residual atomic tasks is performed subsequently, because the atomic tasks of the first batch are already completed, the created corresponding threads are in an idle state, and the threads are allowed not to be destroyed but to be reused for creating and executing the atomic tasks of the next batch, so that the times of creating and destroying the threads are reduced, and the effective realization of a thread pool is completed.
According to the method, the atomic tasks of each batch are executed in batches through the thread pool, the task execution is prevented from excessively depending on the thread scheduling of an operating system, and meanwhile, the system performance loss caused by excessive thread queuing and frequent switching of system threads is also avoided.
According to the scheme, the execution range of the original task is identified by acquiring the task scheduling request, and the necessary data related to the task configuration is obtained; recording the current state of the original task as a preparation state, and sending the preparation state to a task monitoring process, wherein the task monitoring process monitors the state of the original task; based on the task scheduling request, confirming the total workload of the original task; based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks; setting runway quantity according to the plurality of atomic tasks and the system resources; based on a runway model, selecting a batch of parallel atomic tasks from the plurality of atomic tasks to create according to the quantity of the runways, and obtaining a first batch of atomic tasks; executing the atomic tasks of the first batch in parallel; and after the task monitoring process monitors that the execution of the atomic tasks of the first batch is completed, continuously creating and executing parallel atomic tasks of the next batch according to the number of runways based on a runway model.
By adopting the runway model, the total atomic task amount of the original task is reasonably divided, the thread parallel quantity of the task and the execution of the atomic task are effectively controlled, meanwhile, the effective realization of a thread pool is completed, the excessive dependence of the execution of the task on the thread scheduling of an operating system is avoided, the performance loss of the system is reduced, the effective utilization of the computer system resource is realized, the efficiency and the stability of the task scheduling are improved, and the performance and the stability of the system are improved.
Referring to fig. 4, fig. 4 is a schematic flow chart of task monitoring in an embodiment of the task scheduling method of the present application. Based on the embodiment shown in fig. 2, in this embodiment, the step S240: based on the runway model, task scheduling is performed according to the runway quantity, and the method further comprises the following steps:
step S410: after the atomic tasks of the first batch are created, the current state of the original task is updated, and the updated current state of the original task is sent to a task monitoring process;
step S420: after the atomic tasks of each batch start to be executed, recording the execution information of the atomic tasks of each batch, sending the execution information to the task monitoring process, and monitoring the atomic tasks by the task monitoring process.
It can be appreciated that, in order to realize efficient task monitoring, the task scheduling device needs to record the current state of the original task, and because the current stage is to schedule the original task, the current state of the original task needs to be updated, and the state record is sent to the task monitoring process, so that the progress of the original task can be known in real time.
Specifically, in task scheduling, after a batch of parallel atomic tasks are executed, the task scheduling device needs to update the current state of the original task to a scheduling state, and meanwhile, needs to record execution information of a batch of parallel atomic tasks, including an execution state, an execution progress and an execution result, and send the execution result to a task monitoring process.
For example, in performing financial data analysis tasks using a computer system, account information for some people is required, including open time, log-off time, account status, account balance, etc. When the computer system is used for inquiring account information of a plurality of persons, atomic task division can be performed through the task scheduling device. The opening time, the cancellation time, the account state and the account balance are used as independent data analysis indexes, 4 atomic tasks can be divided correspondingly, and the account information of different personnel is 4 atomic tasks which are divided correspondingly. When inquiring account information of a large number of people, the task scheduling device performs atomic task division and then performs task scheduling according to the number of runways. In the task scheduling process, updating the current state of original tasks, namely financial data analysis, as a scheduling state, monitoring account opening time, cancellation time, account state and account balance data query tasks of each person through a task monitoring process, and recording whether the execution state of each task is in execution or completed, the execution progress and the searched related data result.
According to the scheme, the execution range of the original task is identified by acquiring the task scheduling request, and the necessary data related to the task configuration is obtained; recording the current state of the original task as a preparation state, and sending the preparation state to a task monitoring process, wherein the task monitoring process monitors the state of the original task; based on the task scheduling request, confirming the total workload of the original task; based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks; setting runway quantity according to the plurality of atomic tasks and the system resources; based on a runway model, selecting a batch of parallel atomic tasks from the plurality of atomic tasks to create according to the quantity of the runways, and obtaining a first batch of atomic tasks; executing the atomic tasks of the first batch in parallel; and after the task monitoring process monitors that the execution of the atomic tasks of the first batch is completed, continuously creating and executing parallel atomic tasks of the next batch according to the number of runways based on a runway model. After the atomic tasks of the first batch are created, the current state of the original task is updated, and the updated current state of the original task is sent to a task monitoring process; after the atomic tasks of each batch start to be executed, recording the execution information of the atomic tasks of each batch, sending the execution information to the task monitoring process, and monitoring the atomic tasks by the task monitoring process.
The task monitoring process is used for realizing effective task monitoring, mastering the execution state and execution progress of all tasks of the system, realizing task progress observation and task execution control mechanism under the multi-task parallel state, and ensuring that a large number of parallel tasks can be orderly executed under the limited system hardware resources. Based on the method, effective utilization of system resources can be ensured, and stability of task scheduling is further improved.
Referring to fig. 5, fig. 5 is a schematic flow chart of task retry in an embodiment of the task scheduling method of the present application. Based on the embodiment shown in fig. 2, in step S240: based on the runway model, after task scheduling according to the runway quantity, the method further comprises the following steps:
step S510: after the execution of the plurality of atomic tasks is completed, detecting all the atomic tasks based on a preset failure retry mechanism, and automatically performing task retry on the atomic tasks conforming to the preset failure retry mechanism.
Specifically, whether the plurality of atomic tasks are executed and completed can be confirmed through a task monitoring process, when the plurality of atomic tasks are executed and completed, all the atomic tasks are detected according to a preset failure retry mechanism, the failed atomic tasks conforming to the preset failure retry mechanism are detected, and then task scheduling is carried out again on the detected failed atomic tasks conforming to the preset failure retry mechanism, so that the realization of the task retry mechanism is completed. The failed atomic task refers to an atomic task with a wrong task state or task result; the preset failed retry mechanism is a rule set manually according to experience, for example, a rule for detecting failed atomic tasks is set, and the number of retries of failed atomic tasks is limited.
Further, referring to fig. 6, fig. 6 is a schematic diagram of a refinement flow chart of task retry in an embodiment of the task scheduling method of the present application. Based on the embodiment shown in fig. 5, in this embodiment, the step S510: after the execution of the plurality of atomic tasks is completed, detecting all the atomic tasks based on a preset failure retry mechanism, and automatically performing task retry on the atomic tasks conforming to the preset failure retry mechanism comprises:
step S5101: after the execution of the plurality of atomic tasks is completed, acquiring the execution information of all the atomic tasks through a task monitoring process;
specifically, the task monitoring process confirms whether the plurality of atomic tasks are all executed and completed, and specifically, the number of atomic tasks which are currently executed can be counted and compared with the plurality of atomic tasks, and when the number is the same, the plurality of atomic tasks are all executed and completed. And then detecting all the atomic task states or atomic task results through a task monitoring process, and judging that the atomic task is a failed atomic task if the atomic task state is the execution failure or the atomic task result is different from a preset result according to the detection result.
Step S5102: based on the execution information of all the atomic tasks, screening out failed atomic tasks meeting a preset failure retry mechanism according to the preset failure retry mechanism;
step S5103: and carrying out task scheduling again on the failed atomic task meeting the preset failed retry mechanism.
It can be appreciated that, in order to ensure the stability of the computer system, infinite retry of an atomic task is avoided, resource consumption of the computer system is avoided, data consistency is protected, and the number of retries of a failed atomic task is limited by the task scheduling device.
Specifically, after detecting all failed atomic tasks in the executed atomic tasks, performing secondary detection on the failed atomic tasks, detecting whether the retry number of the failed atomic tasks is smaller than a preset value, and if the retry number is smaller than the preset value, automatically performing task scheduling on the atomic tasks through a task scheduling device. The preset value is set by relevant operation staff according to the resource allocation and experience of the computer system, and generally can be taken 3 to 5 times.
According to the scheme, the execution range of the original task is identified by acquiring the task scheduling request, and the necessary data related to the task configuration is obtained; recording the current state of the original task as a preparation state, and sending the preparation state to a task monitoring process, wherein the task monitoring process monitors the state of the original task; based on the task scheduling request, confirming the total workload of the original task; based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks; setting runway quantity according to the plurality of atomic tasks and the system resources; performing task scheduling according to the number of runways based on a runway model; after the execution of the plurality of atomic tasks is completed, detecting all the atomic tasks based on a preset failure retry mechanism, and automatically performing task retry on the atomic tasks conforming to the preset failure retry mechanism.
The failed atomic task meeting the requirements is detected through a task monitoring process and a preset failure retry mechanism, and the task retry is automatically carried out through a thread pool for task execution, so that the complete execution of the whole task is ensured, and the efficiency and the reliability of task scheduling are improved.
Referring to fig. 7, fig. 7 is another detailed flowchart of task retry in an embodiment of the task scheduling method of the present application. Based on the embodiment shown in fig. 5, in step S5101: after the execution of the plurality of atomic tasks is completed, the task monitoring process acquires the execution information of all the atomic tasks, and then the method further comprises the following steps:
step S710: based on the execution information of all the atomic tasks, screening out failed atomic tasks which do not meet a preset failure retry mechanism according to the preset failure retry mechanism;
step S720: and generating early warning information based on the failed atomic task which does not meet a preset failed retry mechanism.
Specifically, to ensure complete execution of the task, further processing is required for failed atomic tasks that have exceeded the retry limit. In this embodiment, the task scheduling device detects the failed atomic tasks exceeding the retry limit through the task monitoring process, and generates early warning information according to the failed atomic tasks exceeding the retry limit, and feeds the early warning information back to the relevant operation and maintenance personnel, so that the relevant operation and maintenance personnel can perform error analysis on the failed atomic tasks exceeding the retry limit, and re-execute the failed atomic tasks after modifying the errors, thereby ensuring complete execution of the tasks. The early warning information includes attributes such as type, size, priority, complexity, execution time and the like of the failed atomic task, dependency relationship, related logs of the failed atomic task which are retried previously and the like.
According to the scheme, the execution range of the original task is identified by acquiring the task scheduling request, and the necessary data related to the task configuration is obtained; recording the current state of the original task as a preparation state, and sending the preparation state to a task monitoring process, wherein the task monitoring process monitors the state of the original task; based on the task scheduling request, confirming the total workload of the original task; based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks; setting runway quantity according to the plurality of atomic tasks and the system resources; performing task scheduling according to the number of runways based on a runway model; after the execution of the plurality of atomic tasks is completed, acquiring the execution information of all the atomic tasks through a task monitoring process; based on the execution information of all the atomic tasks, screening out failed atomic tasks meeting a preset failure retry mechanism according to the preset failure retry mechanism; and carrying out task scheduling again on the failed atomic task meeting the preset failed retry mechanism. Based on the execution information of all the atomic tasks, screening out failed atomic tasks which do not meet a preset failure retry mechanism according to the preset failure retry mechanism; and generating early warning information based on the failed atomic task which does not meet a preset failed retry mechanism.
And detecting the failure atomic task which does not meet the requirements through a task monitoring process and a preset failure retry mechanism so as to generate related early warning information, so that follow-up related operation and maintenance personnel can perform task error analysis, further ensure the complete execution of the whole task, and improve the efficiency and reliability of task scheduling.
In addition, in order to achieve the above object, the present invention also provides a task scheduling device, including:
the task analysis module is used for confirming the total workload of the original task based on the task scheduling request;
the task segmentation module is used for carrying out atomic task segmentation based on the total workload of the original task to obtain a plurality of atomic tasks;
the runway setting module is used for setting the number of runways according to the plurality of atomic tasks and the system resources;
and the task scheduling module is used for performing task scheduling according to the runway quantity based on the runway model.
The principle and implementation process of task scheduling are implemented in this embodiment, please refer to the above embodiments, and are not repeated here.
The embodiment of the application also provides a terminal device, which comprises a memory, a processor and a task scheduler which is stored on the memory and can run on the processor, wherein the task scheduler realizes the steps of the task scheduling method when being executed by the processor.
Because the task scheduler adopts all the technical schemes of all the embodiments when being executed by the processor, the task scheduler at least has all the beneficial effects brought by all the technical schemes of all the embodiments and is not described in detail herein.
The embodiments of the present application also provide a computer readable storage medium having a task scheduler stored thereon, which when executed by a processor, implements the steps of the task scheduling method as described above.
Because the task scheduler adopts all the technical schemes of all the embodiments when being executed by the processor, the task scheduler at least has all the beneficial effects brought by all the technical schemes of all the embodiments and is not described in detail herein.
It is noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present application are merely for describing, and do not represent advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as above, including several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to perform the method of each embodiment of the present application.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (10)
1. A task scheduling method, characterized in that the task scheduling method comprises:
based on the task scheduling request, confirming the total workload of the original task;
based on the total workload of the original task, performing atomic task segmentation to obtain a plurality of atomic tasks;
setting runway quantity according to the plurality of atomic tasks and the system resources;
and performing task scheduling according to the runway quantity based on the runway model.
2. The task scheduling method of claim 1, wherein before confirming the total work amount of the original task based on the task scheduling request, further comprising:
acquiring a task scheduling request, identifying an execution range of an original task, and configuring relevant necessary data according to the task;
recording the current state of the original task as a preparation state, and sending the preparation state to a task monitoring process, wherein the task monitoring process monitors the state of the original task.
3. The task scheduling method of claim 2, wherein the step of performing task scheduling according to the number of runways based on a runway model comprises:
based on a runway model, selecting a batch of parallel atomic tasks from the plurality of atomic tasks to create according to the quantity of the runways, and obtaining a first batch of atomic tasks;
Executing the atomic tasks of the first batch in parallel;
and after the task monitoring process monitors that the execution of the atomic tasks of the first batch is completed, continuously creating and executing parallel atomic tasks of the next batch according to the number of runways based on a runway model.
4. A task scheduling method according to claim 3, wherein the step of performing task scheduling according to the number of runways based on a runway model further comprises:
after the atomic tasks of the first batch are created, the current state of the original task is updated, and the updated current state of the original task is sent to a task monitoring process;
after the atomic tasks of each batch start to be executed, recording the execution information of the atomic tasks of each batch, sending the execution information to the task monitoring process, and monitoring the atomic tasks by the task monitoring process.
5. The task scheduling method of claim 2, wherein after performing task scheduling according to the number of runways based on the runway model, further comprising:
after the execution of the plurality of atomic tasks is completed, detecting all the atomic tasks based on a preset failure retry mechanism, and automatically performing task retry on the atomic tasks conforming to the preset failure retry mechanism.
6. The task scheduling method of claim 5, wherein after the execution of the plurality of atomic tasks is completed, detecting all the atomic tasks based on a preset failure retry mechanism, and automatically performing task retry on the atomic tasks that meet the preset failure retry mechanism comprises:
after the execution of the plurality of atomic tasks is completed, acquiring the execution information of all the atomic tasks through a task monitoring process;
based on the execution information of all the atomic tasks, screening out failed atomic tasks meeting a preset failure retry mechanism according to the preset failure retry mechanism;
and carrying out task scheduling again on the failed atomic task meeting the preset failed retry mechanism.
7. The method for scheduling tasks according to claim 6, wherein after the execution of all the plurality of atomic tasks is completed, the method further comprises:
based on the execution information of all the atomic tasks, screening out failed atomic tasks which do not meet a preset failure retry mechanism according to the preset failure retry mechanism;
and generating early warning information based on the failed atomic task which does not meet a preset failed retry mechanism.
8. A task scheduling method apparatus, wherein the apparatus comprises:
the task analysis module is used for confirming the total workload of the original task based on the task scheduling request;
the task segmentation module is used for carrying out atomic task segmentation based on the total workload of the original task to obtain a plurality of atomic tasks;
the runway setting module is used for setting the number of runways according to the plurality of atomic tasks and the system resources;
and the task scheduling module is used for performing task scheduling according to the runway quantity based on the runway model.
9. A task scheduling method apparatus, characterized in that the apparatus comprises: memory, a processor and a task scheduler stored on the memory and executable on the processor, the task scheduler being configured to implement the steps of the task scheduling method of any one of claims 1 to 7.
10. A storage medium having stored thereon a task scheduler, which when executed by a processor, implements the steps of the task scheduling method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311769303.5A CN117762587A (en) | 2023-12-20 | 2023-12-20 | Task scheduling method, device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311769303.5A CN117762587A (en) | 2023-12-20 | 2023-12-20 | Task scheduling method, device, terminal equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117762587A true CN117762587A (en) | 2024-03-26 |
Family
ID=90319395
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311769303.5A Pending CN117762587A (en) | 2023-12-20 | 2023-12-20 | Task scheduling method, device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117762587A (en) |
-
2023
- 2023-12-20 CN CN202311769303.5A patent/CN117762587A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9946577B1 (en) | Systems and methods for distributed resource management | |
| US10452448B2 (en) | Systems and methods for distributed resource management | |
| CN113535367B (en) | Task scheduling method and related device | |
| CN106557369A (en) | A kind of management method and system of multithreading | |
| US11311722B2 (en) | Cross-platform workload processing | |
| WO2006123177A1 (en) | Data processing network | |
| CN111338791A (en) | Method, device and equipment for scheduling cluster queue resources and storage medium | |
| CN116541134B (en) | Method and device for deploying containers in multi-architecture cluster | |
| CN112579267A (en) | Decentralized big data job flow scheduling method and device | |
| CN111666141A (en) | Task scheduling method, device and equipment and computer storage medium | |
| Maruf et al. | Extending resources for avoiding overloads of mixed‐criticality tasks in cyber‐physical systems | |
| CN114816730A (en) | Robot process automation cloud service system and implementation method | |
| CN111709723A (en) | RPA business process intelligent processing method, device, computer equipment and storage medium | |
| CN106775975B (en) | Process scheduling method and device | |
| CN113741961B (en) | Method and device for submitting big data calculation operation based on Kubernetes container arrangement software | |
| CN114816709A (en) | Task scheduling method, device, server and readable storage medium | |
| Harichane et al. | KubeSC‐RTP: Smart scheduler for Kubernetes platform on CPU‐GPU heterogeneous systems | |
| CN113467908A (en) | Task execution method and device, computer readable storage medium and terminal equipment | |
| CN117149372A (en) | Task scheduling method, device, equipment and storage medium | |
| CN112948096A (en) | Batch scheduling method, device and equipment | |
| US8788601B2 (en) | Rapid notification system | |
| CN111831408A (en) | Asynchronous task processing method and device, electronic equipment and medium | |
| CN109189581B (en) | Job scheduling method and device | |
| CN117762587A (en) | Task scheduling method, device, terminal equipment and storage medium | |
| CN110851245A (en) | Distributed asynchronous task scheduling method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |