WO2012088766A1

WO2012088766A1 - Method and device for dynamically scheduling batch processing tasks

Info

Publication number: WO2012088766A1
Application number: PCT/CN2011/002208
Authority: WO
Inventors: 何凯; 何发亮; 刘健
Original assignee: 中国银联股份有限公司
Priority date: 2010-12-29
Filing date: 2011-12-29
Publication date: 2012-07-05
Also published as: CN102541648A

Abstract

A method and device used for dynamically scheduling batch processing tasks, wherein the tasks include a first task and a second task. The method includes the following steps: dividing the first task and the second task into a plurality of task nodes, and grouping the plurality of task nodes into a first flow corresponding to the first task and a second flow corresponding to the second task; and controlling the execution of the first flow and the second flow by taking one task node as one unit according to the information about each task node, wherein the information includes the execution state and execution condition of each task node in the plurality of task nodes.

Description

Method and apparatus for dynamically scheduling batch tasks

The present application claims priority to Chinese Patent Application No. 201010619141.3, entitled "A Method and Apparatus for Dynamically Distributing Batch Processing Tasks", filed on Dec. 29, 2010, the entire contents of In this application. Technical field

The present invention relates to the field of computer technology, and in particular to dynamic scheduling of batch tasks. Background technique

A batch task is an operation that performs a series of programs in a computer system without human intervention. The existing batch task scheduling system pays attention to the scheduled scheduling of tasks, but when the execution time of some tasks exceeds the expected time, multiple tasks will be executed in parallel. If the task consumes too much system resources, it may lead to deadlock or even system crash. . The traditional sequential scheduling on time can guarantee the abundance of available resources for lightweight batch tasks, but it may cause waste of system resources and inefficient work. The scheduling of large tasks lacks controllability, and it is difficult to suspend in time. Schedule operations such as restart and sequel.

Therefore, there is a need for a method and apparatus that can dynamically schedule batch tasks. Summary of the invention

In order to solve the above problems, the present invention provides a method for dynamically scheduling a batch task, the task comprising a first task and a second task, wherein the method comprises the following steps: first step, first task and second task Decomposing a task into a plurality of task nodes, and grouping the plurality of task nodes into a first process corresponding to the first task and a second process corresponding to the second task;

In the second step, the execution of the first process and the second process, the execution state of the information task node, and the execution component are controlled in units of task nodes according to the information of each task node.

- 1 - Confirmation Preferably, the method further comprises the step of acquiring system resource information; the second step further comprising controlling the execution of each task node based on the system resource information.

Preferably, the first step further comprises setting priorities of the first process and the second process; the second step further comprises controlling execution of the first process and the second process according to the priority.

According to another aspect of the present invention, there is provided a device for dynamically scheduling a batch task, the task comprising a first task and a second task, the device comprising a setting module and a control module, wherein

The setting module is configured to decompose the first task and the second task into a plurality of task nodes, and to form the plurality of task nodes into a first process corresponding to the first task and a second process corresponding to the second task;

The control module is configured to control execution of the first process and the second process in units of task nodes according to information of each task node, where the information includes an execution state and an execution condition of each of the plurality of task nodes.

Preferably, the apparatus further comprises a resource monitoring module for acquiring system resource information; and the control module is further configured to control the execution of each task node according to the system resource letter.

Preferably, the setting module is further configured to set a priority of the first process and the second process; the control module is further configured to control the execution of the first process and the second process according to the priority.

Advantageously, the task node is the smallest program unit with data integrity.

The invention provides the resource utilization rate of the system through the above technical solution. By analyzing the dependency relationship, when a task node fails, the corresponding processing will be performed in time, thereby improving the fault tolerance capability of the system. The scheduling of the batch processing task of the invention is based on the dynamic change of the system resources and the static resource characteristics of the task nodes, and has strong flexibility. In addition, because the task is serialized into task nodes, the scheduling and tasks are separated from each other, and the tasks are scheduled in the form of a process, which can be well adapted to business changes. DRAWINGS

After reading the specific embodiments of the present invention with reference to the accompanying drawings, those skilled in the art Various aspects of the invention will be more clearly understood. It should be understood by those skilled in the art that these drawings are only intended to illustrate the technical solutions of the present invention, and are not intended to limit the scope of the present invention. among them,

1 is a flow chart of a method of dynamically scheduling a batch task according to an embodiment of the present invention; FIG. 2 is a diagram of a method of dynamically scheduling a batch task according to an embodiment of the present invention; FIG. 3 is a dynamic diagram according to an embodiment of the present invention. Schematic diagram of a device that schedules batch tasks. detailed description

Specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Descriptions of well-known functions and structures are omitted herein for clarity and conciseness, otherwise they will obscure the subject matter of the present invention.

As shown in FIG. 1, it is a flowchart of a method of dynamically scheduling a batch task according to an embodiment of the present invention, wherein:

In step S1, the batch task is serialized into a process consisting of task nodes. As an example, the task can be disassembled into multiple node units with data integrity, and then these node units are spliced into a process that can be scheduled by the configuration, and the generated process corresponds to the original task. The smallest program unit that a task node can be. This unit is a logical unit that can be executed independently.

In step S2, the flow is dynamically scheduled according to the information of the task node. The information of the task node may include an execution state and an execution condition of the task node. The execution status of each independently executable task node may include preparation for execution, execution, and execution completion. Execution conditions may include performing corresponding resource consumption data for the task node to describe its resource demand status or may also include execution time of the task node and dependencies of the task node with other task nodes in the process.

Reference is now made to Fig. 2, which is an illustration of a method of dynamically scheduling a batch task, in accordance with an embodiment of the present invention. As shown, the first process and the second process may be composed of a plurality of such task nodes A, B, C, D, E, F in a serial-parallel manner. According to the idea of the present invention, the start, pause and sequel of the process are implemented by the task node, and the data integrity of the node ensures that the process scheduling can be freely performed among multiple tasks, which greatly increases the flexibility of the call. example PT/CN2011/002208 For example, in FIG. 2, the dependency relationship between the task node D and other nodes, such as nodes B and C, can be known that the execution condition of the task node D in the first process is that the task nodes B and C are executed, then The execution condition of the task node D can be set accordingly, for example, the execution time of the task node D can be set after the execution time of the task nodes B and C. Alternatively, the execution of the task node D can also be triggered by judging the execution states of the task node task nodes B and C. When a pause operation is required, the task node to be released may be determined according to resource consumption data of the task node, and thus the task node information also provides a prerequisite for resource release of operations such as suspension.

3 is a schematic diagram of an apparatus for dynamically scheduling a batch task in accordance with an embodiment of the present invention. As shown in the figure, the device comprises a setting module, a control module and a resource monitoring module, wherein the setting module is used for decomposing tasks into a plurality of task nodes, and the plurality of task nodes are grouped into a process 1-4. The control module is configured to control the process 1-4 in units of task nodes according to the information of each task node. The information of the task node may include an execution state and an execution condition of the task node. The resource monitoring module is configured to acquire system resource information, and then the control module controls execution of each task node according to system resource information. For example, system resource information can be obtained through the API of the operating system and provided as a reference condition for process scheduling. For example, priority can be set for the process. When the available resources of the system exceed the warning line, the process with low priority is suspended. When the system resources are sufficient, the suspended process is resumed in time. Thus, the resource information can be used as a scheduling decision for the process, and a pre-judgment can be made for the real-time scheduling of the process. Resolve resource conflicts between tasks while ensuring efficient operation of the program. As an example, the resource consumption data for a task node is described in Table 1 below:

(Table 1 )

Assume that the current resource alert value is 10, then the following scheduling decisions are generated: 1) A and C cannot coexist CPU demand exceeds warning value 2) A and B cannot coexist memory index exceeds warning value

3) B and C can coexist. The sum of the indicators is lower than 10

For example, if two running processes are to start a task node that cannot coexist, the control module will make a scheduling decision based on the process priority, and the suboptimal process will be suspended.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software in combination with a hardware platform, and of course, all can be implemented by hardware. It will be apparent to those skilled in the art that various changes and substitutions may be made in the particular embodiments of the invention without departing from the spirit and scope of the invention. Such changes and substitutions are intended to fall within the scope of the appended claims.

Claims

Rights request

A method for dynamically scheduling a batch task, the task comprising a first task and a second task, the method comprising the steps of:

a first step of decomposing the first task and the second task into a plurality of task nodes, and grouping the plurality of task nodes into a first process corresponding to the first task and corresponding to the second task Second process;

In a second step, the execution of the first process and the second process is controlled in units of task nodes according to the information of each task node, where the information includes an execution state and an execution condition of the task node.

2. The method of claim 1 wherein:

The method also includes the step of obtaining system resource information;

The second step further includes controlling execution of each of the task nodes based on system resource information.

3. The method of claim 1 or 2, wherein

The first step further includes setting priorities of the first process and the second process;

The second step further includes controlling execution of the first and second flows in accordance with the priority.

A device for dynamically scheduling a batch task, the task comprising a first task and a second task, the device comprising a setting module and a control module, wherein the setting module is configured to: Decomposing the first task and the second task into a plurality of task nodes, and composing the plurality of task nodes into a first process corresponding to the first task and a second process corresponding to the second task;

The control module is configured to control execution of the first process and the second process by using a task node according to the information of each task node, where the information includes an execution state and an execution condition of the task node.

5. Apparatus according to claim 4 wherein:

The device further includes a resource monitoring module, configured to acquire system resource information; The control module is further configured to control execution of the each task node based on system resource information.

6. Apparatus according to claim 4 or claim 5 wherein:

The setting module is further configured to set a priority of the first process and the second process;

The control module is further configured to control execution of the first and second flows in accordance with the priority.

7. Apparatus according to claim 5 wherein said task node is a minimum program unit having data integrity.