TW201734827A - Cloud batch scheduling system and batch management server computer program product thereof used for assigning the jobs of the execution server according to the measured data, the blueprint job file and the group classification setting file - Google Patents

Abstract

The present invention provides a cloud batch scheduling system and a batch management server computer program product thereof. An execution server of the system is used for executing a plurality of jobs, while a management server is used for Internet connection and assigning jobs for the execution server. For software modules in the management server, for example, a resource monitoring and measuring module is used for providing measured data after accessing and analyzing execution information of the execution server. A blueprint-type job execution process engine is used for dispatching blueprint jobs. A batch group classification module is used for classifying batch groups of the blueprint job so as to provide a group classification setting file. A dynamic batch job dispatching module is used for dynamically dispatching jobs in batches by calculating the load capability of the execution server according to the measured data, the blueprint job file and the group classification setting file after receiving the blueprint job.

Description

Cloud batch scheduling system and its batch management server computer program product

The invention relates to a cloud batch scheduling system and a batch management server computer program product thereof, in particular to an end batch scheduling system using a blueprint distribution technology and a batch management server computer program product thereof.

In order to save the cost of building computer equipment and management personnel, the current enterprises choose to rent the servo room to the cloud service provider, and install the online service that needs to be performed or required to be provided in the server room.

When the work content is batch operation, the traditional batch scheduling system is operated by calling the remote execution host to perform the scheduling work, and the execution result is integrated into the scheduling server for the administrator to view. Due to the increasing complexity of today's cloud computing business requirements, for example, cloud systems need to support complex execution workflows, high reusability of scheduling workspaces, and multiple execution environments (various types of programming languages, operating systems), making the past only Batch scheduling systems that offer a single batch program type are no longer sufficient for current needs.

In summary, how to provide a means for solving the above technical problems is a technical problem that needs to be solved in the field.

In order to solve the problems disclosed above, it is an object of the present invention to provide a technical means for dispatching a server to work by a blueprint dispatching technique.

To achieve the above object, the present invention provides a cloud batch scheduling system. The system includes at least one execution server for performing a plurality of tasks, and a management server for wiring and configuring the execution of the server. The foregoing management server further comprises a communication port, a memory and a processor, the communication port is used for the communication connection execution server, the processor is connected to the communication port and the memory, and the call is performed and the batch memory is executed under the execution of the work distribution. The management server software module, the batch management server software module includes a resource monitoring and measurement module, a blueprint work execution process engine, a batch group classification module, and a dynamic dispatch batch work module. The resource monitoring and measurement module is used to access and analyze the execution information of the execution server and provide measurement data. The blueprint work execution process engine is used to dispatch blueprint work. The blueprint work described above is assigned by the blueprint definition file, and the blueprint definition file is the blueprint structure formed by the set of nodes formed by the execution order of the work room. The batch group classification module performs batch group classification on the blueprint work to provide a group classification profile. The dynamic dispatch batch work module will dynamically calculate the workload according to the measurement data, the blueprint work file, and the group classification profile after receiving the blueprint work to calculate the load of the execution server to dynamically batch the work.

In order to achieve the above object, the present invention further provides a batch management server computer program product for a cloud batch scheduling system, the computer program product is applied and executed on a management server, and the foregoing management server is used for connection. And configuring at least one execution server to perform a plurality of tasks. The batch management server computer program product includes a resource monitoring and measurement module, a blueprint work execution process engine, a batch group classification module, and a dynamic dispatch batch work module. The resource monitoring and measurement module is used to access and analyze execution information of the execution server to provide measurement data. The blueprint work execution process engine is used to dispatch the blueprint work. The blueprint work described above is assigned by the blueprint definition file, and the blueprint definition file is the node set formed by the execution order of the work room. The blueprint structure formed by the combination. The batch group classification module performs batch group classification on the blueprint work to provide a group classification profile. The dynamic dispatch batch work module will dynamically calculate the workload according to the measurement data, the blueprint work file, and the group classification profile after receiving the blueprint work to calculate the load of the execution server to dynamically batch the work.

In summary, the cloud batch scheduling system and the batch management server computer program product of the present invention can efficiently distribute the work through the blueprint distribution technology to improve the processing performance of the cloud server.

100‧‧‧Execution server side

110‧‧‧ batch execution agent module

111‧‧‧Execution result collection module

120‧‧‧Automatic Extended Batch Server Module

200‧‧‧Manage server side

210‧‧‧Batch Management Server Software Module

211‧‧‧Resource monitoring and measurement module

212‧‧‧Blueprint Work Execution Process Engine

213‧‧‧Batch group classification module

214‧‧‧Dynamic Dispatch Batch Work Module

FIG. 1 is a schematic diagram of a system platform of a cloud batch scheduling system according to the present invention.

Figure 2 is a schematic diagram of the blueprint structure of the present invention.

3 is a flow chart of the decentralized work execution flow of the present invention.

4 is a flow chart of the process of dynamically expanding a batch server according to the present invention.

The specific embodiments are described below to illustrate the embodiments of the invention, but are not intended to limit the scope of the invention.

Please refer to FIG. 1 , which is a schematic diagram of a system platform of a cloud batch scheduling system according to the present invention. The foregoing system includes an execution server terminal 100 and a management server terminal 200. The foregoing execution server terminal 100 includes at least one execution server, and the management server terminal 200 also includes at least one management server, and the server terminal can be Adjust the number of working servers according to the workload.

The execution server includes a batch execution agent module 110 (agent: Agent), and a software module such as an automatic extension batch server module 120, and a batch execution agent. An execution result collection module 111 is disposed inside the module 110. The execution server includes hardware components such as a communication port, a memory, a processor, and the like, and the communication port is used to communicate with the foregoing management server. The memory includes one or more software modules, and the processor is connected to the communication port and the memory. At the time of execution, the software module in the memory is called and executed. The batch execution agent module 110 is configured to execute the work assigned by the management server, and the execution result collection module 111 periodically synchronizes the batch program execution information and the system resource information to the management server. The foregoing execution information includes memory usage, hard disk consumption, and number of batch program executions. The automatic extension batch server module 120 checks whether the resources are insufficient before each batch program is assigned to execute the agent, and expands the number of agents when insufficient.

The management server includes a hardware component such as a communication port, a memory, a processor, a communication port for communicating with the foregoing execution server, the memory includes a batch management server software module 210, and the processor is connected to the communication port and the memory. And the batch management server software module 210 is called and executed under the execution of the assignment. The foregoing batch management server software module 210 includes the following software modules: a resource monitoring and measurement module 211, a blueprint work execution flow engine 212, a batch group classification module 213, and a dynamic dispatch batch. Work module 214. The batch management server software module 210 is used for dispatching and classifying work, collecting execution information and analysis, workflow control, and providing a working-side management interface to set a work definition file, a blueprint definition file, a group classification profile, and a row. The program profile, the aforementioned profile and profile can be a physical file file (txt file), or a database file (DB file).

In another embodiment, the batch management server software module 210 is a batch management server computer program product, and the batch management server computer program product can be written in a programming language such as C, C++, Java, or the like. However, the type of programming language is not limited to this.

The foregoing work definition file is used to define the work to be performed. The definition scope includes the programming language of the work writing, the Chinese description of the work, whether the setting parameters have preset values, the number of parameters, etc., and the foregoing definition content is used as a schedule. Initialization data referenced during work.

The blueprint definition file is a blueprint structure formed by a set of nodes formed by the execution order of the work room. Referring to the description of FIG. 2, the execution sequence of the foregoing working room constitutes that the node 1 connects the node 2 through the parameter 1, and the node (n-1) connects the blueprint structure of the node n through the parameter (n-1). The blueprint structure includes the periodicity. Unscheduled execution time.

The group classification setting file classifies the blueprint work, and the limited work is performed on a specific allocated resource. The scheduling profile is scheduled for the blueprint. The scheduling information includes the expected execution time, the parameter value input by the user terminal, whether to send the notification after execution, and the programming language to be executed.

The resource monitoring and measurement module 211 accesses and analyzes execution information of the execution server to provide measurement data. The blueprint work execution flow engine 212 is used for dispatching blueprint work, the blueprint work described above is assigned by the blueprint definition file, and the blueprint definition file is the blueprint structure formed by the node set formed by the execution order of the work; The group classification module 213 performs batch group classification on the blueprint work to provide a group classification profile; the dynamic dispatch batch work module 214 receives the blueprint work, and according to the measurement data, the blueprint work file, and the group classification. The profile is used to calculate the load of the execution server to dynamically batch assign work to the corresponding execution server (eg, assign work to the execution server with the lightest load).

Please refer to the description of Figures 1, 3 and 4 together, which is an embodiment of the present invention. The working terminal adds work settings through the work management interface provided by the batch management server module. After the work is added, the corresponding work configuration file is generated (as shown in the following table). For example, the user adds JOB1. JOB2 and JOB3 work in three different batches, and correspondingly generate three different job profiles. The job profile contains the written programming language, Chinese description, whether the parameters have preset values, and the number of parameters. Job information, where JOB1 requires two parameters, and the second parameter does not set a preset value.

Work profile

After the work is set, the use terminal adds the blueprint work setting through the work management interface, and inputs the parameter value that will be used when the blueprint work is executed, and the scheduled execution time of the last schedule also generates the blueprint profile and the schedule profile accordingly. For example, the blueprint BPT1 is set to JOB1→JOB2, the execution order is JOB1 after the execution of JOB1 is successful, and the blueprint BPT001 is set periodically at 10:30 every month. The blueprint BPT2 is set to a single JOB3, and the blueprint BPT2 is set to perform at a time of 9:30 on February 3, 2015, and the notification is sent after execution.

Blueprint profile

The batch group can also selectively set the batch group classification module 213. After the setting, the corresponding group classification profile will be generated. For example, there are three batch programs of JOB1, JOB2 and JOB3, and JOB1 and JOB2 are The classification module is classified into the same group GROUP1, and GROUP1 is added to the qualification rule, which can only be executed on a machine named AGENT1, and will be forced to suspend when the system memory usage reaches 80%, pause to low usage rate. At 80%, there is no limit to any group in JOB3.

Group classification profile

3 is a flow chart of the distributed work execution flow of the present invention. The server triggers the start process after the execution time of the blueprint BPT1 at 9:30 on February 3, 2015, and the blueprint execution execution engine starts. Using the blueprint workflow (S101) set by the end, it is first determined whether there is still a blueprint work waiting to be executed (S102), JOB1 is first executed, and then the setting information of the initial scheduling of the use end (S103) is obtained, including the parameter value. Execution result collection module on each agent 111 will periodically synchronize batch program execution information and system resource information to the monitoring and measurement module. The server obtains batch related information through the monitoring and measurement module (S104), according to system memory usage rate and hard disk consumption. The quantity and the number of executions of the batch program are analyzed to obtain the measurement data. If the batch group classification module 213 is used, the system will capture the batch group classification file in addition to the measurement data (S105). ), assigning the agent that is currently the lightest for the batch workload to execute the batch program (S106). After the group classification profile of JOB1 determines the executed AGENT1, the server will set the work profile and batch classification. The file is transmitted to the Agent (S107). The work profile contains the data required for the job execution. JOB1 is the Java programming language. The parameter values entered during execution are 1 and 2 respectively. The group classification profile sets the file. The system resources that the batch work can access or be restricted, the JOB1 batch program will be set to be forced to suspend when the system memory usage reaches 80%. The last batch program will be executed smoothly on the assigned AGENT1 (S108). When the work execution starts, the work will first read the schedule configuration file to achieve execution on the restricted resource (S109). Then judge whether there is any next work until all the blueprint work is completed, JOB2 continues to execute, and after JOB2 is executed (S110), the work execution flow in the blueprint is ended.

Schedule profile

The aforementioned parameter value refers to the data required for the execution of the batch program, and is input by the user. The purpose of the parameter value is to become a bridge between the user and the batch program. The purpose of the parameter is to allow the user to determine the parameter value. The scope and method of data processed by the batch program.

Figure 4 shows the dynamic extended batch server process. The dynamically extended batch server module checks whether the resources are insufficient before each batch is assigned an agent (S201), and does not extend the agent immediately (S202). The action of the extended agent includes a new virtual machine (VM), a deployment agent, an extended startup agent (S203), and the agent registers with the server (S204), the process is completed, and the server can be allocated next time. The batch is executed on this agent.

The detailed description of the preferred embodiments of the present invention is intended to be limited to the scope of the invention, and is not intended to limit the scope of the invention. The patent scope of this case.

100‧‧‧Execution server side

110‧‧‧ batch execution agent module

111‧‧‧Execution result collection module

120‧‧‧Automatically expand batch servo module

200‧‧‧Manage server side

210‧‧‧Batch Management Server Software Module

211‧‧‧Resource monitoring and measurement module

212‧‧‧Blueprint Work Execution Process Engine

213‧‧‧Batch group classification module

214‧‧‧Dynamic Dispatch Batch Work Module

Claims (10)

A cloud batch scheduling system includes: at least one execution server for performing a plurality of tasks; a management server for connecting and configuring the execution of the execution server, wherein the management server further comprises: a communication port for communicating with the execution server; a memory comprising a batch management server software module; a processor connecting the communication port and the memory, the processor calling and executing the work under execution of the work assignment The batch management server software module, wherein the batch management server software module comprises: a resource monitoring and measurement module, accessing and analyzing execution information of the execution servers to provide measurement data; blueprint A work execution flow engine for dispatching blueprint work, wherein the blueprint work is a work assigned by a blueprint definition file, wherein the blueprint definition file is a blueprint structure formed by a set of nodes formed by execution orders of the work spaces Batch group classification module, batch group classification of the blueprint work to provide group classification profile; dynamic dispatch batch work module Receiving the blueprint work, and the measured data based on the blueprint file work, the group classification profile, to perform such calculation load of servers, dynamically assigns the work batch. The cloud batch scheduling system of claim 1, wherein the dynamic dispatch batch work module is a distribution line execution agent. The cloud batch scheduling system of claim 2, wherein the execution server further expands the agent for performing the work if the execution server is insufficient. The cloud batch scheduling system of claim 1, wherein the dynamically dispatching batch work module assigns the work to the execution server with the lightest load. The cloud batch scheduling system of claim 1, wherein the blueprint structure package performs time periodically or irregularly. A batch management server computer program product for a cloud batch scheduling system, applied and executed on a management server, which is used for wiring and configuring at least one execution server to perform a plurality of jobs, batches The management server computer program product comprises: a resource monitoring and measurement module, accessing and analyzing execution information of the execution servers to provide measurement data; and a blueprint work execution flow engine for dispatching blueprint work, wherein The blueprint work is the work assigned by the blueprint definition file, wherein the blueprint definition file is a blueprint structure formed by a set of nodes formed by the execution order of the work rooms; the batch group classification module works on the blueprint Performing batch group classification to provide a group classification profile; dynamically dispatching a batch work module, receiving the blueprint work, and calculating the data according to the measurement data, the blueprint work file, and the group classification profile Wait for the load of the server to dynamically allocate the work. The batch management server computer program product of claim 6, wherein the dynamic dispatch batch work module is a distribution line execution agent. The batch management server computer program product of claim 7, wherein the execution server is more The agent that performs the work is extended when the execution is insufficient. The batch management server computer program product of claim 6, wherein the dynamically dispatching batch work module assigns the work to the execution server having the lightest load. The batch management server computer program product of claim 6, wherein the blueprint structure package performs time periodically or irregularly.