CN116700919A

CN116700919A - Task processing method and device, computer readable storage medium and electronic equipment

Info

Publication number: CN116700919A
Application number: CN202310491574.2A
Authority: CN
Inventors: 李昂
Original assignee: BOE Technology Group Co Ltd; Beijing BOE Technology Development Co Ltd
Current assignee: BOE Technology Group Co Ltd; Beijing BOE Technology Development Co Ltd
Priority date: 2023-05-04
Filing date: 2023-05-04
Publication date: 2023-09-05

Abstract

The disclosure relates to a task processing method and device, a computer readable storage medium and electronic equipment, and relates to the technical field of computer resource scheduling, wherein the method comprises the following steps: receiving a first task execution instruction sent by terminal equipment; the first task execution instruction is generated by the terminal equipment in response to an input event aiming at a task configuration interface; analyzing the first task execution instruction to obtain a task to be executed and first task configuration information corresponding to the task to be executed; matching a target task execution node corresponding to the task to be executed from current task execution nodes according to the first task configuration information; generating a scheduling task according to the first task configuration information and a target task execution node, and executing the scheduling task in the target task execution node to obtain a task execution result corresponding to the task to be executed. The present disclosure improves task execution efficiency.

Description

Task processing method and device, computer readable storage medium and electronic equipment

Technical Field

The embodiment of the disclosure relates to the technical field of computer resource scheduling, in particular to a task processing method, a task processing device, a computer readable storage medium and electronic equipment.

Background

In the existing method, task execution nodes cannot be automatically matched for the tasks to be executed according to task configuration information, so that the task execution efficiency is low.

It should be noted that the information of the present invention in the above background section is only for enhancing understanding of the background of the present disclosure, and thus may include information that does not form the prior art that is already known to those of ordinary skill in the art.

Disclosure of Invention

An object of the present disclosure is to provide a task processing method, a task processing device, a computer-readable storage medium, and an electronic apparatus, which further overcome, at least to some extent, the problem of low execution efficiency of tasks due to limitations and drawbacks of the related art.

According to one aspect of the present disclosure, there is provided a task processing method including:

receiving a first task execution instruction sent by terminal equipment; the first task execution instruction is generated by the terminal equipment in response to an input event aiming at a task configuration interface;

analyzing the first task execution instruction to obtain a task to be executed and first task configuration information corresponding to the task to be executed;

matching a target task execution node corresponding to the task to be executed from current task execution nodes according to the first task configuration information;

Generating a scheduling task according to the first task configuration information and a target task execution node, and executing the scheduling task in the target task execution node to obtain a task execution result corresponding to the task to be executed.

In an exemplary embodiment of the present disclosure, according to the first task configuration information, matching a target task execution node corresponding to the task to be executed from current task execution nodes, including:

acquiring current task type information included in the first task configuration information, and matching and executing a first task execution node of a history executed task consistent with the current task type information from a task scheduling template;

acquiring a current task execution node in an idle state from a resource pool, and judging whether the first task execution node exists in the current task execution node or not;

if the first task execution node exists in the current task execution node, the first task execution node is used as a target task execution node corresponding to the task to be executed;

and if the first task execution node does not exist in the current task execution node, matching the target task execution node corresponding to the task to be executed from the current task execution node according to the first task configuration information.

In an exemplary embodiment of the present disclosure, matching, according to the first task configuration information, a target task execution node corresponding to the task to be executed from the current task execution nodes, includes:

acquiring current node configuration information of the current task execution node, and judging whether target node configuration information matched with the first task configuration information exists in the current node configuration information;

if the current node configuration information contains target node configuration information matched with the first task configuration information, using the current task execution node corresponding to the target node configuration information as a target task execution node corresponding to the task to be executed.

In an exemplary embodiment of the present disclosure, the task processing method further includes:

if the current node configuration information does not have the target node configuration information matched with the first task configuration information, generating first failure prompt information corresponding to a target task execution node, and sending the first failure prompt information to the terminal equipment;

and receiving a second task execution instruction which is sent by the terminal equipment after the first task configuration information in the first task execution instruction is dynamically adjusted according to the first failure prompt information, and matching a target task execution node corresponding to the task to be executed from the current task execution node according to the second task configuration information included in the second task execution instruction.

according to the first task configuration information and the current node configuration information, calculating the network configuration adaptation degree and/or the resource configuration adaptation degree between the task to be executed and the current task execution node;

and matching the target task execution node corresponding to the task to be executed from the current task execution nodes according to the network configuration adaptation degree and/or the resource configuration adaptation degree.

In an exemplary embodiment of the present disclosure, matching, according to the network configuration adaptation degree and/or the resource configuration adaptation degree, a target task execution node corresponding to the task to be executed from the current task execution nodes includes:

sorting the current task execution nodes according to the network configuration adaptation degree to obtain a first node sorting result, and selecting the current task execution node with the largest network adaptation degree from the first node sorting result as a target task execution node corresponding to the task to be executed; or alternatively

Sorting the current task execution nodes according to the resource allocation adaptation degree to obtain a second node sorting result, and selecting the current task execution node with the largest resource adaptation degree from the second node sorting result as a target task execution node corresponding to the task to be executed; or alternatively

Calculating the total adaptation degree between the task to be executed and the current task execution node according to the network configuration adaptation degree and the resource configuration adaptation degree, and sequencing the current task execution node according to the total adaptation degree to obtain a third node sequencing result;

and selecting the current task execution node with the maximum resource adaptation degree from the third node sequencing result as the target task execution node corresponding to the task to be executed.

In an exemplary embodiment of the present disclosure, calculating a total adaptation degree between the task to be performed and the current task performing node according to the network configuration adaptation degree and the resource configuration adaptation degree includes:

configuring a first weight value and a second weight value for the network configuration adaptation degree and the resource configuration adaptation degree; wherein the first weight value is greater than or equal to the second weight value;

Calculating a first product operation result between the first weight value and the network configuration adaptation degree, and calculating a second product operation result between the second weight value and the resource configuration adaptation degree;

and calculating a first sum operation result between a first product operation result and a second product operation result, and obtaining the total adaptation degree between the task to be executed and the current task execution node according to the first sum operation result.

In an exemplary embodiment of the present disclosure, the first task configuration information includes first network configuration information and/or first resource configuration information; the current node configuration information comprises second network configuration information and/or second resource configuration information;

according to the first task configuration information and the current node configuration information, calculating a network configuration adaptation degree and/or a resource configuration adaptation degree between the task to be executed and the current task execution node, including:

according to the first network configuration information and the second network configuration information, calculating the network configuration adaptation degree between the task to be executed and the current task execution node; and/or

And calculating the resource allocation adaptation degree between the task to be executed and the current task execution node according to the first resource allocation information and the second resource allocation information.

In an exemplary embodiment of the present disclosure, the first network configuration information includes a first network bandwidth and/or a first network delay, and the second network configuration information includes a second network bandwidth and/or a second network delay;

according to the first network configuration information and the second network configuration information, calculating the network configuration adaptation degree between the task to be executed and the current task execution node, including:

calculating a first quotient operation result between the first network bandwidth and the second network bandwidth; and/or

Calculating a second quotient result between the first network delay and a second network delay;

and obtaining the network configuration adaptation degree between the task to be executed and the current task execution node according to the first quotient operation result and/or the second quotient operation result.

In one exemplary embodiment of the disclosure, the first resource configuration information includes one or more of a resource configuration of the first central processor, a resource configuration of the first memory, and a resource configuration of the first graphics processor;

the second resource configuration information comprises one or more of a resource configuration of a second central processing unit, a resource configuration of a second memory and a resource configuration of a second graphic processor;

According to the first resource configuration information and the second resource configuration information, calculating the resource configuration adaptation degree between the task to be executed and the current task execution node includes:

calculating a third quotient operation result between the resource allocation of the second central processing unit and the resource allocation of the first central processing unit; and/or

Calculating a fourth quotient operation result between the resource allocation of the second memory and the resource allocation of the first memory; and/or

Calculating a fifth quotient operation result between the resource allocation of the second graphic processor and the resource allocation of the first graphic processor;

and obtaining the resource allocation adaptation degree between the task to be executed and the current task execution node according to the third quotient operation result and/or the fourth quotient operation result and/or the fifth quotient operation result.

In an exemplary embodiment of the present disclosure, executing the scheduling task in the target task execution node, to obtain a task execution result corresponding to the task to be executed, includes:

issuing the scheduling task to the target task execution node, and submitting a container task mirror image included in the scheduling task to a task execution tool in a container orchestration engine through the target task execution node;

And executing the container task mirror image through the task execution tool to obtain a task execution result corresponding to the task to be executed.

In an exemplary embodiment of the present disclosure, executing, by the task execution tool, the container task image to obtain a task execution result corresponding to the task to be executed, including:

executing the task mirror image of the container through the task execution tool to obtain a task execution container;

and starting and executing the executable mirror image included in the task execution container to obtain a task execution result corresponding to the task to be executed.

in the process of executing mirror image running, monitoring a first running state of the task execution container and a second running state of a task execution tool in real time, and generating a monitoring log according to the first running state and the second running state; and/or

And displaying the first running state and the second running state.

deleting the executable mirror image in the task execution container when the executable mirror image execution is determined to be completed, and releasing second network occupied resources and second memory occupied resources of target task execution nodes occupied by executing the scheduling task;

Updating the node state of the target task executing node from the occupied state to the idle state, and placing the target task executing node in the idle state into a resource pool.

and calling a task callback interface corresponding to the terminal equipment, and feeding back the task execution result to the terminal equipment through the task callback interface.

In an exemplary embodiment of the present disclosure, the first task execution instruction is generated by the terminal device in the following manner:

displaying a task execution interface on a display interface of the terminal equipment;

responding to a first input event aiming at a task execution interface, determining a task to be executed, and displaying a task configuration interface corresponding to the task to be executed; the tasks to be executed comprise model training tasks and/or data analysis tasks;

determining first task configuration information of a task to be executed in response to a second input event acting on the task configuration interface;

and responding to a third input event acting on the task configuration interface, generating a first task execution instruction according to the task to be executed and the first task configuration information, and sending the first task execution instruction to a task execution framework.

According to an aspect of the present disclosure, there is provided a task processing device including:

the first receiving module is used for receiving a first task execution instruction sent by the terminal equipment; the first task execution instruction is generated by the terminal equipment in response to an input event aiming at a task configuration interface;

the first analysis module is used for analyzing the first task execution instruction to obtain a task to be executed and first task configuration information corresponding to the task to be executed;

the task execution node determining module is used for matching a target task execution node corresponding to the task to be executed from the current task execution nodes according to the first task configuration information;

and the task execution module is used for generating a scheduling task according to the first task configuration information and a target task execution node, and executing the scheduling task in the target task execution node to obtain a task execution result corresponding to the task to be executed.

According to one aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the task processing method of any one of the above.

According to one aspect of the present disclosure, there is provided an electronic device including:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform any one of the task processing methods described above via execution of the executable instructions.

According to the task processing method provided by the embodiment of the disclosure, on one hand, a first task execution instruction sent by a terminal device is received; then analyzing the first task execution instruction to obtain a task to be executed and first task configuration information corresponding to the task to be executed; matching a target task execution node corresponding to the task to be executed from the current task execution nodes according to the first task configuration information; finally, generating a scheduling task according to the first task configuration information and the target task execution node, and executing the scheduling task in the target task execution node to obtain a task execution result corresponding to the task to be executed; because the target task execution node can be matched according to the first task configuration information, and then the task is executed in the target task execution node, the problem that the task execution node cannot be matched for the task to be executed according to the task configuration information automatically in the prior art, and further the task execution efficiency is lower is solved; on the other hand, the problem of node resource waste of the target task execution node can be avoided; on the other hand, the first task execution instruction is generated by the terminal equipment in response to the input event aiming at the task configuration interface, so that flexible configuration of the task to be executed is realized.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort.

Fig. 1 schematically illustrates a flow chart of a task processing method according to an example embodiment of the present disclosure.

FIG. 2 schematically illustrates a block diagram of a task processing system according to an example embodiment of the present disclosure.

Fig. 3 schematically illustrates a structural example diagram of a server side according to an exemplary embodiment of the present disclosure.

Fig. 4 schematically illustrates a structural example diagram of a containerized application management system Kubernetes according to an example embodiment of the present disclosure.

Fig. 5 schematically illustrates an example diagram of a specific generation process of a first task execution instruction according to an example embodiment of the present disclosure.

FIG. 6 schematically illustrates an interface example diagram of a task configuration interface, according to an example embodiment of the present disclosure.

Fig. 7 schematically illustrates a flowchart of a method for matching a target task execution node corresponding to the task to be executed from current task execution nodes according to the first task configuration information according to an exemplary embodiment of the present disclosure.

FIG. 8 schematically illustrates an interface example diagram of a display interface resulting from a scheduled task creation failure, according to an example embodiment of the present disclosure.

Fig. 9 schematically illustrates an interface exemplary diagram of a use state of a first central processor, a use state of a first memory, a use state of a first graphics processor, according to an exemplary embodiment of the present disclosure.

Fig. 10 schematically illustrates an example diagram of an interface of a monitoring log according to an example embodiment of the present disclosure.

FIG. 11 schematically illustrates a flow chart for dynamically adjusting nodes that are abnormally operated according to an example embodiment of the present disclosure.

Fig. 12 schematically illustrates an interface presentation of a resource release scenario according to an example embodiment of the present disclosure.

Fig. 13 schematically illustrates a block diagram of a task processing device according to an example embodiment of the present disclosure.

Fig. 14 schematically illustrates an electronic device for implementing the task processing method described above according to an exemplary embodiment of the present disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present disclosure. One skilled in the relevant art will recognize, however, that the aspects of the disclosure may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.

In some Kubernetes-based resource scheduling schemes, which provide a Job grouping algorithm and a resource allocation policy, only partial resource usage and reclamation are considered, and the considered resource matching is not comprehensive; at the same time, ease of use considerations are lacking since it provides scheduling policies only for the pre-existing job group. Further, in some schemes based on intelligent scheduling, although an intelligent scheduling policy is provided, the scheme lacks a corresponding elastic mechanism for other distributed cooperation fields; meanwhile, the scheme does not support user decision, and a user cannot timely fine-tune the strategy when errors occur, and only strategy allocation resources can be regenerated.

Based on this, a task processing method is provided in this exemplary embodiment, and the method may be executed on a server, a server cluster, a cloud server, or the like; of course, those skilled in the art may also operate the methods of the present disclosure on other platforms as desired, which is not particularly limited in the present exemplary embodiment. Specifically, referring to fig. 1, the task processing method may include the following steps:

s110, receiving a first task execution instruction sent by terminal equipment; the first task execution instruction is generated by the terminal equipment in response to an input event aiming at a task configuration interface;

S120, analyzing the first task execution instruction to obtain a task to be executed and first task configuration information corresponding to the task to be executed;

s130, matching a target task execution node corresponding to the task to be executed from current task execution nodes according to the first task configuration information;

and S140, generating a scheduling task according to the first task configuration information and a target task execution node, and executing the scheduling task in the target task execution node to obtain a task execution result corresponding to the task to be executed.

In the task processing method, on one hand, a first task execution instruction sent by the terminal equipment is received; then analyzing the first task execution instruction to obtain a task to be executed and first task configuration information corresponding to the task to be executed; matching a target task execution node corresponding to the task to be executed from the current task execution nodes according to the first task configuration information; finally, generating a scheduling task according to the first task configuration information and the target task execution node, and executing the scheduling task in the target task execution node to obtain a task execution result corresponding to the task to be executed; because the target task execution node can be matched according to the first task configuration information, and then the task is executed in the target task execution node, the problem that the task execution node cannot be matched for the task to be executed according to the task configuration information automatically in the prior art, and further the task execution efficiency is lower is solved; on the other hand, the problem of node resource waste of the target task execution node can be avoided; on the other hand, the first task execution instruction is generated by the terminal equipment in response to the input event aiming at the task configuration interface, so that flexible configuration of the task to be executed is realized.

The task processing method described in the exemplary embodiments of the present disclosure will be further explained and illustrated below with reference to the accompanying drawings.

First, proper nouns involved in the exemplary embodiments of the present disclosure are explained and explained.

Kubernetes: may also be referred to as K8s for short, is an open source system for automatically deploying, scaling and managing containerized applications.

Interface callback: also referred to as Callback, refers to a feedback mechanism that triggers after some task is performed.

And (3) monitoring a container: mainly refers to the state monitoring and the running state checking of the container generated by the k8s service.

Next, the technical implementation principle of the exemplary embodiments of the present disclosure will be explained and illustrated. Specifically, the task processing method described in the exemplary embodiments of the present disclosure provides an implementation method of Kubernetes resource scheduling system supporting multi-node task collaboration, which aims to solve the problems of how to select an execution node and monitor the execution condition of the node and simplify Kubernetes management in a large-scale deep learning training task or a multi-node concurrent collaboration task; meanwhile, the complex scheduling problems of Kubernetes service call, container management and the like in actual production practice can be solved; in the actual application process, the task management method may involve a plurality of different aspects such as task management, interface callback, and container monitoring.

Further, the task processing systems referred to in the exemplary embodiments of the present disclosure are explained and illustrated. Specifically, referring to fig. 2, the task processing system may include a terminal device 210 and a server 220; the terminal equipment can be in communication connection with the server side in a wired network or wireless network mode; meanwhile, in the actual application process, the terminal equipment can be used for generating a task execution instruction and sending the task execution instruction to the server side; the server side may be used to implement the task processing method described in the exemplary embodiments of the present disclosure.

In an exemplary embodiment, referring to fig. 3, the server side described herein may also be considered a server side where the task execution framework is located; meanwhile, in the task execution framework described in the exemplary embodiment of the present disclosure, it may include a task registration module 310, a deployment scheduling module 320, a task management module 330, a resource monitoring module 340, a resource reclamation module 350, a callback management module 360, a containerized application management system Kubernetes370, and a call service interface 380; wherein the connection relationship between the modules can be shown with reference to fig. 3. The task registration module 310 may be configured to read task configuration information in the task execution instruction, and register the task configuration information to the task management module 330; the task management module 330 may be configured to load task configuration information, read Kubernetes node status, and further generate a scheduling task according to the task configuration information and the node status in combination with an intelligent scheduling policy, and issue the scheduling task to the deployment scheduling module 320; the deployment scheduling module 320 reads scheduling task information, issues jobs to Kubernetes, registers information such as Pod and the like included in jobs and jobs to the resource monitoring module 340 after jobs run successfully, and monitors running states such as follow-up jobs, pod and the like by the resource monitoring module 340; the resource monitoring module 340 may be configured to monitor the running states of Job, pod, etc., and feed back the running states to the task management module 330 in real time, so as to display the running situation of the task through the task management module 330; the resource recycling module 350 may be configured to release node resources used for executing the task after receiving the task completion information issued by the task management module, and return the node resources to the resource pool; the callback management module 360 may be configured to allow a user to register a callback address, and after receiving a callback request, feed back execution information and execution results of a task sent by the task management module to an external service in a callback task manner through the call service interface 380, so as to facilitate docking and integration of the external service.

In an example embodiment, the task registration module described above may include a configuration parsing module, a subtask disassembling and registering module, and a callback registration module; the configuration analysis module can be used for analyzing the first task execution instruction so as to obtain first task configuration information; the subtask disassembly module can be used for carrying out task disassembly and/or task registration according to the first task configuration information; and meanwhile, the callback registration module can be used for receiving a registration request of a callback interface sent by a user and registering a calling service interface, so that a task execution result or a character execution log is fed back based on the calling service interface. Further, in the practical application process, the task may be transmitted in the form of a data packet via an information channel (the information channel may be, for example, a message queue telemetry transport MQTT or the like); for example, the scheduling task is transmitted from the task registration module to the task management module through an information channel, or alternatively, from the task management module to the deployment scheduling module, or the like. Meanwhile, the data packet recorded herein can cover the resource information of the current task execution node in an idle state and the like; the resource monitoring module can monitor node information of the current task execution node, the idle task execution node and the like, and further write corresponding information into the data packet.

In one example embodiment, the resource pool described herein is a resource pool provided in a task execution framework for placing a current task execution node in an idle state; one or more current task execution nodes can be included in the resource pool; meanwhile, in the current task execution node, a service based on a plurality of nodes can be provided under a micro-service-oriented architecture based on a Kubernetes technology, and the service can be performed based on cooperation among multiple nodes; meanwhile, in the process of deploying the current task execution node, the following two different situations may exist: one is that each current task execution node may be respectively set in different servers, that is, only one current task execution node is set in one server; in this scenario, when performing actual task management or task execution, a collaborative job task depending on different servers may be included, or a task that may be independently executed by a single server may be included; in addition, the method can be suitable for a large-scale cluster deployment environment of massive data under the scene of relying on collaborative operation tasks among different servers, and can be beneficial to the operation of intelligent parks/intelligent cities; alternatively, the current task execution nodes may be set in the same server, that is, a plurality of current task execution nodes may be set in one server, or the current task execution nodes may be uniformly set in a certain number of servers; in this scenario, when tasks are actually executed, the cooperation of virtual memories opened up for different nodes under the server is relied on, and this case is used in a deployment environment involving less data, for example, a banking scenario under a single function (such as passenger flow identification).

In one example embodiment, a containerized application management system Kubernetes may be used to perform scheduling tasks. As shown in fig. 4, the containerized application management system Kubernetes may include a container (Pod) 410, a Service port (Service) 420 exposed outside the container, a container load balancing (Ingress) 430, a first management tool (discovery) 440, and a second management tool (Job) 450; wherein the container may be used to perform the corresponding task; the service port is used for providing communication connection between the container and the outside, for example, the scheduling task can be written into the Pod through the service port; the container load balancing can be used for realizing the load balancing of the Pod, so as to achieve the purpose of balancing the load of each Pod; the first management tool may be used to manage Pod, service ports, and container load balancing, and the second management tool may be used to control Pod to perform the corresponding tasks.

Based on the foregoing, it can be seen that the task processing system according to the exemplary embodiments of the present disclosure has the following characteristics compared to other Kubernetes-based scheduling systems: on the one hand, the configuration is simple and flexible; specifically, the scheduling is managed from the dimension of the task, only basic information and requirements for task operation are needed to be provided, and a Kubernetes configuration file can be automatically generated and the task is issued; on the other hand, the monitoring system is perfect; specifically, the execution condition can be monitored from the full life cycle of the task, and the task creation state, the execution condition, the completion condition and the resource recovery are started from the creation condition verification of the task; on the other hand, the system can run through the whole task flow, can alarm the user when any problem occurs, and allows the user to make corresponding decisions; furthermore, the system can also automatically generate a scheduling strategy according to the running requirement information configured by the user, select the node for executing the task, complete the task issuing, and further avoid the problem of failure execution caused by the fact that the user does not meet the requirement on the selected resource.

The specific generation process of the first task execution instruction referred to in the present disclosure will be explained and explained with reference to fig. 5. Specifically, referring to fig. 5, a specific generation process of the first task execution instruction may include the following steps:

step S510, displaying a task execution interface on the display interface of the terminal device.

Step S520, a task to be executed is determined in response to a first input event aiming at a task execution interface, and a task configuration interface corresponding to the task to be executed is displayed; the tasks to be executed comprise model training tasks and/or data analysis tasks.

Step S530, in response to a second input event acting on the task configuration interface, determines first task configuration information of a task to be executed.

Step S540, in response to a third input event acting on the task configuration interface, generates a first task execution instruction according to the task to be executed and the first task configuration information, and sends the first task execution instruction to a task execution framework.

Hereinafter, step S510 to step S540 will be explained and explained. Specifically, in the practical application process, there is a scenario in which a corresponding task needs to be executed at a server side, which is hindered by the limitation of computing power or data security at the terminal device side; based on the scene, when the service end is required to be requested to assist in executing the task, firstly, a task execution interface can be displayed on a display interface of the terminal equipment; the task execution interface described herein may be implemented based on a certain client; that is, the task execution interface can be displayed by opening the corresponding client; then, a user can determine a corresponding task to be executed according to actual needs, and further a task configuration interface corresponding to the task to be executed is displayed; the tasks to be executed described herein may include a model training task, a data analysis task, an interface test task, and the like, which is not particularly limited in this example; meanwhile, the task configuration interface described herein may be specifically shown in fig. 6; furthermore, the user can input corresponding task configuration information according to the displayed task configuration interface; the task configuration information described herein may include mirror information, mount file information, resource configuration information, node type information, node specification information, delay start-stop information, operation command information, network bandwidth information, network delay information, operation parameter mode information, operation parameter information, and operation command preview information, etc. The mirror image information described herein may include a mirror image name, a mirror image version, and the like, and the mount file may include a corresponding script file and the like; the resource configuration information can comprise single machine training or multiple machines, multiple cards and the like, and the node type can comprise a CPU or a GPU and the like; meanwhile, after the task configuration information is input, a task issuing control on the task configuration interface can be clicked, so that a first task execution instruction is obtained, and the task instruction is submitted to a task execution framework, namely a server side.

It should be noted that, because the tasks may include different types, the task configuration information included in the task configuration interface described herein is described by taking a model training task as an example, and for other tasks, such as a data request task or an interface test task, a corresponding task configuration interface may be displayed according to actual situations, which is not limited in this example; meanwhile, the model training tasks described herein may include deep neural network model training tasks, federal model training tasks, and cross-modal model (which may be understood as tasks of performing comprehensive semantic understanding on the basis of merging a plurality of single-mode features, and further completing specific classification, matching, generation, and the like) training tasks, and the like.

The task processing method shown in fig. 1 will be further explained and explained below with reference to fig. 2 to 6. Specific:

in step S110, a first task execution instruction sent by a terminal device is received; the first task execution instruction is generated by the terminal equipment in response to an input event aiming at a task configuration interface.

Specifically, the first task execution instruction is defined herein to be generated by the terminal device in response to the input event for the task configuration interface, so that if the scheduled task is detected to be created to fail during the creation of the scheduled task, the task configuration information included in the first task execution instruction can be manually adjusted, so that the problem that the task to be processed cannot be executed due to the failure of the creation of the scheduled task is avoided.

In step S120, the first task execution instruction is parsed to obtain a task to be executed and first task configuration information corresponding to the task to be executed.

Specifically, the first task configuration information described herein may include task image information, mount file information, first network configuration information, first resource configuration information, and so on; the first network configuration information may include first network bandwidth information, first network delay information, and the like, and the first resource configuration information may include first CPU information, first GPU information, first memory information, and the like. In the actual application process, the task configuration information can be configured according to actual needs, and the example does not have special limitation on the task configuration information; meanwhile, for different types of tasks to be executed, the corresponding first task configuration information may be partially the same or completely different, which is not particularly limited in this example.

In step S130, according to the first task configuration information, a target task execution node corresponding to the task to be executed is matched from current task execution nodes.

Specifically, referring to fig. 7, according to the first task configuration information, matching a target task execution node corresponding to the task to be executed from current task execution nodes may include the following steps:

Step S710, obtaining current task type information included in the first task configuration information, and matching and executing a first task execution node of a history executed task consistent with the current task type information from a task scheduling template;

step S720, obtaining a current task execution node in an idle state from a resource pool, and judging whether the first task execution node exists in the current task execution node;

step S730, if the first task execution node exists in the current task execution node, using the first task execution node as a target task execution node corresponding to the task to be executed;

step S740, if the first task execution node does not exist in the current task execution node, matching a target task execution node corresponding to the task to be executed from the current task execution node according to the first task configuration information.

Hereinafter, step S710 to step S740 will be explained and explained. Specifically, in the actual application process, the first task execution node may be matched according to the current task type information of the task to be executed; that is, judging whether a consistent historical task consistent with the current task type information exists in the task scheduling template; if yes, determining a first task execution node to which the historical task is distributed; of course, it is also necessary to determine whether the first task execution node is in an idle state, and if so, the first task execution node may be directly used as a target task execution node; if the first task execution node does not exist or is in an occupied state, the target task execution node corresponding to the task to be executed is matched from the current task execution nodes according to the first task configuration information.

In an example embodiment, according to the first task configuration information, matching a target task execution node corresponding to the task to be executed from the current task execution nodes may be implemented as follows: firstly, acquiring current node configuration information of the current task execution node, and judging whether target node configuration information matched with the first task configuration information exists in the current node configuration information; secondly, if target node configuration information matched with the first task configuration information exists in the current node configuration information, taking the current task execution node corresponding to the target node configuration information as a target task execution node corresponding to the task to be executed; further, if the current node configuration information does not have the target node configuration information matched with the first task configuration information, generating first failure prompt information corresponding to the target task execution node, and sending the first failure prompt information to the terminal equipment; and finally, receiving a second task execution instruction which is sent by the terminal equipment after the first task configuration information in the first task execution instruction is dynamically adjusted according to the first failure prompt information, and matching a target task execution node corresponding to the task to be executed from the current task execution node according to the second task configuration information included in the second task execution instruction. Specifically, in the actual application process, whether the current node configuration information has target node configuration information corresponding to first task configuration information required for executing the task to be executed can be judged; that is, judging whether a target task execution node capable of executing the task to be executed exists in the current task execution nodes; if yes, directly determining the target task execution node; if the first failure prompt information does not exist, the first failure prompt information can be generated according to a specific mismatching reason; for example, the required network bandwidth or network delay is not matched, or the required CPU resource or GPU resource or memory resource is not matched, etc.; then, the first failure prompt message is sent to the terminal equipment; after the terminal equipment receives the first failure prompt information, the first task configuration information can be subjected to self-adaptive dynamic adjustment according to specific mismatch reasons, so that second task configuration information is obtained, a second task execution instruction is generated according to the second task configuration information, and the corresponding target task execution nodes are re-matched. The display interface obtained by the failure of the creation of the scheduled task may be specifically shown in fig. 8.

In one example embodiment, in the display interface of scheduled tasks shown in FIG. 8, it may include a plurality of different scheduled tasks, e.g., the scheduled tasks may include V0001-V0012, etc.; in the actual application process, under some special application scenes, a scene in which a plurality of scheduling tasks need to be executed simultaneously exists; for example, in the smart banking or smart medical scenario, a plurality of related tasks such as face recognition, human body recognition, passenger flow recognition, etc. are required to be executed, which are simultaneously set in the smart park; in this scenario, the task to be performed may include a platform for model training by a developer focused on the image recognition field, while performing a convolutional neural network model, an attention mechanism model, a two-way long-short-term memory network model, a cyclic neural network model, and the like; in this case, resources need to be reasonably allocated under the condition of limited computing power, and then a corresponding model can be trained at the fastest speed, so that the purpose of face recognition, human body recognition or passenger flow recognition can be achieved, and corresponding measures, such as a guest service or a general user service, are executed according to the face recognition result, the human body recognition result or the passenger flow recognition result.

In an example embodiment, according to the first task configuration information, matching a target task execution node corresponding to the task to be executed from the current task execution nodes may be implemented as follows: firstly, calculating network configuration adaptation degree and/or resource configuration adaptation degree between the task to be executed and the current task execution node according to the first task configuration information and the current node configuration information; and secondly, matching a target task execution node corresponding to the task to be executed from the current task execution nodes according to the network configuration adaptation degree and/or the resource configuration adaptation degree. That is, if a target task execution node corresponding to a task to be executed needs to be matched from the current task execution nodes, the matching can be considered from two dimensions of network configuration adaptation degree, resource configuration adaptation degree and the like; meanwhile, in the practical application process, a scheduling strategy can be generated according to the network configuration adaptation degree and the resource configuration adaptation degree; wherein, the scheduling only can be embodied in the following aspects: in one aspect, if only the first network configuration information is present in the first task configuration information and the first resource configuration information is not present, the target task execution node may be matched based only on the dimension of the network configuration adaptation degree; on the other hand, if only the first resource configuration information is present in the first task configuration information and the first network configuration information is not present, the target task execution node may be matched based on only the dimension of the resource configuration adaptation degree; in yet another aspect, the target task execution node may be matched from two dimensions if both the first resource configuration information and the first network configuration information are present in the first task configuration information.

In an example embodiment, according to the network configuration adaptation degree and/or the resource configuration adaptation degree, matching a target task execution node corresponding to the task to be executed from the current task execution nodes may be implemented as follows: the current task execution nodes are sequenced according to the network configuration adaptation degree to obtain a first node sequencing result, and the current task execution node with the largest network adaptation degree is selected from the first node sequencing result to serve as a target task execution node corresponding to the task to be executed; in another implementation manner, the current task execution nodes are ordered according to the resource allocation adaptation degree to obtain a second node ordering result, and the current task execution node with the largest resource adaptation degree is selected from the second node ordering result to serve as the target task execution node corresponding to the task to be executed; according to the network configuration adaptation degree and the resource configuration adaptation degree, calculating the total adaptation degree between the task to be executed and the current task execution node, and sequencing the current task execution node according to the total adaptation degree to obtain a third node sequencing result; and selecting the current task execution node with the maximum resource adaptation degree from the third node sequencing result as the target task execution node corresponding to the task to be executed. That is, in the actual application process, if the first task configuration information only includes the first network configuration information, the current task execution node with the greatest network configuration adaptation degree may be directly selected as the target task execution node; if the first task configuration information only comprises the first resource configuration information, the current task execution node with the maximum resource configuration adaptation degree can be directly selected as the target task execution node; of course, if the first task configuration information includes both the first resource configuration information and the first network configuration information, the resource configuration adaptation degree and the network configuration adaptation degree may be weighted and summed, so as to determine the corresponding target task execution node.

In an example embodiment, calculating the total adaptation degree between the task to be performed and the current task performing node according to the network configuration adaptation degree and the resource configuration adaptation degree may be implemented as follows: firstly, configuring a first weight value and a second weight value for the network configuration adaptation degree and the resource configuration adaptation degree; wherein the first weight value is greater than or equal to the second weight value; secondly, calculating a first product operation result between the first weight value and the network configuration adaptation degree, and calculating a second product operation result between the second weight value and the resource configuration adaptation degree; and then, calculating a first sum operation result between a first product operation result and a second product operation result, and obtaining the total adaptation degree between the task to be executed and the current task execution node according to the first sum operation result. Specifically, if the corresponding target task execution node is determined by weighting and summing the resource configuration adaptation degree and the network configuration adaptation degree, the corresponding first weight value and the corresponding second weight value are required to be configured for the network configuration adaptation degree and the resource configuration adaptation degree; the specific configuration process of the first weight value and the second weight value can be configured in a manual experience mode or a model prediction mode, and the specific configuration process is not limited in this example; meanwhile, when configured by means of model prediction, the network model used herein may include, but is not limited to, a convolutional neural network model, a cyclic neural network model, a deep neural network model, a decision tree model, and the like, which is not particularly limited in this example; meanwhile, the first weight value is limited to be larger than or equal to the second weight value, because the network configuration adaptation degree needs to be prioritized over the resource configuration adaptation degree in the actual application process, so that the aim of improving the task execution efficiency can be achieved.

In an example embodiment, the current node configuration information described above may include second network configuration information and/or second resource configuration information; according to the first task configuration information and the current node configuration information, calculating the network configuration adaptation degree and/or the resource configuration adaptation degree between the task to be executed and the current task execution node can be realized in the following manner: according to the first network configuration information and the second network configuration information, calculating the network configuration adaptation degree between the task to be executed and the current task execution node; and/or calculating the resource configuration adaptation degree between the task to be executed and the current task execution node according to the first resource configuration information and the second resource configuration information.

In an example embodiment, the second network configuration information described above may include a second network bandwidth and/or a second network delay; according to the first network configuration information and the second network configuration information, the network configuration adaptation degree between the task to be executed and the current task execution node is calculated, and the method can be realized by the following steps: calculating a first quotient operation result between the first network bandwidth and the second network bandwidth; and/or calculating a second quotient result between the first network delay and a second network delay; and obtaining the network configuration adaptation degree between the task to be executed and the current task execution node according to the first quotient operation result and/or the second quotient operation result. The specific calculation manner of the network configuration adaptation degree can be referred to as the following formula (1):

Wherein Fitness is the network configuration adaptation degree, bandwidth _require Is a first network bandwidth, i.e. a required network bandwidth; bandwith _actual Is a second network bandwidth, i.e. the actual network bandwidth; delay of _require Is a first network delay, i.e. the required network delay; delay of _actual Is the second network delay, i.e. the actual network delay. It should be added here that, since the network bandwidth and the network delay in the same area are substantially identical,therefore, the network bandwidths and network delays of different areas can be calculated in advance, and the matching degree is calculated based on the target bandwidths and delays; meanwhile, the quotient of the required bandwidth and the average value of the actual bandwidth of each node is summed with the quotient of the required delay and the average value of the actual delay of each node, and the larger the value is, the higher the adaptation degree is.

In an example embodiment, the first resource configuration information described above may include a resource configuration of the first central processor, a resource configuration of the first memory, and a resource configuration of the first graphics processor, and so on, and the second resource configuration information includes a resource configuration of the second central processor, a resource configuration of the second memory, and a resource configuration of the second graphics processor, and so on; according to the first resource configuration information and the second resource configuration information, the resource configuration adaptation degree between the task to be executed and the current task execution node is calculated, and the method can be realized by the following steps: calculating a third quotient operation result between the resource allocation of the second central processing unit and the resource allocation of the first central processing unit; and/or calculating a fourth quotient result between the resource allocation of the second memory and the resource allocation of the first memory; and/or calculating a fifth quotient result between the resource allocation of the second graphics processor and the resource allocation of the first graphics processor; and obtaining the resource allocation adaptation degree between the task to be executed and the current task execution node according to the third quotient operation result and/or the fourth quotient operation result and/or the fifth quotient operation result. The specific calculation formula of the resource allocation adaptation degree can be shown by the following formula (2):

Wherein Fitness is provided _node Is the resource allocation adaptation degree; CPU (Central processing Unit) _require Is the resource allocation of the first central processor, i.e. the resource allocation of the central processor required; CPU (Central processing Unit) _node The resource allocation of the second central processing unit and the actual central processing unit of the target task execution node; MEM (MEM) _require Is a first memoryResource allocation of the memory, namely the resource allocation of the memory required; MEM (MEM) _node The resource allocation of the second memory, namely the resource allocation of the actual memory of the target task execution node; GPU (graphics processing Unit) _require Is the resource configuration of the first graphics processor, i.e., the required resource configuration of the graphics processor; GPU (graphics processing Unit) _node Is the resource configuration of the second graphics processor, i.e. the actual graphics processor that the target task execution node has.

It should be noted that, the calculation mode of the resource adaptation degree is mainly determined according to the matching condition of the requirement of the hardware resource, for example, the requirement is 4cpu 8gb memory, and all the configurations greater than or equal to this meet the adaptation requirement; meanwhile, the more the node resource value is larger than the required resource value, the higher the adaptation degree is; of course, in the practical application process, there may be a situation that a certain value is very high and other values do not meet the requirement, so it is also necessary to check whether each value is greater than or equal to 1 before calculating, and then sum; that is, the subsequent summation operation can be performed only on the basis that the third quotient operation result, the fourth quotient operation result, and the fifth quotient operation result are all 1 or more. Of course, the purpose of limiting the execution of the subsequent summation operation to be performed only on the basis that the third quotient operation result, the fourth quotient operation result and the fifth quotient operation result are all greater than or equal to 1 is to enable the target task execution node to smoothly implement the specific execution of the task based on the own CPU resource, GPU resource and memory resource, so that the problem of task execution failure is avoided.

The method is characterized in that when node scheduling is performed, the area is determined preferentially according to the network adaptation degree, and then the area is selected preferentially according to the resource allocation adaptation degree; meanwhile, after each scheduling success, the scheduling result is stored as a scheduling template, when similar scheduling tasks are executed again, nodes of the scheduling template are directly used, and when the nodes are occupied, node matching is performed.

In step S140, a scheduling task is generated according to the first task configuration information and a target task execution node, and the scheduling task is executed in the target task execution node, so as to obtain a task execution result corresponding to the task to be executed.

In the present exemplary embodiment, first, a scheduling task is generated according to first task configuration information and a target task execution node; the scheduling task not only includes a Node identifier (node_id) of a target task execution Node, but also includes first task configuration information of a task to be executed, a task identifier, and a device identifier of a terminal device that sends the first task execution instruction, and of course, may also include other information, which is not particularly limited in this example; meanwhile, the target task execution node described herein may be a server having different functions, such as an interface test server, a model training server, a data request server, etc., and may be provided with a corresponding server according to actual needs, which is not particularly limited in this example.

And secondly, executing the scheduling task in the target task execution node to obtain a task execution result corresponding to the task to be executed. Specifically, the method can be realized by the following steps: firstly, issuing the scheduling task to the target task execution node, and submitting a container task mirror image included in the scheduling task to a task execution tool in a container orchestration engine through the target task execution node; and secondly, executing the task mirror image of the container through the task execution tool to obtain a task execution result corresponding to the task to be executed. The task executing tool executes the task mirror image of the container to obtain a task executing result corresponding to the task to be executed, and the task executing result can be realized in the following manner: firstly, executing the task mirror image of the container by the task execution tool to obtain a task execution container; and secondly, starting and executing the executable mirror image included in the task execution container to obtain a task execution result corresponding to the task to be executed. The executable mirror image recorded in the text can be obtained from the corresponding mirror image platform according to the mirror image name and the mirror image version included in the first task configuration information; meanwhile, in the actual application process, the acquired task mirror image of the container can be subjected to instantiation processing according to the mounting file included in the first task configuration information to obtain a task execution container, and then the executable mirror image included in the task execution container is started and executed to obtain a corresponding task execution result; the task execution results described herein may include model training results, data analysis results, interface test results, or the like, which are not particularly limited in this example.

In an example embodiment, whether the current task execution node is the current task execution node or the target task execution node is the target task execution node, a resident program can be set in each node, and then the resident program is started to monitor whether a scheduling task is issued in the target task execution node in a monitoring function mode, if the scheduling task is issued, a library file can be directly called to execute the scheduling task; in another possible example embodiment, the task may be performed in other manners, for example, a communication link between the target task execution node and the deployment scheduling module is established, and then after the deployment scheduling module issues the scheduling task to the target task execution node, a corresponding task execution instruction is issued to the target task execution node, so that the target task execution node executes the scheduling task based on the task execution instruction; in the actual application process, the method can be determined by itself according to actual needs, and the example is not particularly limited.

In an example embodiment, the task processing method may further include: in the process of executing mirror image running, monitoring a first running state of the task execution container and a second running state of a task execution tool in real time, and generating a monitoring log according to the first running state and the second running state; and/or displaying the first running state and the second running state. That is, in the process of performing task processing, it is also necessary to detect the first running state of the task execution container Pod and the second running state of the task execution tool Job in real time; the operation states described herein may include a use state of the first central processing unit, a use state of the first memory, a use state of the first graphics processor, an operation state of the container, and a monitoring log; meanwhile, the corresponding running state needs to be fed back to the task management module in real time, and the task running condition is displayed through the task management module; the use state of the first central processing unit, the use state of the first memory, and the use state of the first graphics processor may be specifically shown in fig. 9, and the specific monitoring log may be specifically shown in fig. 10. In the actual application process, the resource monitoring module can monitor the running states of Job, pod and the like and feed back the running states to the task management module 330 in real time, so that the task running conditions are displayed through the task management module 330; a specific illustration is shown with reference to fig. 10.

It should be noted that if any target task execution node is monitored to have abnormal operation, the abnormal operation information can be directly fed back to the corresponding terminal equipment, so that a user can dynamically adjust parameter information included in the first task configuration information through the terminal equipment, and further redeploy the nodes in the task; the specific operation flow may be shown with reference to fig. 11.

In an example embodiment, the task processing method may further include: deleting the executable mirror image in the task execution container when the executable mirror image execution is determined to be completed, and releasing second network occupied resources and second memory occupied resources of target task execution nodes occupied by executing the scheduling task; updating the node state of the target task executing node from the occupied state to the idle state, and placing the target task executing node in the idle state into a resource pool. That is, in the actual application process, after the task to be executed is executed, the corresponding node resource needs to be released, and the node is fed back to the resource pool; meanwhile, a user can check the resource release condition of the node in real time through the cluster state interface; a specific interface display diagram may be shown with reference to fig. 12. The resource recycling module 350 releases node resources used by task execution after receiving the task completion information issued by the task management module, and returns the node resources to the resource pool; a specific illustration of a specific resource release case may be shown in fig. 12.

In an example embodiment, the task processing method may further include: and calling a task callback interface corresponding to the terminal equipment, and feeding back the task execution result to the terminal equipment through the task callback interface. Specifically, in the actual application process, the user can be allowed to register the callback address, and further, the execution result of the task and the log information of the task execution are checked through the callback address, so that the task butt joint and the task integration of the task to be executed by external service are facilitated.

Up to this point, the task execution methods described in the exemplary embodiments of the present disclosure have been fully implemented. As can be seen from the foregoing, the task execution method according to the exemplary embodiments of the present disclosure has at least the following advantages: on the one hand, the focus can be transferred to the task execution itself; that is, in the practical application process, the user can be separated into several task groups without concern about the Kubernetes scheduling, and the task groups are distributed to the nodes; meanwhile, the premise of the use of the user is a simple configuration and perfect monitoring system, and the user can more simply configure and use the resource on the premise of optimally using the resource as much as possible; that is, the user only needs to provide the first task configuration information, so that the task can be automatically distributed and executed, and the purpose of improving the execution efficiency of the task is further achieved; on the other hand, an intelligent scheduling model; namely, the scheduling strategy can be customized according to the running requirement of the user, so that the running nodes are ensured to meet the task requirement and are optimal as much as possible; in yet another aspect, allowing a user to make a decision; that is, the user is allowed to decide when problems occur in the task issuing and task executing processes, whether the allocation mode of the resources is strict or relatively loose is adjusted, and when errors occur, the user is allowed to finely adjust the allocation strategy; furthermore, the modular access is realized without independent deployment; that is, the system can be used as independent service to provide kubernetes container management, and can also be used as a development component to be integrated into the development of other related business systems of containers; meanwhile, the life cycle of the whole kubernetes container can be maintained from the single task, and the use complexity is reduced.

The example embodiment of the disclosure also provides a task processing device. Specifically, referring to fig. 13, the task processing device may include a first receiving module 1310, a first parsing module 1320, a task execution node determining module 1330, and a task execution module 1340. Wherein:

the first receiving module 1310 may be configured to receive a first task execution instruction sent by the terminal device; the first task execution instruction is generated by the terminal equipment in response to an input event aiming at a task configuration interface;

the first parsing module 1320 may be configured to parse the first task execution instruction to obtain a task to be executed and first task configuration information corresponding to the task to be executed;

the task execution node determining module 1330 may be configured to match, according to the first task configuration information, a target task execution node corresponding to the task to be executed from current task execution nodes;

the task execution module 1340 may be configured to generate a scheduled task according to the first task configuration information and a target task execution node, and execute the scheduled task in the target task execution node to obtain a task execution result corresponding to the task to be executed.

In one exemplary embodiment of the present disclosure, the task execution node determination module 1330 may be further configured to: acquiring current task type information included in the first task configuration information, and matching and executing a first task execution node of a history executed task consistent with the current task type information from a task scheduling template; acquiring a current task execution node in an idle state from a resource pool, and judging whether the first task execution node exists in the current task execution node or not; if the first task execution node exists in the current task execution node, the first task execution node is used as a target task execution node corresponding to the task to be executed; and if the first task execution node does not exist in the current task execution node, matching the target task execution node corresponding to the task to be executed from the current task execution node according to the first task configuration information.

In one exemplary embodiment of the present disclosure, the task execution node determination module 1330 may be further configured to: acquiring current node configuration information of the current task execution node, and judging whether target node configuration information matched with the first task configuration information exists in the current node configuration information; if the current node configuration information contains target node configuration information matched with the first task configuration information, using the current task execution node corresponding to the target node configuration information as a target task execution node corresponding to the task to be executed.

In one exemplary embodiment of the present disclosure, the task execution node determination module 1330 may be further configured to: if the current node configuration information does not have the target node configuration information matched with the first task configuration information, generating first failure prompt information corresponding to a target task execution node, and sending the first failure prompt information to the terminal equipment; and receiving a second task execution instruction which is sent by the terminal equipment after the first task configuration information in the first task execution instruction is dynamically adjusted according to the first failure prompt information, and matching a target task execution node corresponding to the task to be executed from the current task execution node according to the second task configuration information included in the second task execution instruction.

In one exemplary embodiment of the present disclosure, the task execution node determination module 1330 may be further configured to: according to the first task configuration information and the current node configuration information, calculating the network configuration adaptation degree and/or the resource configuration adaptation degree between the task to be executed and the current task execution node; and matching the target task execution node corresponding to the task to be executed from the current task execution nodes according to the network configuration adaptation degree and/or the resource configuration adaptation degree.

In one exemplary embodiment of the present disclosure, the task execution node determination module 1330 may be further configured to: sorting the current task execution nodes according to the network configuration adaptation degree to obtain a first node sorting result, and selecting the current task execution node with the largest network adaptation degree from the first node sorting result as a target task execution node corresponding to the task to be executed; or sorting the current task execution nodes according to the resource allocation adaptation degree to obtain a second node sorting result, and selecting the current task execution node with the largest resource adaptation degree from the second node sorting result as a target task execution node corresponding to the task to be executed; or calculating the total adaptation degree between the task to be executed and the current task execution node according to the network configuration adaptation degree and the resource configuration adaptation degree, and sequencing the current task execution node according to the total adaptation degree to obtain a third node sequencing result; and selecting the current task execution node with the maximum resource adaptation degree from the third node sequencing result as the target task execution node corresponding to the task to be executed.

In one exemplary embodiment of the present disclosure, the task execution node determination module 1330 may be further configured to: configuring a first weight value and a second weight value for the network configuration adaptation degree and the resource configuration adaptation degree; wherein the first weight value is greater than or equal to the second weight value; calculating a first product operation result between the first weight value and the network configuration adaptation degree, and calculating a second product operation result between the second weight value and the resource configuration adaptation degree; and calculating a first sum operation result between a first product operation result and a second product operation result, and obtaining the total adaptation degree between the task to be executed and the current task execution node according to the first sum operation result.

In an exemplary embodiment of the present disclosure, the first task configuration information includes first network configuration information and/or first resource configuration information; the current node configuration information comprises second network configuration information and/or second resource configuration information; wherein the task execution node determination module 1330 may be further configured to: according to the first network configuration information and the second network configuration information, calculating the network configuration adaptation degree between the task to be executed and the current task execution node; and/or calculating the resource configuration adaptation degree between the task to be executed and the current task execution node according to the first resource configuration information and the second resource configuration information.

In an exemplary embodiment of the present disclosure, the first network configuration information includes a first network bandwidth and/or a first network delay, and the second network configuration information includes a second network bandwidth and/or a second network delay; wherein the task execution node determination module 1330 may be further configured to: calculating a first quotient operation result between the first network bandwidth and the second network bandwidth; and/or calculating a second quotient result between the first network delay and a second network delay; and obtaining the network configuration adaptation degree between the task to be executed and the current task execution node according to the first quotient operation result and/or the second quotient operation result.

In one exemplary embodiment of the disclosure, the first resource configuration information includes one or more of a resource configuration of the first central processor, a resource configuration of the first memory, and a resource configuration of the first graphics processor; the second resource configuration information comprises one or more of a resource configuration of a second central processing unit, a resource configuration of a second memory and a resource configuration of a second graphic processor; wherein the task execution node determination module 1330 may be further configured to: calculating a third quotient operation result between the resource allocation of the second central processing unit and the resource allocation of the first central processing unit; and/or calculating a fourth quotient result between the resource allocation of the second memory and the resource allocation of the first memory; and/or calculating a fifth quotient result between the resource allocation of the second graphics processor and the resource allocation of the first graphics processor; and obtaining the resource allocation adaptation degree between the task to be executed and the current task execution node according to the third quotient operation result and/or the fourth quotient operation result and/or the fifth quotient operation result.

In an exemplary embodiment of the present disclosure, the task execution module 1340 may also be configured to: issuing the scheduling task to the target task execution node, and submitting a container task mirror image included in the scheduling task to a task execution tool in a container orchestration engine through the target task execution node; and executing the container task mirror image through the task execution tool to obtain a task execution result corresponding to the task to be executed.

In an exemplary embodiment of the present disclosure, the task execution module 1340 may also be configured to: executing the task mirror image of the container through the task execution tool to obtain a task execution container; and starting and executing the executable mirror image included in the task execution container to obtain a task execution result corresponding to the task to be executed.

In an exemplary embodiment of the present disclosure, the task processing device further includes:

the running state monitoring module can be used for monitoring a first running state of the task execution container and a second running state of the task execution tool in real time in the running process of the executable mirror image, and generating a monitoring log according to the first running state and the second running state; and/or

And the running state display module can be used for displaying the first running state and the second running state.

the resource release module is configured to delete the executable image in the task execution container and release a second network occupied resource and a second memory occupied resource of a target task execution node occupied by executing the scheduled task when it is determined that the execution of the executable image is completed;

the node placement module is used for updating the node state of the target task execution node from the occupied state to the idle state and placing the target task execution node in the idle state into the resource pool.

and the task execution result feedback module can be used for calling a task callback interface corresponding to the terminal equipment and feeding back the task execution result to the terminal equipment through the task callback interface.

In an exemplary embodiment of the present disclosure, the first task execution instruction is generated by the terminal device in the following manner: displaying a task execution interface on a display interface of the terminal equipment; responding to a first input event aiming at a task execution interface, determining a task to be executed, and displaying a task configuration interface corresponding to the task to be executed; the tasks to be executed comprise model training tasks and/or data analysis tasks; determining first task configuration information of a task to be executed in response to a second input event acting on the task configuration interface; and responding to a third input event acting on the task configuration interface, generating a first task execution instruction according to the task to be executed and the first task configuration information, and sending the first task execution instruction to a task execution framework.

The specific details of each module in the task processing device are described in detail in the corresponding task processing method, so that the details are not repeated here.

It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.

Furthermore, although the steps of the methods in the present disclosure are depicted in a particular order in the drawings, this does not require or imply that the steps must be performed in that particular order or that all illustrated steps be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.

In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

Those skilled in the art will appreciate that the various aspects of the present disclosure may be implemented as a system, method, or program product. Accordingly, various aspects of the disclosure may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.

An electronic device 1400 according to such an embodiment of the present disclosure is described below with reference to fig. 14. The electronic device 1400 shown in fig. 14 is merely an example and should not be construed as limiting the functionality and scope of use of the disclosed embodiments.

As shown in fig. 14, the electronic device 1400 is embodied in the form of a general purpose computing device. Components of electronic device 1400 may include, but are not limited to: the at least one processing unit 1410, the at least one memory unit 1420, a bus 1430 connecting the different system components (including the memory unit 1420 and the processing unit 1410), and a display unit 1440.

Wherein the storage unit stores program code that is executable by the processing unit 1410 such that the processing unit 1410 performs steps according to various exemplary embodiments of the present disclosure described in the above section of the present description of exemplary methods. For example, the processing unit 1410 may perform step S110 as shown in fig. 1: receiving a first task execution instruction sent by terminal equipment; the first task execution instruction is generated by the terminal equipment in response to an input event aiming at a task configuration interface; step S120: analyzing the first task execution instruction to obtain a task to be executed and first task configuration information corresponding to the task to be executed; step S130: matching a target task execution node corresponding to the task to be executed from current task execution nodes according to the first task configuration information; step S140: generating a scheduling task according to the first task configuration information and a target task execution node, and executing the scheduling task in the target task execution node to obtain a task execution result corresponding to the task to be executed.

The memory unit 1420 may include readable media in the form of volatile memory units, such as Random Access Memory (RAM) 14201 and/or cache memory 14202, and may further include Read Only Memory (ROM) 14203.

The memory unit 1420 may also include a program/utility 14204 having a set (at least one) of program modules 14205, such program modules 14205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.

Bus 1430 may be a local bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or using any of a variety of bus architectures.

The electronic device 1400 may also communicate with one or more external devices 1500 (e.g., keyboard, pointing device, bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 1400, and/or any device (e.g., router, modem, etc.) that enables the electronic device 1400 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 1450. Also, electronic device 1400 may communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 1460. As shown, the network adapter 1460 communicates with other modules of the electronic device 1400 via the bus 1430. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with electronic device 1400, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.

From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (may be a personal computer, a server, a terminal device, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.

In an exemplary embodiment of the present disclosure, a computer-readable storage medium having stored thereon a program product capable of implementing the method described above in the present specification is also provided. In some possible implementations, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the disclosure as described in the "exemplary methods" section of this specification, when the program product is run on the terminal device.

A program product for implementing the above-described method according to an embodiment of the present disclosure may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The computer readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

Furthermore, the above-described figures are only schematic illustrations of processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. A method of task processing, comprising:

2. The task processing method according to claim 1, wherein matching a target task execution node corresponding to the task to be executed from current task execution nodes according to the first task configuration information includes:

3. The task processing method according to claim 1, wherein matching a target task execution node corresponding to the task to be executed from among the current task execution nodes according to the first task configuration information includes:

4. A task processing method according to claim 3, characterized in that the task processing method further comprises:

5. The task processing method according to claim 1, wherein matching a target task execution node corresponding to the task to be executed from among the current task execution nodes according to the first task configuration information includes:

6. The task processing method according to claim 5, wherein matching a target task execution node corresponding to the task to be executed from the current task execution nodes according to the network configuration adaptation degree and/or the resource configuration adaptation degree, comprises:

7. The task processing method according to claim 6, wherein calculating the total adaptation degree between the task to be executed and the current task execution node according to the network configuration adaptation degree and the resource configuration adaptation degree includes:

8. The task processing method according to claim 5, wherein the first task configuration information includes first network configuration information and/or first resource configuration information; the current node configuration information comprises second network configuration information and/or second resource configuration information;

9. The task processing method according to claim 8, wherein the first network configuration information includes a first network bandwidth and/or a first network delay, and the second network configuration information includes a second network bandwidth and/or a second network delay;

10. The task processing method according to claim 8, wherein the first resource configuration information includes one or more of a resource configuration of the first central processor, a resource configuration of the first memory, and a resource configuration of the first graphics processor;

11. The task processing method according to claim 1, wherein executing the scheduled task in the target task execution node to obtain a task execution result corresponding to the task to be executed, comprises:

12. The task processing method according to claim 11, wherein executing the container task image by the task execution tool obtains a task execution result corresponding to the task to be executed, comprising:

13. The task processing method according to claim 12, characterized in that the task processing method further comprises:

And displaying the first running state and the second running state.

14. The task processing method according to claim 12, characterized in that the task processing method further comprises:

15. The task processing method according to claim 1, characterized in that the task processing method further comprises:

16. The task processing method according to claim 1, wherein the first task execution instruction is generated by the terminal device by:

17. A task processing device, comprising:

18. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the task processing method of any one of claims 1-16.

19. An electronic device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the task processing method of any one of claims 1-16 via execution of the executable instructions.