CN118132232A - Method, apparatus, device, storage medium and program product for executing offline tasks - Google Patents

Abstract

The disclosure provides a method, a device, electronic equipment, a computer readable storage medium and a computer program product for executing offline tasks, and relates to the technical field of artificial intelligence such as data processing, intelligent cloud, cloud service and the like. One embodiment of the method comprises the following steps: based on the execution mode associated with each offline task to be processed, determining the execution priority corresponding to each offline task, wherein the execution mode comprises one of the following steps: synchronous triggering, timing triggering and asynchronous triggering; constructing a task execution sequence for each offline task based on the order indicated by the execution priority; each offline task is executed in turn based on the task execution sequence. The embodiment can assign the priority to the offline task based on the triggering mode of the offline task. Therefore, the offline tasks are satisfied and processed differently and strategically by using the management and execution modes, so that the processing quality and the processing efficiency of the offline tasks are improved.

Description

Method, apparatus, device, storage medium and program product for executing offline tasks

Technical Field

The present disclosure relates to the field of computer technology, and in particular, to the field of artificial intelligence technologies such as data processing, intelligent cloud, cloud service, and the like, and more particularly, to a method, an apparatus, an electronic device, a computer readable storage medium, and a computer program product for performing an offline task.

Background

Offline tasks, or offline task platforms, are typically computing platforms based on cloud computing or distributed computing technology for processing batches of offline data. It can automatically perform a series of offline data processing tasks such as data cleansing, data conversion, data analysis, machine learning model training, etc. An offline task platform typically includes three main components, data storage, data processing, and data output. On an offline task platform, a user may define and configure data processing tasks, then submit the tasks and wait for the tasks to complete. After the task is completed, the user may obtain the result data from the output catalog.

The offline task platform is widely applied in the fields of data processing and data analysis, and particularly in a big data scene, the offline task platform can greatly improve the efficiency and accuracy of data processing. Thus, how to more reasonably schedule and execute offline tasks to improve the efficiency and quality of offline task execution is of great concern and urgent need.

Disclosure of Invention

Embodiments of the present disclosure provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for performing offline tasks.

In a first aspect, an embodiment of the present disclosure proposes a method for performing an offline task, including: based on the execution mode associated with each offline task to be processed, determining the execution priority corresponding to each offline task, wherein the execution mode comprises one of the following steps: synchronous triggering, timing triggering and asynchronous triggering; constructing a task execution sequence for each offline task based on the order indicated by the execution priority; each offline task is executed in turn based on the task execution sequence.

In a second aspect, an embodiment of the present disclosure proposes an apparatus for performing an offline task, including: the execution priority determining unit is configured to determine the execution priority corresponding to each offline task based on the execution mode associated with each offline task to be processed, wherein the execution mode comprises one of the following steps: synchronous triggering, timing triggering and asynchronous triggering; an execution sequence construction unit configured to construct a task execution sequence for each offline task based on the order indicated by the execution priority; and the task execution unit is configured to sequentially execute each offline task based on the task execution sequence.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of performing an offline task as described in any of the implementations of the first aspect when executed.

In a fourth aspect, embodiments of the present disclosure provide a non-transitory computer-readable storage medium storing computer instructions for enabling a computer to implement a method of performing offline tasks as described in any of the implementations of the first aspect when executed.

In a fifth aspect, embodiments of the present disclosure provide a computer program product comprising a computer program which, when executed by a processor, is capable of implementing a method of performing an offline task as described in any of the implementations of the first aspect.

The embodiment of the disclosure provides a method, a device, an electronic device, a computer readable storage medium and a computer program product for executing offline tasks, wherein under the condition that a group of offline tasks to be processed exist, execution priorities corresponding to the offline tasks are respectively determined based on execution modes associated with the offline tasks to be processed. The execution mode comprises one of the following steps: synchronous triggering, timing triggering and asynchronous triggering; then, constructing a task execution sequence for each offline task based on the order indicated by the execution priority; and finally, sequentially executing each offline task based on the task execution sequence.

The method and the device can assign the priority to the offline task based on the triggering mode of the offline task. Therefore, the offline tasks are satisfied and processed differently and strategically by using the management and execution modes, so that the processing quality and the processing efficiency of the offline tasks are improved.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

Other features, objects and advantages of the present disclosure will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings:

FIG. 1 is an exemplary system architecture in which the present disclosure may be applied;

FIG. 2 is a flow chart of a method of performing offline tasks provided by embodiments of the present disclosure;

FIG. 3 is a flow chart of another method of performing offline tasks provided by embodiments of the present disclosure;

FIG. 4 is a flowchart of a method for performing offline tasks in an application scenario provided by an embodiment of the present disclosure;

FIG. 5 is a block diagram of an apparatus for performing offline tasks according to an embodiment of the present disclosure;

Fig. 6 is a schematic structural diagram of an electronic device adapted to perform a method for performing an offline task according to an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness. It should be noted that, without conflict, the embodiments of the present disclosure and features of the embodiments may be combined with each other.

In addition, in the technical scheme related to the disclosure, the processes of acquiring, storing, using, processing, transporting, providing, disclosing and the like of related user personal information (such as offline tasks possibly uploaded by a user related to later in the disclosure) all conform to the regulations of related laws and regulations and do not violate the well-known and good custom.

FIG. 1 illustrates an exemplary system architecture 100 in which embodiments of the methods, apparatus, electronic devices, and computer-readable storage media of performing offline tasks of the present disclosure may be applied.

As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various applications for implementing information communication between the terminal devices 101, 102, 103 and the server 105, such as an offline task execution application, a cloud service application, an instant messaging application, and the like, may be installed on the terminal devices.

The terminal devices 101, 102, 103 and the server 105 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices with display screens, including but not limited to smartphones, tablets, laptop and desktop computers, etc.; when the terminal devices 101, 102, 103 are software, they may be installed in the above-listed electronic devices, which may be implemented as a plurality of software or software modules, or may be implemented as a single software or software module, which is not particularly limited herein. When the server 105 is hardware, it may be implemented as a distributed server cluster formed by a plurality of servers, or may be implemented as a single server; when the server is software, the server may be implemented as a plurality of software or software modules, or may be implemented as a single software or software module, which is not particularly limited herein.

The server 105 may provide various services through various built-in applications, and for example, an offline task execution class application that may provide offline task execution, the server 105 may achieve the following effects when running the offline task execution class application: first, an offline task to be executed, or an offline task configured by the terminal device 101, 102, 103, is acquired from the terminal device 101, 102, 103 through the network 104. Then, the server 105 determines, based on the execution mode associated with each offline task to be processed, the execution priority corresponding to each offline task, where the execution mode includes one of the following: synchronous triggering, timing triggering and asynchronous triggering; then, the server 105 constructs a task execution sequence for each offline task based on the order indicated by the execution priority; finally, the server 105 sequentially executes each offline task based on the task execution sequence.

It should be noted that the offline tasks to be processed may be stored in advance in the server 105 in various ways, in addition to being acquired from the terminal devices 101, 102, 103 via the network 104. Thus, when the server 105 detects that such data has been stored locally (e.g., an offline task to be processed that remains prior to starting processing), it may choose to obtain such data directly from the local, in which case the exemplary system architecture 100 may also exclude the terminal devices 101, 102, 103 and the network 104.

Since the offline task needs to occupy more computing resources and stronger computing power, the method for executing the offline task provided in the subsequent embodiments of the present disclosure is generally performed by the server 105 having stronger computing power and more computing resources, and accordingly, the device for executing the offline task is also generally disposed in the server 105. However, it should be noted that, when the terminal devices 101, 102, 103 also have the required computing capability and computing resources, the terminal devices 101, 102, 103 may also complete each operation performed by the server 105 through the offline task execution class application installed thereon, and further output the same result as the server 105. Particularly, in the case where a plurality of terminal devices having different computing capabilities exist at the same time, when the offline task execution class application determines that the terminal device has a relatively strong computing capability and relatively more computing resources remain, the terminal device may be allowed to execute the above-mentioned computing, so that the computing pressure of the server 105 is properly reduced, and accordingly, the device for executing the offline task may be provided in the terminal devices 101, 102, 103. In this case, the exemplary system architecture 100 may also not include the server 105 and the network 104.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Referring to fig. 2, fig. 2 is a flowchart of a method for performing an offline task according to an embodiment of the disclosure, wherein the flowchart 200 includes the following steps:

Step 201: based on the execution mode associated with each offline task to be processed, determining the execution priority corresponding to each offline task;

This step is intended to be performed by an executing body (e.g., server 105 shown in fig. 1) of a method of executing offline tasks, after having acquired and being configured to process the offline tasks to be processed, the executing body may acquire the execution manner associated with the (respective) offline tasks to be processed. For example, the execution body may acquire description information of the offline task specified by the user side by publishing the online platform, and then the execution body packages the corresponding code (or determines a script code directory to be packaged) based on the description information to determine the offline task. The execution mode can comprise one of synchronous triggering (needing to wait for returning the result), timing triggering and asynchronous triggering (not waiting for the script result to be directly returned). For example, the execution subject may sort the offline tasks by way of offline task registration before the offline tasks are online.

Specifically, the offline task corresponding to the synchronous triggering mode may also be referred to as a synchronous offline task or a non-time-consuming offline task, and in such a triggering mode, after the offline task is triggered, the whole processing procedure of the offline task is sequentially executed, and after all the procedures are executed, the result is returned. The synchronously triggered offline task can be used for intervening in-line data in real time or operating material data warehouse entry, and has the characteristics of waiting for a return result to ensure successful operation and possibly continuous operation for a plurality of times. Typically, this type of task may have the highest priority to keep it from being reduced to the 0 instance. The off-line task triggered by timing can be applied to a full-volume library establishment scene, starts to run at a designated time, belongs to a predictable task type, and only needs to exist at the designated time under some special extreme conditions. An asynchronously triggered offline task may be applied where the run time is long, there is no need to obtain the returned results in time, such a task is insensitive to the validation time, e.g., it may be delayed by one minute for deployment instance re-trigger task execution, but because the asynchronous task is triggered manually, the trigger time is unpredictable, its corresponding priority may be lowest. Thus, the execution subject can configure and determine the execution priority corresponding to each offline task based on the different execution modes. In some embodiments, the synchronous trigger may have a highest first priority, the subsequent sequential timing triggers may be configured to have a second priority, and the asynchronous trigger may be configured to have a lowest third priority of the three. Therefore, the execution efficiency and the running stability of the offline task are improved by using the important synchronous trigger which is more severe to the execution condition to be satisfied preferentially.

It should be appreciated that in some scenarios, an offline task may be correspondingly configured with some execution conditions, e.g., in a timed trigger mode, an offline task may be associated with a corresponding timing (timer) to determine whether the trigger condition is met. In this case, if the trigger condition of the offline task is not satisfied, it may not be configured with priority corresponding to the execution subject. That is, the execution subject may choose to configure execution priorities for offline tasks that are only, preferably, satisfying the conditions, that are already in a state that can be triggered.

Alternatively or additionally, in addition to the execution mode, the execution subject may further sub-divide the tasks configured with the same priority based on the execution mode based on the description information associated with the offline task to further determine the execution order of the subsequent tasks at the same priority. For example, the execution subject may determine a reference score (e.g., a weighted value of scores corresponding to each task) corresponding to each task based on an uploading time of an offline task, a resource consumed for execution, a number of execution failures, etc., and then determine a specific execution order of each task in the same execution manner (or, a task of the same priority) based on a level of the reference score.

It should be noted that the offline tasks to be processed may be obtained directly from the local storage device by the execution body, or may be obtained from a non-local storage device (for example, the terminal devices 101, 102, 103 shown in fig. 1). The local storage device may be a data storage module, such as a server hard disk, provided in the execution body, in which case the offline task to be processed may be read quickly locally; the non-local storage device may also be any other electronic device arranged to store data, such as a user terminal or the like, in which case the executing entity may obtain the required offline task to be processed by responding to a task upload command sent by the electronic device.

Alternatively or additionally, the online platform may be a user-oriented platform that is responsible for the construction, version, release of functions, and management of some function meta-information, while supporting viewing of function runtime logs and execution histories. The user can provide offline tasks through the online platform and view basic information such as the operation history, the operation time length and the like of the offline tasks through the online platform. The user can log in the offline task instance to check the real-time log checking problem, so that the user authority management can be realized through the online platform, and the task instance management isolation is realized by limiting the mode that the user logs in the authorized task instance.

Step 202: constructing a task execution sequence for each offline task based on the order indicated by the execution priority;

On the basis of step 201, this step aims at constructing, by the above-mentioned execution subject, a task execution sequence for each (to-be-processed) offline task based on the order indicated by the execution priorities corresponding to these offline tasks. In the task execution sequence, the respective offline tasks may be ordered in an order indicated by the execution priority, i.e., the execution body may subsequently execute the respective offline tasks sequentially, based on the execution priority (ordering) of the respective offline tasks indicated in the task execution sequence.

Step 203: each offline task is executed in turn based on the task execution sequence.

On the basis of step 202, this step aims at sequentially and orderly executing each offline task by the execution subject based on the task execution sequence determined in step 202.

The method for executing the offline tasks provided by the embodiment of the disclosure can allocate the priorities to the offline tasks based on the triggering mode of the offline tasks. Therefore, the offline tasks are satisfied and processed differently and strategically by using the management and execution modes, so that the processing quality and the processing efficiency of the offline tasks are improved.

In some embodiments, the executing body may also estimate the flow required for the next offline task to be performed. Correspondingly, the execution body can respond under the condition that the network traffic usage value of the execution body exceeds the first traffic threshold value when the execution body predicts and detects the execution of the first offline task, and acquire a second offline task which is positioned behind the first offline task in sequence of executing the tasks. The network traffic usage value required when the second offline task is executed is lower than the first offline task and the network traffic usage value of the local machine when the second offline task is executed is less than or equal to the first traffic threshold. Accordingly, the execution subject may choose to interchange execution order of the second offline task and the first offline task. Therefore, by adjusting the off-line task execution order in the task execution sequence, the later task with smaller network traffic usage value is more preferentially executed at the current time than the current off-line task with larger current network traffic usage value, so that the network traffic usage value of the current using local machine is controlled not to exceed the first traffic threshold value.

In general, the first flow threshold may be determined based on a flow usage upper limit value that allows the execution subject, or that the execution subject configures for the task execution sequence. For example, when the upper limit value of the flow rate usage configured by the execution body for the task execution sequence is X, the first flow rate threshold value is X. In some embodiments, the upper traffic usage limit may be associated with a bandwidth associated with a hardware device that processes tasks in the task execution sequence. For example, in a case where the execution body is embodied as a server cluster, the traffic usage upper limit value may be a bandwidth corresponding to the physical machine of the corresponding instance. Therefore, the execution main body can detect the traffic use condition, and the stable execution of the offline task is ensured.

In some alternative implementations of the present embodiment, the first traffic threshold is determined based on the local upper limit of network traffic and network traffic usage values associated with other task execution sequences. In particular, in some scenarios an executing subject may be associated with multiple task execution sequences. For example, in the case where the offline task execution computing power is provided by the physical machine cluster, the execution subject may configure corresponding task execution sequences for a plurality of physical machines (subset group physical machines) respectively so that these physical machines may be more fully used. In this case, to ensure communication capability and stability, the execution subject may configure corresponding "upper limit of traffic usage value" for the physical machine and the subset group physical machine, respectively. For example, for some more computationally powerful physical machines, subset group physical machines, the execution body may configure it with execution task sequences that are "more difficult" to execute, and correspondingly configure it with more "upper traffic usage value limits" for it to process "offline tasks in the more difficult execution task sequences. Thus, by means of differentiated configuration, the execution subject can distinguish the offline tasks with higher and lower execution difficulty and process the offline tasks respectively by different task execution sequences, so that the problem that the offline tasks with lower difficulty can wait longer to be executed and met because the offline tasks with higher and lower difficulty are mixed into the same task execution sequence is avoided.

In some embodiments, in addition to adjusting the order of execution of the offline tasks, the executing entity may respond to determining that the network traffic usage value of the local machine exceeds the second traffic threshold upon detecting execution of the third offline task, and then determine a downgraded processing result of the third offline task using a task processing result lookup table based on a pre-configuration. In some embodiments, a task processing result comparison table may be generated by using historical processing results and manual configuration of operation and maintenance personnel, and the task result comparison table may record "degradation processing results" corresponding to offline tasks. The "demotion process result" may be considered as a "generic result" of processing a corresponding offline task, or a class of offline tasks, or a "minimum standard result" that meets the use, task processing requirements. Accordingly, the execution subject may utilize the demotion processing result of the third offline task as a processing result of the third offline task, instead of executing the third offline task. Thus, when the local network traffic usage value required by the offline task exceeds the second traffic threshold, the execution subject can replace the offline task to generate a result by a mode of 'degrading the result'. The method can control the use of the flow when the required flow exceeds the standard, and can provide the result of executing the offline task for the subsequent and downstream, thereby supporting and intervening the abnormal flow access and ensuring the normal operation of the downstream task instance.

Similarly, the execution body may also estimate the local network traffic usage value required by each offline task after determining the offline task to be processed. Accordingly, the execution body may determine a demotion processing result of the fourth offline task based on the pre-configured task processing result comparison table in response to detecting that the local network traffic usage value required to perform the fourth offline task exceeds the third traffic threshold. That is, in the case where the task processing result comparison table is configured, the execution subject may adopt a demotion policy for the "out-of-standard" offline task after estimating the local network traffic usage value required for the offline task, or, for the "out-of-standard" fourth offline task, directly select the demotion result of the fourth offline task as the processing result of the fourth offline task, so as to replace executing the fourth offline task. The offline task is interfered earlier, and on the premise that abnormal flow access can be supported and interfered and limited, and the normal operation of a downstream task instance can be ensured, the operation resources are saved more.

It should be appreciated that for both the second flow threshold and the third flow threshold described above, the flow threshold may be determined in a manner similar to the first flow threshold and will not be repeated here. In some embodiments, the second flow threshold and the third flow threshold may be configured in advance, and are considered as network flow usage values corresponding to "out-of-standard" and "oversized" offline tasks. Thus, by the configuration mode, the operation staff and the management staff can adjust the processing strategy of the execution main body for the offline task by configuring the second flow threshold and the third flow threshold. For example, the execution subject can adopt a mode of 'downgrade' processing to the offline tasks according to the 'overscale' and 'oversized' offline tasks configured by operation staff and management staff to promote the execution efficiency and save the operation resources.

In some embodiments, resource usage value thresholds may also be configured for offline tasks from the computing resource usage (value) dimension based on a manner similar to the network flow value usage. Therefore, the execution subject adjusts the execution order of the offline task in a relation mode that the resource usage value corresponding to the offline task is compared with the resource usage value threshold value, and determines whether to adopt degradation processing for the offline task.

In some embodiments, some anomalies may occur in the offline task during the process of being executed, for example, the offline task may lack a certain environmental control, or the offline task may not be executed successfully due to an error in code encoding in the offline task, etc. In these cases, the execution subject may send the association information of the abnormal offline task to the target user (e.g., the operation and maintenance person) based on a pre-configured communication address (e.g., a communication address of the supervisor set by the operation and maintenance person) in response to the abnormal offline task existing during the task execution sequence. In some optional implementations of this embodiment, the association information includes at least one of: task running log of abnormal offline task, and execution duration of abnormal offline task. Thus, the execution subject can provide the relevant information of the abnormal offline task generating the execution abnormality to the user for prompting and providing reference. Therefore, the user can timely understand and acquire the execution state of the offline task, and analyze the abnormality based on the associated information serving as a reference and auxiliary when the abnormality exists, so that the decision can be made more efficiently and qualitatively.

In some embodiments, in order to improve the execution capability of the offline task, a cloud native cluster containerized deployment manner may be utilized to implement dynamic expansion and contraction of resources, so as to improve the execution capability of the offline task and disaster recovery capability of an execution subject. For example, a Kubernetes cluster may be utilized to implement a containerized deployment. Referring specifically to fig. 3, fig. 3 is a flowchart of another method for performing an offline task according to an embodiment of the disclosure, where the flowchart 300 includes the following steps:

Step 301: based on the execution mode associated with each offline task to be processed, determining the execution priority corresponding to each offline task;

step 302: constructing a task execution sequence for each offline task based on the order indicated by the execution priority;

The above steps 301 to 302 are identical to the steps 201 to 202 shown in fig. 2, and the same parts are referred to the corresponding parts of the previous embodiment, and will not be described herein again.

Step 303: based on the task execution sequence, sequentially generating respective corresponding execution mirror images of each offline task;

Specifically, the execution body may sequentially package the code corresponding to each offline task and the configuration description (e.g., trigger mode) and other information according to the order of the task execution sequence, so as to generate a corresponding execution mirror image.

Step 304: sequentially deploying each execution image into the Kubernetes cluster to generate an execution instance corresponding to each execution image;

Specifically, the execution body may deploy the execution images obtained in the step 303 into each container in the Kubernetes cluster, and generate corresponding execution instances in each container.

Step 305: each execution instance is executed in turn to execute each offline task in turn.

In particular, the execution body may execute respective (execution) instances in respective containers in the Kubernetes cluster to execute respective offline tasks in the task execution sequence. The execution instance may execute corresponding script code logic therein to actually complete the offline task.

Therefore, the Kubernetes cluster can simultaneously replace one high-allocation physical device with a plurality of low-allocation machines, and the effective utilization of resources is realized through the dynamic expansion capability of the cloud native (for example, through dynamically and elastically adjusting the configuration quantity of execution examples).

Similar to the discussion above, there may be multiple Kubernetes clusters for performing different task execution sequences. In some embodiments, the execution body may also scale down or up the Kubernetes cluster based on the execution of the instance. For example, in the event that the traffic usage value loaded in the Kubernetes cluster does not reach a threshold (e.g., the first traffic threshold described above), the executing entity may configure more containers, instances in the Kubernetes cluster so that the Kubernetes cluster may handle more offline tasks. Similarly, in the case where the traffic usage value carried by the Kubernetes cluster exceeds a threshold, the executing body may reduce the size of the Kubernetes cluster by reducing the number of containers in the Kubernetes cluster to reduce its traffic usage value. Therefore, the elastic expansion and contraction of the examples are carried out through the information such as the flow, the load and the like of the task examples in the cluster, so that the utilization rate and the stability of the Kubernetes cluster are balanced.

In some embodiments, to enhance the management capability for the Kubernetes cluster, an access OpenFaaS platform may also be selected to control the Kubernetes cluster. For example, step 305 described above may choose to utilize OpenFaaS platforms to execute each execution instance in turn to execute each offline task in turn. Because OpenFaaS and cloud native technology stacks such as Kubernetes are matched, in the process of the second iteration OpenFaaS, code grammar and thought do not need to be switched, and the method has lower transformation cost. Further, considering that the task is triggered by events (http request, timer, faaS client operation) to realize the timing operation of the offline task through a connector, the real-time triggering of the task can be realized by the way of the http request, and simultaneously, the method supports the input parameters to the offline task, so that the method has high flexibility. The join OpenFaaS retains the property of task instances, manually triggering the time without waiting for task instance creation. When the manual triggering is performed, the OpenFaaS platform can route to a specific task instance according to the flow characteristics to trigger the task to be executed. Tasks are triggered by events (http requests, timers, faaS client operations), openFaaS platform can route to specific task instances according to the characteristics of traffic, triggering task execution. For example, the Of-watchdog associated with OpenFaaS platform may encapsulate a request parameter as stdin to execute a cmd instruction to trigger a task to run, waiting for the task execution to complete fetching the data return result from stdout. OpenFaaS the components mainly utilized are promethus, alarm (ALERTMANAGER). When the cloud function is used, the corresponding connector is firstly arranged in the cluster, and then the cloud function can be triggered according to time by adding the content appointed by the connector to the annotations attribute in yaml for packaging the cloud function. Therefore, by utilizing the combination of the OpenFaaS platform and the Kubernetes cluster, the management and operation capacity of the Kubernetes cluster can be improved, and the processing efficiency of the Kubernetes cluster is further improved.

For further understanding, the present disclosure also provides a specific implementation in connection with a specific application scenario, please refer to the flowchart 400 shown in fig. 4.

An executing body (e.g., server 105 in fig. 1, described above) may provide an online platform 410 to collect and acquire offline tasks. For example, in flow 400, an executing subject may obtain a set of offline tasks 420 through an online platform 410.

Further, the execution body may continue the execution manner to determine the execution priority of each offline task. For example, for the offline task 421, the executing body may determine that it has the highest first priority because its execution mode is synchronous triggering. For offline task 422, the executing body may determine that it has the lowest third priority because its execution is asynchronously triggered. For offline task 423, the executing body may determine that it has a second priority between the first priority and the third priority because its execution is a timed trigger.

Further, the execution body constructs a task execution sequence 430, and in the task execution sequence 430, the execution order of the offline tasks may be an offline task 421, an offline task 423, and an offline task 422 in sequence.

Next, the execution body may sequentially generate execution images corresponding to the respective offline tasks based on the order of the task execution sequence 430. For example, the execution body may sequentially generate an execution image 421' of the offline task 421, an execution image 423' of the offline task 423, and an execution image 422' of the offline task 422.

Next, the execution body may provide the respective execution images to Kubernetes container 440 to generate corresponding execution instances. For ease of understanding, the Kubernetes container 440 may provide processing power for two execution examples, for example. Thus, the execution body may provide an execution instance 441 in the Kubernetes container 440 to correspond to the execution image 421 'and an execution instance 442 to correspond to the execution image 423'. Thus, the execution body may actually perform the offline task 421 and the offline task 423 by executing the execution instance 441 and the execution instance 442. And after completion of execution of one of the offline tasks, the execution body may continue to populate the execution image 422' to execute the offline task 422.

Further, the executing body may communicate the offline task 450, and its corresponding association information, to the target user (or, in other words, the target user device, such as the terminal device 101) based on the pre-configured communication address, upon determining that the offline task exists.

In addition, the execution body may also perform step S401 to continuously or periodically monitor the state and condition of the flow usage value. For example, in practice, a gateway may be configured in the execution body to monitor traffic. Accordingly, the executing body adjusts the quantity in Kubernetes container 440 upon determining that the flow usage value is not as expected (e.g., differs from the first flow threshold by more than an allowable threshold). For example, the execution body may choose to provide more instances into the Kubernetes container 440 so that it can handle more offline tasks.

Accordingly, in the case that the executing body determines that the flow usage value exceeds the predetermined first flow threshold, the executing body may select to execute S403 to adjust the task order in the task execution sequence 430, so as to achieve the purpose of "throttling". Similarly, for the offline tasks in the Kubernetes container 440 and the task execution sequence 430, the execution entity may also select whether to execute S404 by determining a current local traffic usage value and/or a desired traffic usage value to be used when the offline tasks are executed (e.g., the local network traffic usage value for executing the offline tasks exceeds a second traffic threshold and the local network traffic usage value for executing the offline tasks exceeds a third traffic threshold), so as to implement a downgrade process for the offline tasks associated with exceeding the second traffic threshold and the third traffic threshold, for the purpose of "throttling". For example, when executing the mirror 422', i.e. when executing the offline task 422, the local network traffic usage value exceeds the second traffic threshold, the executing body may select to determine the processing result corresponding to the offline task 422 by using the pre-configured task processing result comparison table, so as to replace executing the mirror 422' and executing the offline task 422.

With further reference to fig. 5, as an implementation of the method shown in the foregoing figures, the present disclosure provides an embodiment of an apparatus for performing offline tasks, where the apparatus embodiment corresponds to the method embodiment shown in fig. 2, and the apparatus may be specifically applied in various electronic devices.

As shown in fig. 5, the apparatus 500 for performing an offline task of the present embodiment may include: an execution priority determining unit 501, an execution sequence constructing unit 502, and a task executing unit 503. Wherein, the execution priority determining unit 501 is configured to determine, based on an execution manner associated with each offline task to be processed, an execution priority corresponding to each offline task, where the execution manner includes one of the following: synchronous triggering, timing triggering and asynchronous triggering; an execution sequence construction unit 502 configured to construct a task execution sequence for each offline task based on the order indicated by the execution priorities; the task execution unit 503 is configured to sequentially execute each offline task based on the task execution sequence.

In this embodiment, in the apparatus 500 for performing an offline task: the specific processes and technical effects of the execution priority determining unit 501, the execution sequence constructing unit 502, and the task executing unit 503 may refer to the relevant descriptions of steps 201 to 203 in the corresponding embodiment of fig. 2, and are not repeated herein.

In some optional implementations of this embodiment, the apparatus 500 further includes: the post-task obtaining unit is configured to obtain a second offline task which is positioned behind the first offline task in the sequence of task execution in order in response to detection that the network traffic usage value of the local machine exceeds a first traffic threshold when the first offline task is executed, wherein the network traffic usage value required for executing the second offline task is lower than the first offline task and the network traffic usage value of the local machine is smaller than or equal to the first traffic threshold when the second offline task is executed; and the task order interchange unit is configured to interchange the execution order of the second offline task and the first offline task.

In some alternative implementations of the present embodiment, the first traffic threshold is determined based on the local upper limit of network traffic and network traffic usage values associated with other task execution sequences.

In some optional implementations of this embodiment, the apparatus 500 further includes: a first demotion processing unit configured to determine a demotion processing result of the third offline task based on a pre-configured task processing result comparison table in response to detecting that the network traffic usage value of the local machine exceeds the second traffic threshold when the third offline task is executed; the first result replacing unit is configured to replace the third offline task by using the degradation processing result of the third offline task as the processing result of the third offline task.

In some optional implementations of this embodiment, the apparatus 500 further includes: a second demotion processing unit configured to determine a demotion processing result of the fourth offline task based on a pre-configured task processing result comparison table in response to detecting that a network traffic usage value of the local machine required to perform the fourth offline task exceeds a third traffic threshold; and a second result replacing unit configured to replace the execution of the fourth offline task by using the demotion processing result of the fourth offline task as a processing result of the fourth offline task.

In some optional implementations of this embodiment, the apparatus 500 further includes: an abnormal task prompting unit configured to send, based on a pre-configured communication address, association information of an abnormal offline task to a target user in response to the existence of the abnormal offline task in the task execution sequence, wherein the association information includes at least one of: task running log of abnormal offline task, and execution duration of abnormal offline task.

In some optional implementations of the present embodiment, the task execution unit 503 includes: the mirror image generation subunit is configured to sequentially generate execution mirror images corresponding to the offline tasks based on the task execution sequences; an instance generation subunit configured to sequentially deploy each execution image into the Kubernetes cluster to generate an execution instance corresponding to each execution image; and the task execution subunit is configured to sequentially execute each execution instance to sequentially execute each offline task.

In some optional implementations of this embodiment, the task execution subunit is further configured to sequentially execute each execution instance using the OpenFaaS platform to sequentially execute each offline task.

The present embodiment exists as an embodiment of a device corresponding to the foregoing method embodiment, and the device for executing an offline task provided in the present embodiment can assign a priority to an offline task based on a triggering manner of the offline task. Therefore, the offline tasks are satisfied and processed differently and strategically by using the management and execution modes, so that the processing quality and the processing efficiency of the offline tasks are improved.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

Fig. 6 illustrates a schematic block diagram of an example electronic device 600 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 6, the apparatus 600 includes a computing unit 601 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the device 600 may also be stored. The computing unit 601, ROM602, and RAM603 are connected to each other by a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Various components in the device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, mouse, etc.; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as methods of performing offline tasks. For example, in some embodiments, the method of performing offline tasks may be implemented as a computer software program tangibly embodied on a machine readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM 602 and/or the communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the method of performing offline tasks described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the method of performing offline tasks in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of large management difficulty and weak service expansibility in the traditional physical host and Virtual Private Server (VPS) PRIVATE SERVER service. Servers may also be divided into servers of a distributed system or servers that incorporate blockchains.

According to the technical scheme of the embodiment of the disclosure, the priority can be allocated to the offline task based on the triggering mode of the offline task. Therefore, the offline tasks are satisfied and processed differently and strategically by using the management and execution modes, so that the processing quality and the processing efficiency of the offline tasks are improved.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions provided by the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims (19)

1. A method of performing an offline task, comprising:

based on the execution mode associated with each offline task to be processed, determining the execution priority corresponding to each offline task, wherein the execution mode comprises one of the following steps: synchronous triggering, timing triggering and asynchronous triggering;

Constructing a task execution sequence for each of the offline tasks based on the order indicated by the execution priorities;

And sequentially executing each off-line task based on the task execution sequence.

2. The method of claim 1, further comprising:

In response to detecting that the network traffic usage value of the local machine exceeds a first traffic threshold when the first offline task is executed, acquiring a second offline task which is positioned behind the first offline task in the sequence of executing the tasks, wherein the network traffic usage value required for executing the second offline task is lower than the first offline task and the network traffic usage value of the local machine is lower than or equal to the first traffic threshold when the second offline task is executed;

and exchanging the execution order of the second offline task and the first offline task.

3. The method of claim 2, wherein the first traffic threshold is determined based on a local upper bound of network traffic and network traffic usage values associated with other task execution sequences.

4. The method of claim 1, further comprising:

in response to detecting that the network traffic usage value of the local machine exceeds a second traffic threshold when executing a third offline task, determining a degradation processing result of the third offline task based on a pre-configured task processing result comparison table;

And using the degradation processing result of the third offline task as the processing result of the third offline task to replace the execution of the third offline task.

5. The method of claim 1, further comprising:

in response to detecting that a local network traffic usage value required to perform a fourth offline task exceeds a third traffic threshold, determining a degradation processing result of the fourth offline task based on a pre-configured task processing result comparison table;

and using the degradation processing result of the fourth offline task as the processing result of the fourth offline task to replace the fourth offline task.

6. The method of claim 1, further comprising:

In response to the abnormal offline task existing in the task execution sequence process, based on a pre-configured communication address, sending association information of the abnormal offline task to a target user, wherein the association information comprises at least one of the following: the task running log of the abnormal offline task and the execution time of the abnormal offline task.

7. The method of claim 1, wherein the sequentially performing each of the offline tasks based on the task execution sequence comprises:

sequentially generating execution images corresponding to the offline tasks based on the task execution sequences;

Sequentially deploying the execution images into a Kubernetes cluster to generate execution examples corresponding to the execution images;

and sequentially executing each execution instance to sequentially execute each off-line task.

8. The method of claim 7, wherein the sequentially executing each of the execution instances to sequentially execute each of the offline tasks comprises:

And sequentially executing each execution instance by using a OpenFaaS platform so as to sequentially execute each offline task.

9. An apparatus for performing an offline task, comprising:

An execution priority determining unit, configured to determine, based on an execution manner associated with each offline task to be processed, an execution priority corresponding to each offline task, where the execution manner includes one of the following: synchronous triggering, timing triggering and asynchronous triggering;

an execution sequence construction unit configured to construct a task execution sequence for each of the offline tasks based on the order indicated by the execution priorities;

And the task execution unit is configured to sequentially execute each off-line task based on the task execution sequence.

10. The apparatus of claim 9, further comprising:

a post-task obtaining unit configured to obtain, in response to detecting that a network traffic usage value of a local machine exceeds a first traffic threshold value when a first offline task is executed, a second offline task that is located downstream of the first offline task in the task execution sequence, wherein the network traffic usage value required for executing the second offline task is lower than the first offline task and the network traffic usage value of the local machine is lower than or equal to the first traffic threshold value when the second offline task is executed;

And the task order interchange unit is configured to interchange the execution order of the second offline task and the first offline task.

11. The apparatus of claim 10, wherein the first traffic threshold is determined based on a local upper bound for network traffic and network traffic usage values associated with other task execution sequences.

12. The apparatus of claim 9, further comprising:

a first demotion processing unit configured to determine, in response to detection of execution of a third offline task, a demotion processing result of the third offline task based on a pre-configured task processing result comparison table, in which a network traffic usage value of the local exceeds a second traffic threshold;

And the first result replacing unit is configured to replace the third offline task by using the degradation processing result of the third offline task as the processing result of the third offline task.

13. The apparatus of claim 9, further comprising:

a second demotion processing unit configured to determine a demotion processing result of a fourth offline task based on a pre-configured task processing result comparison table in response to detecting that a network traffic usage value of a local machine required to perform the fourth offline task exceeds a third traffic threshold;

and a second result replacing unit configured to replace execution of the fourth offline task by using a demotion processing result of the fourth offline task as a processing result of the fourth offline task.

14. The apparatus of claim 9, further comprising:

An abnormal task prompting unit configured to send, in response to the presence of an abnormal offline task in the task execution sequence based on the task execution sequence, association information of the abnormal offline task to a target user based on a pre-configured communication address, wherein the association information includes at least one of: the task running log of the abnormal offline task and the execution time of the abnormal offline task.

15. The apparatus of claim 9, wherein the task performing unit comprises:

the mirror image generation subunit is configured to sequentially generate execution mirror images corresponding to the offline tasks based on the task execution sequences;

An instance generation subunit, configured to sequentially deploy each execution image to a Kubernetes cluster, so as to generate an execution instance corresponding to each execution image;

and the task execution subunit is configured to sequentially execute each execution instance so as to sequentially execute each offline task.

16. The apparatus of claim 15, wherein the task execution subunit is further configured to sequentially execute each of the execution instances using a OpenFaaS platform to sequentially execute each of the offline tasks.

17. An electronic device, comprising:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of performing offline tasks of any of claims 1-8.

18. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of performing an offline task of any of claims 1-8.

19. A computer program product comprising a computer program which, when executed by a processor, implements the method of performing offline tasks according to any of claims 1-8.