CN117762590A - Execution method, system and device of computing task and related equipment - Google Patents

Abstract

The disclosure provides a method, a system, a device and related equipment for executing a computing task, and relates to the technical field of big data processing, wherein the method comprises the following steps: task information of a computing task to be executed is obtained, wherein the task information at least comprises: a task type; distributing the calculation tasks to be executed to calculation connectors of corresponding task types according to the task information of the calculation tasks to be executed, wherein the calculation connectors are used for submitting the calculation tasks to be executed to calculation clusters of the corresponding task types; and receiving a task execution result returned by the computing cluster, wherein the task execution result is a result of the computing cluster executing the computing task to be executed. The method and the device can solve the problems of complex docking adaptation and large workload of various computing engines in the related art, dynamically generate the computing engines required by the user according to the condition of resources, effectively reduce the use threshold of the user and improve the computing efficiency of the application.

Description

Execution method, system and device of computing task and related equipment

Technical Field

The disclosure relates to the technical field of big data processing, and in particular relates to a method, a system, a device and related equipment for executing a computing task.

Background

In the application of big data and AI, there are plentiful computing engines, such as Spark, flink, tensorFlow, according to statistics, the number of computing engines in common use is tens and hundreds, and different computing engines solve the problems of different scenes, but when a user calls each computing engine, the user needs to know the environment condition of the engine and learn and verify the task submitting command of the engine, so that higher labor cost is brought to the user, and in this case, a widely applicable method and device are urgently needed to mask the middle tedious steps and work for the user.

It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Disclosure of Invention

The present disclosure provides a method, a system, an apparatus, and a related device for executing a computing task, which at least overcome to some extent the problem of low efficiency when different computing engines are invoked in the related art.

Other features and advantages of the present disclosure will be apparent from the following detailed description, or may be learned in part by the practice of the disclosure.

According to one aspect of the present disclosure, there is provided a method of performing a computing task, including: task information of a computing task to be executed is obtained, wherein the task information at least comprises: a task type; distributing the to-be-executed computing task to a computing connector of a corresponding task type according to the task information of the to-be-executed computing task, wherein the computing connector is used for submitting the to-be-executed computing task to a computing cluster of the corresponding task type; and receiving a task execution result returned by the computing cluster, wherein the task execution result is a result of the computing cluster executing the computing task to be executed.

In some exemplary embodiments of the present disclosure, based on the foregoing solution, before assigning the to-be-performed computing task to the computing connector of the corresponding task type according to the task information of the to-be-performed computing task, the method further includes: judging whether a computing connector of a corresponding task type exists or not; if not, creating a computing connector of the corresponding task type; if yes, multiplexing the existing computing connectors of the corresponding task types.

In some exemplary embodiments of the present disclosure, based on the foregoing solution, after assigning the to-be-performed computing task to a computing connector of a corresponding task type according to the task information of the to-be-performed computing task, the method further includes: judging whether a computing cluster for executing the corresponding task type exists or not; if not, creating a computing cluster capable of executing the corresponding task type by pulling up; if so, multiplexing the existing computing clusters capable of executing the corresponding task types.

In some exemplary embodiments of the present disclosure, based on the foregoing solution, before creating the pull-up computing cluster capable of performing the respective task type, the method further comprises: evaluating the feasibility of the computing clusters which can be pulled up to execute the corresponding task types, and if the computing clusters are feasible, pulling up the clusters; if feasible, pulling up a computing cluster capable of executing the corresponding task type; and if not, receiving an error result returned by the computing cluster.

In some exemplary embodiments of the present disclosure, based on the foregoing solution, according to task information of the to-be-executed computing task, assigning the to-be-executed computing task to a computing connector of a corresponding task type includes: grouping the computing tasks to be executed according to the task types of the computing tasks to be executed; and distributing the grouped computing tasks to be executed to the computing connectors of the corresponding task types.

In some exemplary embodiments of the disclosure, based on the foregoing solution, before receiving the task execution result returned by the computing cluster, the method further includes: and the computing connectors of the corresponding task types compute the computing tasks to be executed to obtain the task execution results.

In some exemplary embodiments of the present disclosure, based on the foregoing solution, before acquiring the task information of the computing task to be performed, the method further includes: persistence is carried out on the acquired execution computing task, and analysis is carried out on the execution computing task; and obtaining the task information of the computing task to be executed.

According to another aspect of the present disclosure, there is also provided a system for performing a computing task, including: the application gateway is used for acquiring task information of a computing task to be executed, wherein the task information at least comprises: a task type; the computing scheduler is used for distributing the computing tasks to be executed to computing connectors of corresponding task types according to the task information of the computing tasks to be executed, wherein the computing connectors are used for submitting the computing tasks to be executed to computing clusters of the corresponding task types; and the computing cluster is used for executing the computing task to be executed and returning the task execution result to the application gateway.

According to another aspect of the present disclosure, there is also provided an apparatus for performing a computing task, including: the task information acquisition module is used for acquiring task information of a computing task to be executed, wherein the task information at least comprises: a task type; the task information distribution module is used for distributing the computing tasks to be executed to the computing connectors of the corresponding task types according to the task information of the computing tasks to be executed, wherein the computing connectors are used for submitting the computing tasks to be executed to the computing clusters of the corresponding task types; the task result receiving module is used for receiving a task execution result returned by the computing cluster, wherein the task execution result is a result of the computing cluster executing the computing task to be executed.

According to still another aspect of the present disclosure, there is also provided an electronic apparatus including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the method of performing any one of the above-described computing tasks via execution of the executable instructions.

According to yet another aspect of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of performing any one of the above-described computing tasks.

According to the method, the system, the device and the related equipment for executing the computing task, task information including task types in the computing task to be executed is obtained, then the computing task information is adapted and distributed according to the task types of the computing task to be executed, and is submitted to a computing cluster of a corresponding task type for computing.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort.

FIG. 1 illustrates an exemplary application system architecture diagram of a method of performing a computing task in an embodiment of the present disclosure;

FIG. 2 illustrates a schematic diagram of a method of performing a computing task in an embodiment of the present disclosure;

FIG. 3 illustrates a schematic diagram of a system for performing a computing task in an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a task submission stage in a method for executing a computing task according to an embodiment of the disclosure;

FIG. 5 is a schematic diagram illustrating a task scheduling stage in a method for executing a computing task according to an embodiment of the disclosure;

FIG. 6 is a schematic diagram illustrating a task execution phase in a method for executing a computing task according to an embodiment of the disclosure;

Fig. 7 is a schematic diagram of a result pushing stage in a method for executing a computing task according to an embodiment of the disclosure;

FIG. 8 illustrates a schematic diagram of a method of performing four stages of a computing task in an embodiment of the disclosure;

FIG. 9 illustrates a schematic diagram of a computing task execution device in an embodiment of the present disclosure;

fig. 10 is a schematic diagram of an electronic device for executing a computing task according to an embodiment of the disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the disclosed aspects may be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.

The flow diagrams depicted in the figures are exemplary only, and do not necessarily include all of the elements and operations/steps, nor must they be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the order of actual execution may be changed according to actual situations.

FIG. 1 illustrates an exemplary application system architecture diagram to which a method of performing computing tasks in embodiments of the present disclosure may be applied. As shown in fig. 1, the system architecture may include a terminal device 101, a network 102, and a server 103.

The medium used by the network 102 to provide a communication link between the terminal device 101 and the server 103 may be a wired network or a wireless network.

Alternatively, the wireless network or wired network described above uses standard communication techniques and/or protocols. The network is typically the Internet, but may be any network including, but not limited to, a local area network (Local Area Network, LAN), metropolitan area network (Metropolitan Area Network, MAN), wide area network (Wide Area Network, WAN), mobile, wired or wireless network, private network, or any combination of virtual private networks. In some embodiments, data exchanged over a network is represented using techniques and/or formats including HyperText Mark-up Language (HTML), extensible markup Language (Extensible MarkupLanguage, XML), and the like. All or some of the links may also be encrypted using conventional encryption techniques such as secure sockets layer (Secure Socket Layer, SSL), transport layer security (Transport Layer Security, TLS), virtual private network (Virtual Private Network, VPN), internet protocol security (Internet ProtocolSecurity, IPsec), etc. In other embodiments, custom and/or dedicated data communication techniques may also be used in place of or in addition to the data communication techniques described above.

The terminal device 101 may be a variety of electronic devices including, but not limited to, smart phones, tablet computers, laptop portable computers, desktop computers, wearable devices, augmented reality devices, virtual reality devices, and the like.

Alternatively, the clients of the applications installed in different terminal devices 101 are the same or clients of the same type of application based on different operating systems. The specific form of the application client may also be different based on the different terminal platforms, for example, the application client may be a mobile phone client, a PC client, etc.

The server 103 may be a server providing various services, such as a background management server providing support for devices operated by the user with the terminal apparatus 101. The background management server can analyze and process the received data such as the request and the like, and feed back the processing result to the terminal equipment.

Optionally, the server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content delivery networks), basic cloud computing services such as big data and artificial intelligence platforms, and the like. The terminal may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited herein.

Those skilled in the art will appreciate that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative, and that any number of terminal devices, networks, and servers may be provided as desired. The embodiments of the present disclosure are not limited in this regard.

Under the system architecture described above, embodiments of the present disclosure provide a method for executing a computing task, where the method may be executed by any electronic device with computing processing capabilities.

In some embodiments, the method for executing the computing task provided in the embodiments of the present disclosure may be executed by the terminal device of the system architecture described above; in other embodiments, the method for performing the computing task provided in the embodiments of the present disclosure may be performed by a server in the system architecture described above; in other embodiments, the method for executing the computing task provided in the embodiments of the present disclosure may be implemented by the terminal device and the server in the system architecture in an interactive manner.

Firstly, the embodiment of the disclosure provides an execution method which can be applied to but not limited to a computing task, and can also be an execution method of a storage task, an execution method of a communication task and the like, if the task to be executed is storage, the task to be executed is distributed to a storage connector of a corresponding task type according to the task type of the task to be executed, and finally, the task to be executed is processed through a cluster of the corresponding task type to obtain a task execution result.

Fig. 2 is a schematic diagram of a method for executing a computing task in an embodiment of the disclosure, as shown in fig. 2, where the method for executing a computing task provided in the embodiment of the disclosure includes the following steps:

s202, task information of a computing task to be executed is obtained, wherein the task information at least comprises: task type.

It should be noted that, in the embodiment of the present disclosure, the computing task to be executed may be any computing task to be executed, may be a computing task provided from a user interface program, for example, python (computer programming language), scala (a multi-paradigm programming language), java (an object oriented programming language), SQL (Structured Query Language, structured query language database), etc., and the task information of the computing task to be executed may include task code, an execution engine, parameters, etc., and in more detail, the task type in the embodiment of the present disclosure may be a computing engine that needs to be used for the computing task to be executed.

S204, distributing the calculation tasks to be executed to the calculation connectors of the corresponding task types according to the task information of the calculation tasks to be executed, wherein the calculation connectors are used for submitting the calculation tasks to be executed to the calculation clusters of the corresponding task types.

It should be noted that, in the embodiment of the present disclosure, the computing connector is used to connect to the underlying computing engine, where the computing connector has multiple types, and functions of the computing connector support plug-in extension, including supporting the computing capability of interactive computing, streaming computing, offline computing, and machine learning, while clusters are distributed in resource pool units in different physical environments, and the clusters provide computing, storage, transmission, and other infrastructures and operation platforms for services.

S206, receiving a task execution result returned by the computing cluster, wherein the task execution result is a result of the computing cluster executing the computing task to be executed.

In more detail, in the embodiment of the present disclosure, after submitting a to-be-executed computing task to a computing cluster of a corresponding task type, the computing cluster performs computing processing on the to-be-executed computing task to obtain a task execution result.

The execution method of the computing task provided in the embodiment of the present disclosure includes first obtaining task information including a task type in a computing task to be executed; then, according to the task information of the to-be-executed computing task, the to-be-executed computing task is distributed to a computing connector of a corresponding task type, and the computing connector submits the to-be-executed computing task to a computing cluster of the corresponding task type; and finally, performing calculation processing on the calculation task to be executed through the corresponding calculation cluster to obtain a task execution result.

Compared with the prior art that higher labor cost is caused by invoking each calculation engine, the embodiment of the disclosure dynamically selects the calculation clusters needed by the user according to the task type of the calculation task to be executed, thereby effectively reducing the use threshold of the user, improving the calculation efficiency, and bringing considerable benefits to the cost reduction and efficiency enhancement of enterprises by wide application.

In some embodiments, before assigning the computing task to be performed to the computing connector of the corresponding task type according to the task information of the computing task to be performed, the method further comprises: judging whether a computing connector of a corresponding task type exists or not; if not, creating a computing connector of the corresponding task type; if yes, multiplexing the existing computing connectors of the corresponding task types. According to the method and the device, the computing connectors of the corresponding task types are determined according to the task information, the corresponding computing connectors are used for connecting the corresponding computing clusters, and the operation of calling the computing engine is completed.

In some embodiments, after assigning the computing task to be performed to the computing connector of the corresponding task type according to the task information of the computing task to be performed, the method further comprises: judging whether a computing cluster for executing the corresponding task type exists or not; if not, creating a computing cluster capable of executing the corresponding task type by pulling up; if so, multiplexing the existing computing clusters capable of executing the corresponding task types. According to the task information of the to-be-executed computing task, the to-be-executed computing task is adapted and distributed, and under the condition that the computing clusters of the corresponding task types exist, the computing clusters are transferred to the corresponding computing clusters to initiate a computing process, so that the problems of complex butt joint adaptation and large workload of various computing engines are solved, the computing efficiency is improved, further, if the computing clusters of the corresponding task types do not exist, the computing clusters which can be executed by the corresponding task types are created and pulled up, the computing engines needed by users are dynamically generated, the use threshold of the users is effectively reduced, and the wide application of the embodiment of the disclosure brings considerable benefits to the cost reduction and efficiency enhancement of the users.

In more detail, in the embodiment of the present disclosure, if there is no computing cluster corresponding to the task type of the computing task to be executed, the computing resource required by the user may be automatically pulled up based on the cloud native technology, the computing task may be executed, and the computing result may be returned, so that the scheme adaptability of the executing method of the computing task in the embodiment of the present disclosure is wider.

In some embodiments, prior to creating the computing clusters that pull up to be able to perform the respective task types, the method further comprises: evaluating the feasibility of the computing clusters which can be pulled up to execute the corresponding task types, and if the computing clusters are feasible, pulling up the clusters; if feasible, pulling up a computing cluster capable of executing the corresponding task type; if not, receiving an error result returned by the computing cluster. By evaluating the feasibility of pulling up the computing clusters capable of executing the corresponding task types, if the computing clusters are feasible, the embodiments of the disclosure automatically pull up computing resources required by users and execute the computing tasks and return task execution results based on the cloud native technology, if the computing clusters are not feasible, the computing clusters cannot perform correct computation on the computing tasks to be executed, and the obtained possible error results are returned to the users so as to ensure the accuracy of the task execution results.

In some embodiments, assigning the computing tasks to be performed to computing connectors of the respective task types according to task information of the computing tasks to be performed includes: grouping the computing tasks to be executed according to the task types of the computing tasks to be executed; and distributing the grouped computing tasks to be executed to the computing connectors of the corresponding task types. By grouping task types of the computing tasks to be executed, each computing task to be executed with different task types does not affect scheduling, and if multiple computing tasks to be executed with different task types are obtained, different computing engines can be scheduled in parallel according to the different task types, so that computing efficiency is improved.

In some embodiments, before receiving the task execution result returned by the computing cluster, the method further includes: the computing connectors of the corresponding task types compute the computing tasks to be executed to obtain task execution results. And calculating the to-be-executed calculation tasks of different task types by using different calculation clusters to obtain corresponding task execution results under different to-be-executed calculation tasks, and returning the obtained task execution results to the user, so that the user can more clearly determine the task execution results of the to-be-executed calculation tasks.

In some embodiments, prior to obtaining task information for a computing task to be performed, the method further comprises: persistence is carried out on the acquired execution computing task, and analysis is carried out on the execution computing task; and obtaining task information of the computing task to be executed. According to the embodiment of the disclosure, the acquired execution computing task is subjected to persistence, and the task information of the computing task to be executed is better saved.

Based on the same inventive concept, the embodiments of the present disclosure also provide a system for executing a computing task, such as the following embodiments. Since the principle of solving the problem of the system embodiment is similar to that of the method embodiment, the implementation of the system embodiment can be referred to the implementation of the method embodiment, and the repetition is omitted.

FIG. 3 illustrates a schematic diagram of a system for performing a computing task in an embodiment of the disclosure, as shown in FIG. 3, the system comprising:

the application gateway is used for acquiring task information of a computing task to be executed, wherein the task information at least comprises: a task type;

the computing scheduler is used for distributing the computing tasks to be executed to computing connectors of corresponding task types according to the task information of the computing tasks to be executed, wherein the computing connectors are used for submitting the computing tasks to be executed to computing clusters of the corresponding task types;

and the computing cluster is used for executing the computing task to be executed and returning the task execution result to the application gateway.

In some embodiments, as shown in fig. 3, the computing scheduler in the execution system of the computing task in the embodiment of the present disclosure further includes: the system comprises a computing connector management module, an application queue management module and a uniform resource management module, wherein the computing connector management module is mainly responsible for selecting a computing connector for executing a task based on different task types, forwarding the task, and managing the whole life cycle of creating, running, deleting and the like of the computing connector; the application queue management module is in charge of managing all queues submitting tasks, and the computing scheduler acquires computing tasks to be executed from the application queues to schedule; the unified resource management module is responsible for resource management in the layer, namely the resource consumption and allocation condition in the system, and mainly provides resource basis for the creation of the computing connector.

In more detail, the application gateway in the embodiment of the disclosure is used as a submitting entrance of a task type of a computing task to be executed, provides functions of receiving and forwarding the task, and is responsible for state tracking and result returning of task execution; the computing scheduler is the brain of the system, manages the adaptive execution of multi-type tasks, and is responsible for the management of task queues, the distribution of computing resources, the management of computing connectors and the like.

In some embodiments, the execution system of a computing task in an embodiment of the present disclosure performs the computing task including: a task submitting stage, a task scheduling stage, a task executing stage and a result pushing stage.

In more detail, the task submitting stage supports various task submitting, including the task to be executed submitted by the user interface program in the modes of Python, scale, java, SQL, etc., and the task information mainly includes task types, task codes, execution engines, parameters, etc., as shown in fig. 4, and specifically includes the following steps:

and S402, submitting the computing task to be executed to an application gateway by the user interface program.

S404, after receiving the calculation task to be executed, the application gateway carries out persistence on task information, analyzes task parameters and codes in advance, and gives the calculation task to be executed to an application queue management module in a calculation scheduler.

S406, the computing scheduler schedules the computing tasks to be executed, and groups the computing tasks to be executed according to task types of the computing tasks to be executed, wherein the computing tasks to be executed with different task types do not affect scheduling.

In some embodiments, as shown in fig. 5, the task scheduling stage in the embodiments of the present disclosure makes a decision, selects a computing task to be executed, constructs a computing connector, and matches the two, and specifically includes the following steps:

s502, a computing dispatcher selects a computing task to be executed, a computing connector for executing the computing task to be executed is obtained through a request computing connector management module, if the computing connector is idle, multiplexing is carried out, and if the computing connector is idle, a new computing connector is created;

and S504, submitting the to-be-executed computing task to a corresponding computing connector by the computing dispatcher, and ending the task dispatching stage.

In some embodiments, as shown in fig. 6, the task execution stage in the embodiments of the present disclosure refers to that the computing connectors link different underlying computing engines, and execute computation on a computing task to be executed, and specifically includes the following steps:

s602, binding a computing cluster, wherein the computing connector in the step judges whether the computing cluster of a computing type required by a user exists, if the computing cluster exists, a computing task is directly sent, if the computing cluster does not exist, the computing cluster is quickly pulled up based on a cloud native technology, the computing cluster is automatically pulled up based on current environment resources and the cluster to be feasible before the computing cluster is pulled up, if the computing cluster is feasible, the computing cluster is pulled up, and otherwise, an error result is returned;

S604, the computing connector is linked to a specific computing engine to submit a task for execution, the real-time state of task execution is returned to an application gateway through a log framework, and the real-time state is returned to a user through the gateway;

in some embodiments, as shown in fig. 7, the result pushing stage in the embodiments of the present disclosure returns a task execution result to the user, and specifically includes the following steps:

s702, writing a task execution result into a file system, acquiring a corresponding path by a computing connector in the process, and returning the path to an application gateway;

s704, the application gateway updates the result and the path information into the task list, and the user reads the result list through the user side program.

In some embodiments, as shown in fig. 8, the method for executing the computing task in the embodiment of the disclosure specifically includes the following steps:

s801, a user submits a task request.

S802, the application gateway receives the calculation task to be executed, persists task information and adds the calculation task to be executed into a queue.

S803, the scheduler schedules the calculation tasks to be executed.

S804, the dispatcher selects a calculation task to be executed, acquires a calculation connector for executing the calculation task to be executed by requesting a calculation connector management module, multiplexes if there is a corresponding idle, and creates a new calculation connector if there is no corresponding idle.

S805, the scheduler submits the to-be-executed computing task to the corresponding computing connector, and the task scheduling phase is ended.

S806, binding the computing clusters, wherein the computing connector can judge whether the computing clusters of the computing type required by the user exist or not, if the computing clusters exist, the computing tasks to be executed are directly sent, if the computing clusters do not exist, the computing clusters can be quickly pulled up based on the cloud native technology, before the computing clusters are pulled up, the computing clusters are automatically pulled up based on the current environment resources and the computing clusters, if the computing clusters are feasible, the computing clusters are pulled up, and otherwise, an error result is returned.

S807, the computing connector links to the specific computing engine to submit tasks for execution, and returns the real-time status of task execution to the application gateway via the log framework, and to the user via the application gateway.

S808, the task execution result is written into the file system, and the computing connector acquires a corresponding path in the process and returns the path to the application gateway.

S809, the application gateway updates the path information of the result set into the task list, and the user reads the result list through the user side program.

Based on the same inventive concept, the embodiments of the present disclosure also provide a device for executing a computing task, as follows. Since the principle of solving the problem of the embodiment of the device is similar to that of the embodiment of the method, the implementation of the embodiment of the device can be referred to the implementation of the embodiment of the method, and the repetition is omitted.

FIG. 9 is a schematic diagram of an apparatus for performing a computing task according to an embodiment of the disclosure, where the apparatus includes:

the task information obtaining module 901 is configured to obtain task information of a computing task to be executed, where the task information at least includes: a task type;

the task information distribution module 902 is configured to distribute a to-be-executed computing task to a computing connector of a corresponding task type according to task information of the to-be-executed computing task, where the computing connector is configured to submit the to-be-executed computing task to a computing cluster of the corresponding task type;

the task result receiving module 903 is configured to receive a task execution result returned by the computing cluster, where the task execution result is a result of the computing cluster executing the computing task to be executed.

The execution device of the computing task provided by the embodiment of the disclosure acquires task information comprising task types in the computing task to be executed through a task information acquisition module; distributing the to-be-executed computing task to a computing connector of a corresponding task type according to task information of the to-be-executed computing task through a task information distribution module, and submitting the to-be-executed computing task to a computing cluster of the corresponding task type through the computing connector; and performing calculation processing on the calculation tasks to be executed through the corresponding calculation clusters by a task information distribution module to obtain task execution results.

Compared with the prior art that higher labor cost is caused by invoking each calculation engine, the embodiment of the disclosure dynamically selects the calculation clusters needed by the user according to the task type of the calculation task to be executed, thereby effectively reducing the use threshold of the user, improving the calculation efficiency, and bringing considerable benefits to the cost reduction and efficiency enhancement of enterprises by wide application.

In some embodiments, the apparatus for performing a computing task in an embodiment of the disclosure further includes: the computing connector judging module is used for judging whether the computing connector of the corresponding task type exists or not before the computing task to be executed is distributed to the computing connector of the corresponding task type according to the task information of the computing task to be executed; the computing connector creation module is used for creating a computing connector of a corresponding task type if not; and the calculation connector multiplexing module is used for multiplexing the existing calculation connectors of the corresponding task types if yes.

In some embodiments, the apparatus for performing a computing task in an embodiment of the disclosure further includes: the computing cluster judging module is used for judging whether a computing cluster for executing the corresponding task type exists or not after the computing task to be executed is distributed to the computing connector of the corresponding task type according to the task information of the computing task to be executed; the computing cluster creation module is used for creating a computing cluster which can be pulled up to execute the corresponding task type if not; and the computing cluster multiplexing module is used for multiplexing the existing computing clusters capable of executing the corresponding task types if yes.

In some embodiments, the apparatus for performing a computing task in an embodiment of the disclosure further includes: the evaluation module is used for evaluating the feasibility of the computing cluster capable of executing the corresponding task type by pulling before the computing cluster capable of executing the corresponding task type by pulling is created, and pulling the cluster if the feasibility is available; the computing cluster pulling module is used for pulling up the computing clusters capable of executing the corresponding task types if the computing clusters are feasible; and the error result returning module is used for receiving the error result returned by the computing cluster if the error result is not feasible.

In some embodiments, the task information distribution module in the embodiments of the present disclosure is further configured to group the computing tasks to be executed according to task types of the computing tasks to be executed; and distributing the grouped computing tasks to be executed to the computing connectors of the corresponding task types.

In some embodiments, the apparatus for performing a computing task in an embodiment of the disclosure further includes: the task execution result determining module is used for calculating the calculation task to be executed by the calculation connector of the corresponding task type before receiving the task execution result returned by the calculation cluster, so as to obtain a task execution result.

In some embodiments, the apparatus for performing a computing task in an embodiment of the disclosure further includes: the task information acquisition module is used for lasting the acquired execution computing task and analyzing the execution computing task before acquiring the task information of the computing task to be executed; and obtaining task information of the computing task to be executed.

Those skilled in the art will appreciate that the various aspects of the present disclosure may be implemented as a system, method, or program product. Accordingly, various aspects of the disclosure may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.

An electronic device 1000 according to such an embodiment of the present disclosure is described below with reference to fig. 10. The electronic device 1000 shown in fig. 10 is merely an example and should not be construed as limiting the functionality and scope of use of the disclosed embodiments.

As shown in fig. 10, the electronic device 1000 is embodied in the form of a general purpose computing device. Components of electronic device 1000 may include, but are not limited to: the at least one processing unit 1001, the at least one memory unit 1002, a bus 1003 connecting the different system components (including the memory unit 1002 and the processing unit 1001).

Wherein the storage unit stores program code that is executable by the processing unit 1001, such that the processing unit 1001 performs steps according to various exemplary embodiments of the present disclosure described in the above section of the "exemplary method" of the present specification.

In some embodiments, when the electronic device is used to control a method of performing a computing task such as described above in the present disclosure, the processing unit 1001 may perform the following steps of the method embodiments described above:

task information of a computing task to be executed is obtained, wherein the task information at least comprises: task type.

And distributing the computing tasks to be executed to the computing connectors of the corresponding task types according to the task information of the computing tasks to be executed, wherein the computing connectors are used for submitting the computing tasks to be executed to the computing clusters of the corresponding task types.

And receiving a task execution result returned by the computing cluster, wherein the task execution result is a result of the computing cluster executing the computing task to be executed.

The storage unit 1002 may include a readable medium in the form of a volatile storage unit, such as a Random Access Memory (RAM) 10021 and/or a cache memory 10022, and may further include a Read Only Memory (ROM) 10023.

The storage unit 1002 may also include a program/utility 10024 having a set (at least one) of program modules 10025, such program modules 10025 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.

Bus 1003 may be a bus representing one or more of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device 1000 may also be in communication with one or more external devices 1004 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 1000, and/or with any device (e.g., router, modem, etc.) that enables the electronic device 1000 to communicate with one or more other computing devices. Such communication may occur through an input/output (I/O) interface 1005. Also, electronic device 1000 can communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the Internet, through network adapter 1006. As shown, the network adapter 1006 communicates with other modules of the electronic device 1000 over the bus 1003. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with the electronic device 1000, including, but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and the like.

From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (may be a personal computer, a server, a terminal device, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.

In particular, according to embodiments of the present disclosure, the process described above with reference to the flowcharts may be implemented as a computer program product comprising: computer program which when executed by a processor implements the method of executing the above-mentioned computing tasks.

In an exemplary embodiment of the present disclosure, a computer-readable storage medium, which may be a readable signal medium or a readable storage medium, is also provided. On which a program product is stored which enables the implementation of the method described above of the present disclosure. In some possible implementations, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the disclosure as described in the "exemplary methods" section of this specification, when the program product is run on the terminal device.

More specific examples of the computer readable storage medium in the present disclosure may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

In this disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, with readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Alternatively, the program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

In particular implementations, the program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.

Furthermore, although the steps of the methods in the present disclosure are depicted in a particular order in the drawings, this does not require or imply that the steps must be performed in that particular order or that all illustrated steps be performed in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform, etc.

From the description of the above embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (may be a personal computer, a server, a mobile terminal, or a network device, etc.) to perform the method according to the embodiments of the present disclosure.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims (11)

1. A method of performing a computing task, comprising:

task information of a computing task to be executed is obtained, wherein the task information at least comprises: a task type;

distributing the to-be-executed computing task to a computing connector of a corresponding task type according to the task information of the to-be-executed computing task, wherein the computing connector is used for submitting the to-be-executed computing task to a computing cluster of the corresponding task type;

and receiving a task execution result returned by the computing cluster, wherein the task execution result is a result of the computing cluster executing the computing task to be executed.

2. The method of claim 1, wherein before assigning the computing task to be executed to a computing connector of a corresponding task type according to task information of the computing task to be executed, the method further comprises:

judging whether a computing connector of a corresponding task type exists or not;

if not, creating a computing connector of the corresponding task type;

if yes, multiplexing the existing computing connectors of the corresponding task types.

3. The method for executing a computing task according to claim 1, wherein after assigning the computing task to be executed to a computing connector of a corresponding task type according to task information of the computing task to be executed, the method further comprises:

Judging whether a computing cluster for executing the corresponding task type exists or not;

if not, creating a computing cluster capable of executing the corresponding task type by pulling up;

if so, multiplexing the existing computing clusters capable of executing the corresponding task types.

4. A method of performing a computing task according to claim 3, wherein prior to creating a computing cluster that is capable of being pulled up to perform the corresponding task type, the method further comprises:

evaluating the feasibility of the computing clusters which can be pulled up to execute the corresponding task types, and if the computing clusters are feasible, pulling up the clusters;

if feasible, pulling up a computing cluster capable of executing the corresponding task type;

and if not, receiving an error result returned by the computing cluster.

5. The method for executing a computing task according to claim 1, wherein assigning the computing task to be executed to a computing connector of a corresponding task type according to task information of the computing task to be executed comprises:

grouping the computing tasks to be executed according to the task types of the computing tasks to be executed;

and distributing the grouped computing tasks to be executed to the computing connectors of the corresponding task types.

6. The method of claim 1, wherein prior to receiving the task execution result returned by the computing cluster, the method further comprises:

and the computing connectors of the corresponding task types compute the computing tasks to be executed to obtain the task execution results.

7. The method for executing a computing task according to claim 1, wherein before acquiring task information of the computing task to be executed, the method further comprises:

persistence is carried out on the acquired execution computing task, and analysis is carried out on the execution computing task; and obtaining the task information of the computing task to be executed.

8. A system for performing a computing task, comprising:

the application gateway is used for acquiring task information of a computing task to be executed, wherein the task information at least comprises: a task type;

the computing scheduler is used for distributing the computing tasks to be executed to computing connectors of corresponding task types according to the task information of the computing tasks to be executed, wherein the computing connectors are used for submitting the computing tasks to be executed to computing clusters of the corresponding task types;

And the computing cluster is used for executing the computing task to be executed and returning the task execution result to the application gateway.

9. An apparatus for performing a computing task, comprising:

the task information acquisition module is used for acquiring task information of a computing task to be executed, wherein the task information at least comprises: a task type;

the task information distribution module is used for distributing the computing tasks to be executed to the computing connectors of the corresponding task types according to the task information of the computing tasks to be executed, wherein the computing connectors are used for submitting the computing tasks to be executed to the computing clusters of the corresponding task types;

the task result receiving module is used for receiving a task execution result returned by the computing cluster, wherein the task execution result is a result of the computing cluster executing the computing task to be executed.

10. An electronic device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the method of performing the computing task of any of claims 1-7 via execution of the executable instructions.

11. A computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements a method of performing a computing task according to any of claims 1 to 7.