CN116069497A

CN116069497A - Method, apparatus, device and storage medium for executing distributed task

Info

Publication number: CN116069497A
Application number: CN202211545713.7A
Authority: CN
Inventors: 徐文栋
Original assignee: Baidu com Times Technology Beijing Co Ltd
Current assignee: Baidu com Times Technology Beijing Co Ltd
Priority date: 2022-12-01
Filing date: 2022-12-01
Publication date: 2023-05-05

Abstract

The disclosure provides a method, a device, equipment and a storage medium for executing distributed tasks, and relates to the technical field of computers, in particular to the field of distributed computing. Detailed description of the invention the scheme is as follows: acquiring a distributed task, wherein the distributed task comprises a plurality of subtasks; and sequentially executing a plurality of subtasks to obtain an execution result for the distributed task, wherein the distributed task is generated according to the following operations: generating a plurality of subtasks according to operator information corresponding to the subtasks, wherein each subtask comprises a conditional operator and a processing operator; according to the dependency relationship among the plurality of subtasks, configuring the topological relationship among the plurality of subtasks; and creating a context space and configuring a subspace corresponding to each subtask in the context space.

Description

Method, apparatus, device and storage medium for executing distributed task

Technical Field

The present disclosure relates to the field of computer technology, and in particular, to the field of distributed computing.

Background

There are many tasks under a microservice architecture that need to be orchestrated, and these tasks may all have large differences in terms of execution flow, yield, and resource consumption bottleneck points (e.g., time consumption, number of listening threads), etc. In a distributed scenario, there is also a consistency problem when tasks are combined together by a series of operations. How to uniformly manage these tasks is an important issue of interest to the industry.

Disclosure of Invention

The present disclosure provides a method, apparatus, device, storage medium, and program product for performing distributed tasks.

According to an aspect of the present disclosure, there is provided a method of performing a distributed task, including: acquiring a distributed task, wherein the distributed task comprises a plurality of subtasks; and sequentially executing the plurality of subtasks to obtain an execution result for the distributed task, wherein the distributed task is generated according to the following operations: generating a plurality of subtasks according to operator information corresponding to the subtasks, wherein each subtask comprises a conditional operator and a processing operator; according to the dependency relationship among the plurality of subtasks, configuring the topological relationship among the plurality of subtasks; and creating a context space and configuring a subspace corresponding to each subtask in the context space.

According to another aspect of the present disclosure, there is provided an apparatus for performing a distributed task, including: the acquisition module is used for acquiring a distributed task, wherein the distributed task comprises a plurality of subtasks; the execution module is used for sequentially executing the plurality of subtasks to obtain an execution result aiming at the distributed task, and the generation module is used for generating a plurality of subtasks according to operator information corresponding to the plurality of subtasks, wherein each subtask comprises a conditional operator and a processing operator; the configuration module is used for configuring the topological relation among the plurality of subtasks according to the dependency relation among the plurality of subtasks; and the creation module is used for creating a context space and configuring a subspace corresponding to each subtask in the context space, wherein the distributed tasks are generated according to the generation module, the configuration module and the creation module.

Another aspect of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the methods shown in the embodiments of the present disclosure.

According to another aspect of the disclosed embodiments, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods shown in the disclosed embodiments.

According to another aspect of the disclosed embodiments, there is provided a computer program product comprising a computer program/instruction, characterized in that the computer program/instruction, when executed by a processor, implements the steps of the method shown in the disclosed embodiments.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 schematically illustrates an exemplary system architecture in which methods and apparatus for performing distributed tasks may be applied in accordance with embodiments of the present disclosure;

FIG. 2 schematically illustrates a flow chart of a method of performing distributed tasks according to an embodiment of the disclosure;

FIG. 3 schematically illustrates a flow chart of a method of generating distributed tasks according to an embodiment of the disclosure;

FIG. 4 schematically illustrates a schematic diagram of a topology according to an embodiment of the present disclosure;

FIG. 5 schematically illustrates a flow chart of a method of sequentially performing a plurality of subtasks in accordance with an embodiment of the present disclosure;

FIG. 6 schematically illustrates a flow chart of a method of performing distributed tasks according to an embodiment of the disclosure;

FIG. 7 is a block diagram of an apparatus for performing distributed tasks to implement an embodiment of the present disclosure;

FIG. 8 schematically illustrates a block diagram of an example electronic device that may be used to implement embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

The system architecture of the method and apparatus for performing distributed tasks provided by the present disclosure will be described below in conjunction with fig. 1.

FIG. 1 schematically illustrates an exemplary system architecture 100 in which methods and apparatus for performing distributed tasks may be employed in accordance with embodiments of the present disclosure. It should be noted that fig. 1 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios.

As shown in fig. 1, a system architecture 100 according to this embodiment may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the

terminal devices

101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 via the network 104 using the

terminal devices

101, 102, 103 to receive or send messages or the like. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the

terminal devices

101, 102, 103.

The

terminal devices

101, 102, 103 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by users using the

terminal devices

101, 102, 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.

According to embodiments of the present disclosure, server 105 may be a server of a distributed system. The server 105 may be equipped with a task orchestration engine for unified management of distributed tasks. The distributed tasks may include, for example, search tasks, data storage tasks, and the like.

It should be noted that the method for performing a distributed task provided by the embodiments of the present disclosure may be generally performed by the server 105. Accordingly, the devices for performing distributed tasks provided by embodiments of the present disclosure may be generally disposed in the server 105. The method of performing distributed tasks provided by the embodiments of the present disclosure may also be performed by a server or cluster of servers other than the server 105 and capable of communicating with the

terminal devices

101, 102, 103 and/or the server 105. Accordingly, the apparatus for performing distributed tasks provided by the embodiments of the present disclosure may also be provided in a server or a server cluster different from the server 105 and capable of communicating with the

terminal devices

101, 102, 103 and/or the server 105.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing, applying and the like of the personal information of the user all conform to the regulations of related laws and regulations, necessary security measures are adopted, and the public order harmony is not violated.

In the technical scheme of the disclosure, the authorization or consent of the user is obtained before the personal information of the user is obtained or acquired.

The method of performing distributed tasks provided by the present disclosure will be described below in connection with fig. 2.

Fig. 2 schematically illustrates a flow chart of a method of performing distributed tasks according to an embodiment of the disclosure. The execution distributed tasks may be performed, for example, by a task orchestration engine in a server.

As shown in fig. 2, the method 200 of executing a distributed task includes acquiring a distributed task in operation S210. Wherein the distributed task may comprise a plurality of subtasks.

Then, in operation S220, a plurality of sub-tasks are sequentially performed, resulting in an execution result for the distributed task.

The method of generating distributed tasks provided by the present disclosure will be described below in connection with fig. 3.

Fig. 3 schematically illustrates a flow chart of a method of generating distributed tasks according to an embodiment of the disclosure.

As shown in fig. 3, the method 300 of generating a distributed task includes generating a plurality of subtasks, each of which includes a conditional operator and a processing operator, according to operator information corresponding to the plurality of subtasks in operation S330.

In operation S340, a topological relationship between the plurality of subtasks is configured according to the dependency relationship corresponding to the plurality of subtasks.

In operation S350, a context space is created, and a subspace corresponding to each subtask in the context space is configured.

According to embodiments of the present disclosure, a distributed Task, i.e., task (Job), may include a plurality of subtasks (tasks). The plurality of subtasks may have a dependency relationship between them when executed. Each subtask may include multiple operators, such as a conditional operator (Condition Operator) and a processing operator (Processing Operator). Wherein the conditional operator may act as an a priori condition for the processing operator. The process operator is executed when the conditional operator is satisfied. Illustratively, the conditional operator spoken language includes a timed task, an asynchronous results checking task, and the like. The processing operators may comprise the main processing flow of the subtasks including, but not limited to, executing computing tasks, timing tasks, network communication tasks, and the like.

According to another embodiment of the present disclosure, the subtasks may be used to maintain information about its execution state, subtasks to be executed later, and degree of invasiveness, in addition to executing conditional operators and processing operators.

According to embodiments of the present disclosure, tasks may maintain the dependencies of all subtasks through the use of Graph structures. Illustratively, the graph may store the dependencies of all subtasks in the form of an adjacency table.

Based on this, for example, a topology map can be generated from the dependency relationship between the plurality of subtasks. The topological graph comprises a plurality of nodes and edges among the plurality of nodes, the plurality of nodes are in one-to-one correspondence with the plurality of subtasks, and the edges among the plurality of nodes are used for representing topological relations among the plurality of subtasks. The topology graph may be a directed graph, edges in the topology graph may have directives, and the directives of the edges may indicate dependency relationships between subtasks, e.g., the nodes of the edges that may be pointed to by the nodes that are corresponded to by the dependents point to the nodes that are corresponded to by the dependents.

According to embodiments of the present disclosure, data communication may be performed between subtasks to communicate context data. Based on this, a block Context space may be provided for storing Context data between subtasks. Subtasks in the same task may communicate context data by sharing a context space. For example, a block space may be configured in a memory as a Context space, for example.

According to embodiments of the present disclosure, a corresponding view (Viewer) module may also be configured for operators in the subtasks for accessing the context space and the context data corresponding to the operators. In this example, a view module may be utilized to define subspaces that each operator may access in the context space. By defining that each operator can only access the corresponding subspace, race conditions (Race conditions) can be avoided when multiple concurrent tasks access the context space. For example, access lock code may be added to the view module to define subspaces that each operator may access in the context space.

According to the embodiment of the disclosure, the task is divided into the plurality of subtasks, each subtask comprises the conditional operator and the processing operator, so that the number of threads waiting concurrently can be effectively reduced, and the thread resources are saved.

The topology provided by the present disclosure will be described below in connection with fig. 4 and the specific embodiments. Those skilled in the art will appreciate that the following example embodiments are merely for the understanding of the present disclosure, and the present disclosure is not limited thereto.

Illustratively, in this embodiment, the tasks may include subtask 1, subtask 2, subtask 3, subtask 4, and subtask 5. Wherein subtask 2 and subtask 3 depend on subtask 1, subtask 4 depends on subtask 2, and subtask 5 depends on subtask 3 and subtask 4.

Fig. 4 schematically illustrates a schematic diagram of a topology according to an embodiment of the present disclosure.

As shown in fig. 4, node 411 may be generated from subtask 1, node 412 may be generated from subtask 2, node 413 may be generated from subtask 3, node 414 may be generated from subtask 4, and node 415 may be generated from subtask 5. Edge 421 may then be generated from the dependency between subtask 2 and subtask 1, edge 422 may be generated from the dependency between subtask 3 and subtask 1, edge 423 may be generated from the dependency between subtask 4 and subtask 2, edge 424 may be generated from the dependency between subtask 3 and subtask 5, and edge 425 may be generated from the dependency between subtask 4 and subtask 5.

The method of generating distributed tasks provided by the present disclosure will be described below in connection with fig. 5.

Fig. 5 schematically illustrates a flow chart of a method of sequentially performing a plurality of subtasks in accordance with an embodiment of the present disclosure.

As shown in fig. 5, the sequentially executing a plurality of subtask methods 520 includes acquiring one of a plurality of subtasks in operation S521.

According to an embodiment of the present disclosure, each subtask may be acquired in turn, for example, according to a dependency relationship.

In operation S522, it is determined whether the subtask is an executable state. In the case where the subtask is in an executable state, operation S523 is performed. In the case where the subtask is in an executable state, operation S521 is returned.

In operation S523, a subtask is performed.

According to the embodiment of the disclosure, for example, whether the degree of penetration of the node corresponding to the subtask in the topological graph is 0 and whether the calculation result of the conditional operator in the subtask is successful can be determined. And determining that the subtask is in an executable state under the condition that the degree of penetration of the node corresponding to the subtask in the topological graph is 0 and the calculation result of the conditional operator in the subtask is successful.

According to the disclosed embodiments, performing the subtasks may include, for example: raw data is acquired from the corresponding subspace. And then, calculating the original data by using a calculation operator in the subtask to obtain a calculation result. The calculation result is then written into the corresponding subspace. Further, a subsequent node corresponding to the subtask may be determined in the topology map, and then the degree of invasiveness of the subsequent node is subtracted by 1. The raw data may be data that is pre-set in the context space and/or may be data that is written to the context space by other subtasks during execution.

According to embodiments of the present disclosure, threads may be created for a subtask in an executable state to execute the subtask, for example. When there are multiple subtasks in the executable state, multiple threads may be created to execute the subtasks in the executable state according to concurrency. The concurrency may be used to represent an upper limit of the number of threads concurrently executing at the same time, and may be set according to actual needs.

In accordance with embodiments of the present disclosure, a view module may also be created for each subtask when the subtask is created. Based on this, raw data may be acquired from the corresponding subspace using the view module. In addition, the view module can be utilized to write the calculation result into the corresponding subspace.

According to an embodiment of the present disclosure, when performing the subtasks may further include: updating the state of the subtask to be a processing operator of the subtask in execution, and acquiring a return result of the processing operator. And when the processing operator returns a successful result, the successful execution of the subtasks can be marked, and the topological graph is updated. When the processing operator returns a failure, the subtask execution failure can be marked, and the failure state is returned to the task layer. The updating the topology map may include, for example, traversing all successor nodes of the nodes corresponding to the current subtask in the topology map, and subtracting 1 from the degree of invasiveness of the successor nodes.

Fig. 6 schematically illustrates a flowchart of a method of performing distributed tasks according to an embodiment of the disclosure.

As shown in fig. 6, the method 600 of performing distributed tasks may further include operations S660-S680 in addition to the methods shown above. Operations S660-S680 may be performed, for example, during execution of a subtask.

In operation S660, it is detected whether the subtask is in a suspended state. In case that the subtask is in a suspended state, operation S670 is performed. In the case where the subtask is not in the suspended state, the process ends.

According to the embodiment of the disclosure, when a condition required for running a subtask is not satisfied and continuous running is impossible, the subtask may be set to a suspended state and the subtask is suspended.

In operation S670, a sleep time period is set according to the number of times of detection of the suspended state.

According to an embodiment of the present disclosure, the initial number of detections may be 0. The number of detections may be increased by 1 for each detection of the suspended state of a subtask.

According to embodiments of the present disclosure, the sleep duration may be calculated, for example, according to the following formula:

t＝n*a

wherein t is sleep time, n is detection times, and a is unit time. The unit time length can be set according to actual needs.

In operation S680, a sleep period is waited. After the sleep time period, the operation of detecting whether the subtask is in the suspended state is returned.

According to the embodiment of the disclosure, when the sleep time is not up, the thread executing the subtask can be dormant, and the time slice occupied by the subtask is yielded. Therefore, the thread resources occupied by the task with longer time consumption in the execution process can be effectively reduced.

The distributed task execution method according to the embodiments of the present disclosure may be executed by a task orchestration engine, for example. According to another embodiment of the present disclosure, an interface may also be provided in the task orchestration engine, allowing the user to customize the executor according to the actual situation, which may be used to execute the configuration policy for the sleep duration corresponding to each task.

According to another embodiment of the present disclosure, a profile may be preset. The configuration file can represent the dependent relationship between the subtasks in the task, operators in the subtasks, context information and other configuration information and the subtasks through character strings. The corresponding tasks may be generated from the configuration file using a reflective approach. For example, a parser may be utilized to parse the configuration file to obtain configuration information, such as subtasks in the task, operators in the subtasks, context space, and the like, and dependencies between the subtasks. Illustratively, the format of the configuration file may be JSON format, and the configuration file may include information such as a name (job_name) of the task (Job), a context (context) and a subtask (task) dependency, and an operator combination (operator) used by each subtask (task). Through the configuration file, a user can flexibly combine operators in the subtasks to construct the tasks.

In some containerized scenarios, the system may migrate under the influence of other systems deployed on-machine, resulting in the process being killed and thus the task being interrupted. Some subtasks which cannot be re-entered and consume longer time cannot be continuously executed, and users are required to manually take means of withdrawal, rollback and the like to process the subtasks, and then the subtasks are re-executed, so that manpower is wasted.

According to another embodiment of the present disclosure, serialization and persistence may be performed from both the task and subtask dimensions. For example, the key information of the task can be organized, so that the task orchestration engine can recover the task and the subtasks according to the key information of the organization orchestration. The key information may include, for example: context data, task name, subtask name, current task state, number of completed tasks, etc. Each task will serialize the key information described above for initializing the state of the task at persistence. For a subtask, the execution state of the subtask may be maintained. The dependency of the subtasks may be obtained by constructing a configuration file for the task.

According to the embodiment of the disclosure, by serializing and persistence from two dimensions of the task and the subtask, manual intervention of a user is not needed for the non-reentrant subtask, and manpower waste is reduced.

According to the embodiment of the disclosure, a persistence method can be decoupled from an orchestration engine by using a static distribution (CRTP) technology, so that a user can realize the persistence method without invading the orchestration engine, performance degradation caused by overlong addressing paths of virtual functions is avoided, and incompatibility caused by inconsistent APIs (such as inconsistent parameters) used in persistence is also avoided. The user needs to provide a storage location (such as a local database, a distributed database, etc.) of the serialized information to complete the persistence function.

According to the embodiment of the disclosure, for example, a persistent switch and a policy configuration can be further provided, and a user can actively turn on or off a persistent function according to own service and performance requirements, and can also configure a persistent time interval by adjusting a persistent frequency.

The apparatus for performing distributed tasks provided by the present disclosure will be described below in connection with fig. 7.

Fig. 7 schematically illustrates a block diagram of an apparatus for performing distributed tasks according to an embodiment of the disclosure.

As shown in fig. 7, an apparatus 700 for performing distributed tasks includes an acquisition module 710, an execution module 720, a generation module 730, a configuration module 740, and a creation module 750.

The obtaining module 710 is configured to obtain a distributed task, where the distributed task includes a plurality of subtasks.

And the execution module 720 is configured to sequentially execute the plurality of subtasks to obtain an execution result for the distributed task.

A generating module 730, configured to generate a plurality of subtasks according to operator information corresponding to the plurality of subtasks, where each subtask includes a conditional operator and a processing operator.

The configuration module 740 is configured to configure a topological relation among the plurality of subtasks according to the dependency relation among the plurality of subtasks.

The creating module 750 is configured to create a context space and configure a subspace corresponding to each subtask in the context space.

The distributed tasks are generated according to a generating module, a configuring module and a creating module.

According to an embodiment of the present disclosure, the configuration module may include: and the graph generation sub-module is used for generating a topological graph according to the dependency relationship, wherein the topological graph comprises a plurality of nodes and edges among the plurality of nodes, the plurality of nodes are in one-to-one correspondence with the plurality of subtasks, and the edges among the plurality of nodes are used for representing the topological relationship among the plurality of subtasks.

According to an embodiment of the present disclosure, the execution module may include: an executable state determination sub-module configured to determine, for each of the plurality of sub-tasks, whether the sub-task is an executable state; and a task execution sub-module for executing the sub-task in case that the sub-task is in an executable state.

According to an embodiment of the present disclosure, the task execution sub-module may include: an acquisition unit for acquiring original data from the corresponding subspace; the calculation unit calculates the original data by using a calculation operator in the subtask to obtain a calculation result; the writing unit is used for writing the calculation result into the corresponding subspace; a successor node unit, configured to determine a successor node corresponding to the subtask in the topology map; and a subtracting unit for subtracting 1 from the ingress of the subsequent node.

According to an embodiment of the present disclosure, the above apparatus may further include: the view creation module is used for creating a view module for each subtask; wherein the acquisition unit includes: the data acquisition subunit is used for acquiring original data from the corresponding subspace by utilizing the view module; and the writing unit includes: and the result writing subunit is used for writing the calculation result into the corresponding subspace by utilizing the view module.

According to an embodiment of the present disclosure, the above apparatus may further include: the input degree determining module is used for determining whether the input degree of the node corresponding to the subtask in the topological graph is 0 or not and determining whether the calculation result of the conditional operator in the subtask is successful or not; and the subtask state determining module is used for determining that the subtask is in an executable state when the degree of entry of the node corresponding to the subtask in the topological graph is 0 and the calculation result of the conditional operator in the subtask is successful.

According to an embodiment of the present disclosure, the above apparatus may further include: the detection module is used for detecting whether the subtasks are in a suspension state or not in the process of executing the subtasks; the setting module is used for setting sleeping time according to the detection times of the suspending state under the condition that the subtasks are in the suspending state; and the return module is used for returning to the operation for detecting whether the subtask is in a suspended state after the sleep time period.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

Fig. 8 schematically illustrates a block diagram of an example electronic device 800 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to the bus 804.

Various components in device 800 are connected to I/O interface 805, including: an input unit 806 such as a keyboard, mouse, etc.; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, or the like. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The computing unit 801 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as methods of performing distributed tasks. For example, in some embodiments, the method of performing distributed tasks may be implemented as a computer software program tangibly embodied on a machine readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 800 via ROM 802 and/or communication unit 809. When a computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the method of performing distributed tasks described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the method of performing distributed tasks by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above can be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual Private Server" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A method of performing a distributed task, comprising:

acquiring a distributed task, wherein the distributed task comprises a plurality of subtasks; and

sequentially executing the plurality of subtasks to obtain an execution result aiming at the distributed task,

wherein the distributed task is generated according to the following operations:

generating a plurality of subtasks according to operator information corresponding to the subtasks, wherein each subtask comprises a conditional operator and a processing operator;

according to the dependency relationship among the plurality of subtasks, configuring the topological relationship among the plurality of subtasks; and

and creating a context space and configuring a subspace corresponding to each subtask in the context space.

2. The method of claim 1, wherein the configuring the topological relationship between the plurality of subtasks according to the dependency relationship between the plurality of subtasks comprises:

and generating a topological graph according to the dependency relationship, wherein the topological graph comprises a plurality of nodes and edges among the plurality of nodes, the plurality of nodes are in one-to-one correspondence with the plurality of subtasks, and the edges among the plurality of nodes are used for representing the topological relationship among the plurality of subtasks.

3. The method of claim 2, wherein the sequentially executing the plurality of sub-tasks results in an execution result for the distributed task, comprising:

for each of the plurality of subtasks,

determining whether the subtask is in an executable state; and

and executing the subtask under the condition that the subtask is in an executable state.

4. The method of claim 3, wherein the performing the subtask comprises:

acquiring original data from the corresponding subspace;

calculating the original data by using a calculation operator in the subtask to obtain a calculation result;

writing the calculation result into a corresponding subspace;

determining a subsequent node corresponding to the subtask in the topological graph; and

subtracting 1 from the ingress of the successor node.

5. The method of claim 4, further comprising: creating a view module for each subtask;

the obtaining the original data from the corresponding subspace includes: acquiring original data from the corresponding subspace by utilizing the view module; and

the writing the calculation result into the corresponding subspace comprises the following steps: and writing the calculation result into the corresponding subspace by utilizing the view module.

6. The method of claim 2, further comprising:

determining whether the degree of penetration of the node corresponding to the subtask in the topological graph is 0 or not, and determining whether the calculation result of the conditional operator in the subtask is successful or not; and

and determining that the subtask is in an executable state under the condition that the degree of penetration of the node corresponding to the subtask in the topological graph is 0 and the calculation result of the conditional operator in the subtask is successful.

7. The method of claim 1, further comprising:

during the execution of the sub-tasks in question,

detecting whether the subtask is in a suspended state;

setting sleeping time according to the detection times of the suspending state under the condition that the subtasks are in the suspending state; and

and returning to the operation of f detecting whether the subtask is in a suspended state after the sleep time period.

8. An apparatus for performing distributed tasks, comprising:

the acquisition module is used for acquiring a distributed task, wherein the distributed task comprises a plurality of subtasks; and

an execution module for sequentially executing the plurality of subtasks to obtain an execution result for the distributed task,

the generating module is used for generating a plurality of subtasks according to operator information corresponding to the subtasks, wherein each subtask comprises a conditional operator and a processing operator;

the configuration module is used for configuring the topological relation among the plurality of subtasks according to the dependency relation among the plurality of subtasks; and

a creation module, configured to create a context space and configure a subspace corresponding to each subtask in the context space,

9. The apparatus of claim 8, wherein the configuration module comprises:

and the graph generation sub-module is used for generating a topological graph according to the dependency relationship, wherein the topological graph comprises a plurality of nodes and edges among the plurality of nodes, the plurality of nodes are in one-to-one correspondence with the plurality of subtasks, and the edges among the plurality of nodes are used for representing the topological relationship among the plurality of subtasks.

10. The apparatus of claim 9, wherein the execution module comprises:

an executable state determination sub-module configured to determine, for each of the plurality of sub-tasks, whether the sub-task is an executable state; and

and the task execution sub-module is used for executing the subtasks under the condition that the subtasks are in executable states.

11. The apparatus of claim 10, wherein the task execution sub-module comprises:

an acquisition unit for acquiring original data from the corresponding subspace;

the calculation unit calculates the original data by using a calculation operator in the subtask to obtain a calculation result;

the writing unit is used for writing the calculation result into the corresponding subspace;

a successor node unit, configured to determine a successor node corresponding to the subtask in the topology map; and

and a subtracting unit, configured to subtract 1 from the ingress of the subsequent node.

12. The apparatus of claim 11, further comprising: the view creation module is used for creating a view module for each subtask;

wherein the acquisition unit includes: the data acquisition subunit is used for acquiring original data from the corresponding subspace by utilizing the view module; and

the writing unit includes: and the result writing subunit is used for writing the calculation result into the corresponding subspace by utilizing the view module.

13. The apparatus of claim 9, further comprising:

the input degree determining module is used for determining whether the input degree of the node corresponding to the subtask in the topological graph is 0 or not and determining whether the calculation result of the conditional operator in the subtask is successful or not; and

and the subtask state determining module is used for determining that the subtask is in an executable state when the degree of penetration of the node corresponding to the subtask in the topological graph is 0 and the calculation result of the conditional operator in the subtask is successful.

14. The apparatus of claim 8, further comprising:

the detection module is used for detecting whether the subtasks are in a suspension state or not in the process of executing the subtasks;

the setting module is used for setting sleeping time according to the detection times of the suspending state under the condition that the subtasks are in the suspending state; and

and the return module is used for returning to the operation for detecting whether the subtask is in a suspended state after the sleep time period.

15. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-7.

17. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the method of any of claims 1-7.