CN116339958B

CN116339958B - Task execution method, device and equipment

Info

Publication number: CN116339958B
Application number: CN202310620838.XA
Authority: CN
Inventors: 袁小花; 周猛; 王牧群
Original assignee: Alipay Hangzhou Information Technology Co Ltd
Current assignee: Alipay Hangzhou Information Technology Co Ltd
Priority date: 2023-05-30
Filing date: 2023-05-30
Publication date: 2023-09-08
Anticipated expiration: 2043-05-30
Also published as: CN116339958A

Abstract

The embodiment of the specification discloses a task execution method, a task execution device and task execution equipment, belongs to the technical field of distribution, and is applied to nodes in a cluster. The scheme comprises the following steps: determining a database for maintaining a task state and a task sub-step state, wherein the task state represents a task execution related state, and the task sub-step state represents execution related states of a plurality of sub-steps separated by the task; acquiring a task state from a database; when the task state is in execution, judging whether a current execution node of the task is available or not, wherein the current execution node belongs to a cluster; if the current execution node is unavailable, updating the current execution node into the current execution node of the task, and keeping the task state as executing; obtaining the next sub-step of the task; the status of the next sub-step is updated to be in progress and the next sub-step is executed. The problem that the execution of a long task can lose the task state along with the restarting operation of the platform can be solved, and the task can be executed more reliably and efficiently.

Description

Task execution method, device and equipment

Technical Field

The present disclosure relates to the field of distributed technologies, and in particular, to a method, an apparatus, and a device for executing a task.

Background

With the rapid development of internet technology and computing power, the intelligence and the efficiency of the application for user service are greatly improved, and the back service platform is also subjected to a large task execution pressure, especially some tasks with large computing power and time consumption, and the tasks are called as long tasks.

In practical applications, the execution of the long task may lose the task state along with the restarting operation of the platform, so that the long task is difficult to be successfully executed on time, and a large amount of computing resources may be wasted to re-execute the long task.

Based on this, a more reliable and efficient task execution scheme is needed.

Disclosure of Invention

One or more embodiments of the present disclosure provide a method, an apparatus, a device, and a storage medium for performing tasks, so as to solve the following technical problems: there is a need for more reliable and efficient task execution schemes.

To solve the above technical problems, one or more embodiments of the present specification are implemented as follows:

one or more embodiments of the present disclosure provide a task execution method, applied to a node in a cluster, including:

determining a database for maintaining a task state and a task sub-step state, wherein the task state represents an execution related state of a task, and the task sub-step state represents respective execution related states of a plurality of sub-steps separated by the task;

Acquiring the task state from the database;

when the task state is in execution, judging whether a current execution node of the task is available or not, wherein the current execution node belongs to the cluster;

if the current execution node is unavailable, updating the current execution node to the current execution node of the task, and keeping the task state as executing;

a next sub-step of acquiring the task;

updating the state of the next sub-step into execution, and executing the next sub-step.

One or more embodiments of the present disclosure provide a task execution device, applied to a node in a cluster, where the device includes:

a state database determining module for determining a database for maintaining a task state and a task sub-step state, wherein the task state represents a task execution related state, and the task sub-step state represents execution related states of a plurality of sub-steps separated by the task;

the task state acquisition module acquires the task state from the database;

the node availability judging module is used for judging whether a current executing node of the task is available or not when the task state is in execution, wherein the current executing node belongs to the cluster;

The task continuing execution module updates the task to the current execution node of the task if the current execution node is unavailable, and keeps the task state as executing;

a next sub-step of acquiring the task;

One or more embodiments of the present disclosure provide a task execution device, applied to a node in a cluster, including:

at least one processor; the method comprises the steps of,

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to:

acquiring the task state from the database;

a next sub-step of acquiring the task;

One or more embodiments of the present specification provide a non-volatile computer storage medium storing computer-executable instructions configured to:

acquiring the task state from the database;

a next sub-step of acquiring the task;

The above-mentioned at least one technical solution adopted by one or more embodiments of the present disclosure can achieve the following beneficial effects: splitting a task into a plurality of sub-steps, storing the task state of the task and the sub-step state of the task sub-steps into a database, and updating and maintaining in response to the task execution condition of each task execution node, so that even if a platform is restarted to cause the task execution interruption of some task execution nodes at present, other nodes which are still available can track the specific sub-step of the task interruption in time according to the task state and the sub-step state in the database, continuously execute the sub-step from the breakpoint to finish, and continuously execute the following sub-step to try to finish the whole task, thereby being beneficial to more reliably and efficiently executing the task, and particularly being more advantageous to execute the long task.

Drawings

In order to more clearly illustrate the embodiments of the present description or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some of the embodiments described in the present description, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow diagram of a method of task execution provided in one or more embodiments of the present disclosure;

FIG. 2 is a schematic flow diagram of one embodiment of the method of FIG. 1 in one practical application scenario provided in one or more embodiments of the present disclosure;

FIG. 3 is a schematic diagram of a scheme for executing a long task of graph data based on topology matching in a scenario of a graph platform according to one or more embodiments of the present disclosure;

FIG. 4 is a schematic diagram of a task execution device according to one or more embodiments of the present disclosure;

fig. 5 is a schematic structural diagram of a task execution device according to one or more embodiments of the present disclosure.

Detailed Description

The embodiment of the specification provides a task execution method, a device, equipment and a storage medium.

In order to make the technical solutions in the present specification better understood by those skilled in the art, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.

In the background art, problems encountered in the actual application of a platform for executing a long task are mentioned. More intuitively, a graph platform scene based on graph data (including graph nodes, graph edges, corresponding attributes and the like) is specifically described, where task execution requirements are frequent and performance requirements are high.

For applications with large-scale user groups, the amount of data in the graph database used by the application is typically tens or even hundreds of T, and tasks such as point-to-point statistics, large-point statistics, or large-scale graph data compression for such graph databases are time-consuming, and the task completion time is often on the order of hours, thus becoming a relatively representative long task for the graph platform. Wherein: point-edge statistics generally refers to full graph traversal, and the data quantity owned by each point and edge type in a statistical graph; large-point statistics generally refers to full graph traversal, and the number of the output degrees and the input degrees of each point in a statistical graph are ordered, so that the input and output degrees larger than a set threshold (for example, 1 hundred million) can be defined as large points; and (3) compressing large-scale graph data, generally referred to as full graph traversal, and performing compression processing on the graph data to optimize the space.

In the prior proposal of the applicant, after a long task such as a point-edge statistics task is initiated, the graph platform is trained until the point-edge statistics task is executed. The process is operated in the thread, if the graphic platform performs restarting operation in the training process, the task state held by the thread is lost, the task cannot be continuously executed, and the whole task needs to be re-executed.

According to the scheme, the management of the platform on the tasks is optimized, and even if the platform is restarted, tasks which are not executed before can be continued, so that the problem of executing long tasks is solved. The following continues to describe the solution of the application in detail.

Fig. 1 is a flow chart illustrating a task execution method according to one or more embodiments of the present disclosure. The process can be executed on a platform with tasks, especially long task execution requirements, such as an e-commerce platform, a payment platform and the like, and can be executed on at least part of nodes in a cluster used by the platform. Some input parameters or intermediate results in the flow allow for manual intervention adjustments to help improve accuracy.

The flow in fig. 1 includes the following steps:

s102: a database of maintenance task states and task sub-step states is determined, the task states representing execution-related states of the corresponding tasks, and the task sub-step states (also simply referred to as sub-step states or sub-step states) representing execution-related states of a plurality of sub-steps separated by the tasks, respectively.

In one or more embodiments of the present disclosure, a task is split into a plurality of sub-steps with a set execution order, and if necessary, a part of the sub-steps are also supported to be executed concurrently, so that, while the overall state of the task is tracked, the specific execution interrupt position (may be referred to as a breakpoint) of the task can be tracked in a fine granularity, and then, on the sub-step with the specific execution interrupt position, a part of the sub-steps that has not been completed yet is executed continuously. For long tasks, more sub-steps can be broken down according to the actual situation.

For a task, its execution-related state may include several states of waiting for execution, executing, success, failure, etc. Waiting for execution indicates that the task as a whole has not yet started executing. The execution indicates that the task is being executed by an existing node, and that the task has not been executed, the node may be executing the task normally, or may be temporarily unavailable due to a restart. Success or failure indicates the execution result of the task, and for the task in a failure state, a policy may be set to allow re-execution of the task.

Similarly, for a sub-step, its execution-related state may include several states of waiting for execution, executing, success, failure, etc. In the case where the sub-steps of the same task are performed in series, the execution-related state may not include a wait for execution, and an attempt may be made to trace the sub-steps in execution.

In one or more embodiments of the present disclosure, in order to track the state more reliably, the task state and the sub-step state are persisted into the database instead of being held all the time by the execution thread, and the task state and the sub-step state in the database are updated in time accordingly according to the execution condition.

In order to facilitate the acquisition of tasks or sub-steps by a node, in particular in half-way, for execution, the content of the tasks and sub-steps themselves (not only the corresponding identification) may also be persisted into the database, so that when the task state or sub-step state is tracked based on the database, tasks or sub-steps that need to be executed in succession corresponding to the state may be conveniently acquired. At the same time, a plurality of different tasks can be maintained in the database at the same time, in which case, the database also maintains the correct corresponding relation among the tasks, sub-steps and states.

In one or more embodiments of the present disclosure, the splitting manner of the task into multiple sub-steps is various, and may be performed according to the selection differentiation of the task execution nodes, so as to facilitate a more reasonable successive execution with lower cost. For example, if a management node (such as a Master machine) is used as a task execution node, the task can be split more broadly from the perspective of the management node, the split sub-steps are relatively not concerned with a lot of specific computing actions, and if a working node (such as a worker machine) is used as a task execution node, the task can be thinned to specific computing actions or computing region splitting.

Intuitively, take the point-edge statistics task under the platform scene as an example. Assuming that the point edge statistics task is performed by multiple management nodes, split into: a plurality of substeps (here, exemplary, a plurality of substeps may be split into a large number of substeps as needed) of issuing a point-side statistics task, querying a task progress, obtaining a point-side statistics task result, and the like; the specific statistical actions involved in the method can be executed by the management node issuing to a plurality of working nodes governed by the management node, but the actions of the working nodes are not required to be related on the task level. Assuming that the point edge statistics task is performed by multiple working nodes, split into: receiving point edge statistics tasks, carrying out point edge statistics on an area 1 in the graph data, carrying out point edge statistics on an area 2 in the graph data, carrying out point edge statistics on an area N in the graph data, reporting local area statistics results, forwarding local area statistics results and the like.

S104: and acquiring the task state from the database.

S106: and when the task state is in execution, judging whether a current execution node of the task is available, wherein the current execution node belongs to the cluster.

In one or more embodiments of the present disclosure, it is determined which state the acquired task state is currently in, e.g., waiting for execution, executing, etc. If so, continuing to execute the steps S106-S112, otherwise, having other optional steps, and supplementing the description.

Even if the task state is executing, it is not necessarily the case that it is only possible to specify that a node has previously had a corresponding task or sub-step next, but whether the node is currently actually executing the corresponding sub-step normally. For example, if the node is restarted, it may temporarily not have availability, and it may take a period of time (e.g., minutes, tenths of a minute, or even longer) before the restart is completed.

In one or more embodiments of the present description, to facilitate tracking node availability, information (including at least node identification, and optionally node current status, node capabilities, node location, etc.) for each task and for each sub-step, if any, may be currently executed, as well as correspondence between that information and tasks and sub-steps, may also be persisted into a database and updated in time. In addition, part of the intermediate results of the sub-steps that have already been performed may be persisted to a database for use in subsequent execution.

The current execution node in step S106 is not the same node as the execution subject node of the step, and the execution subject node can freely pay attention to the situation of different current execution nodes of a plurality of tasks or a plurality of sub-steps so as to continue when needed. At present, the execution main body node can flexibly execute other tasks, so that the status and the identity of the nodes are not required to be limited like a main and standby mechanism based on the task state and the sub-step state in the database, the execution main body node is more flexible and efficient, and node resources are more fully utilized.

And setting a judging strategy for node availability according to actual conditions. If the node itself maintains a relatively real-time node state, for example, during restarting, downtime, active/standby switching, etc., the node availability can be determined by directly referring to the maintained node state. Otherwise, node availability may be determined by attempting to communicate with the currently executing node, or querying a historical communication record (e.g., heartbeat signal, etc.) of the currently executing node.

S108: and if the current execution node is not available, updating the current execution node to the current execution node of the task, and keeping the task state as executing. Steps S110 and S112 are continued.

In one or more embodiments of the present description, information is updated in a database, and the currently executing node of the task is replaced by itself, and since itself is currently available and has to be continued with execution of the task, the task state may be maintained as executing.

For example, assume that tasks and sub-steps are also saved in a database, associated with corresponding execution-related states, for execution. The node may associate its own identity with the task or sub-step in the database when preparing to continue to represent itself as the currently executing node of the task or sub-step, and may directly obtain the sub-step to be performed in succession from the database.

In addition, if the task is allowed to be re-executed after the failure in policy, the task with the failed task state or the sub-step with the failed sub-step state may be concerned, in this case, if the task is failed and the current execution node of the task is not available, the task may be updated to the current execution node of the task, the task state may be updated from the failure to the execution, and further, the sub-step with the failed sub-step may be specifically tracked, the sub-step state may be updated from the failure to the execution, and the re-execution of the sub-step may be attempted. Further, if the failure state of a task is decoupled from the failure state of some of its sub-steps (failure states of some of the sub-steps do not result in the entire task being updated to a failure state), then the sub-step states may be tracked directly in the database without having to look at the task state first.

S110: the next sub-step of the task is acquired.

In one or more embodiments of the present specification, the next sub-step refers to the sequentially foremost sub-step of the task that has not yet been successfully performed. The next sub-step may be executing (which means that the sub-step may have already been executed for a part, but has not been executed normally for a while due to the unavailability of the current executing node) or waiting for execution (which means that the sub-step as a whole has not yet started to execute).

The sub-step state corresponding to the task is obtained from the database, and particularly, the sub-step state can be executed as an executing sub-step, and the sub-step is executed successively in time.

S112: updating the state of the next sub-step into execution, and executing the next sub-step.

If the state of the next sub-step is already in execution, the state is maintained to continue to execute the next sub-step until the next sub-step is successfully executed, and then the state is updated to be successful.

In one or more embodiments of the present disclosure, the sub-step is executed sequentially, and the sub-step may be executed continuously from a portion of the sub-step that has not yet been executed, and if the sub-step is executed successfully, the sub-step may be executed continuously to try to complete the task.

If the node is not available in the execution process, other nodes can try to continue the execution of the interrupted sub-step.

By the method of fig. 1, a task is split into a plurality of sub-steps, the task state of the task and the sub-step state of the task sub-steps are stored in a database, and update maintenance is performed in response to the task execution condition of each task execution node, so that even if a platform is restarted to cause the task execution interruption of some task execution nodes currently, other available nodes can still track the specific sub-step of the task interruption in time according to the task state and the sub-step state in the database, and continue to execute the sub-step continuously from a breakpoint to complete, and continue to execute the following sub-step to try to complete the whole task, thereby being beneficial to more reliably and efficiently executing the task, and particularly, being beneficial to executing long tasks.

Based on the method of fig. 1, the present specification also provides some specific embodiments and extensions of the method, and the following description will proceed.

In one or more embodiments of the present disclosure, for step S104, after the task state is obtained from the database, if it is determined that the task state is not currently being executed but is waiting for execution, it indicates that the corresponding task has not yet started to be executed as a whole, that is, there is no current task execution node of the task temporarily, and the task state may be updated to be the executing current execution node of the task; after such an update, the next sub-step (e.g., the first sub-step) of the task is acquired, the sub-step status of the next sub-step is updated to be in execution, and execution of the next sub-step is started to attempt to complete the task.

The situation belongs to the starting execution of the task at the beginning, but not the continuous execution of the task or the sub-step, when the node is unavailable, other nodes can try to continuously execute, in the process, binding relations such as a master and a slave do not need to be established between different nodes, the method is looser and more flexible, is beneficial to the timely and efficient continuous execution, and can reduce the influence caused by the unavailable current task executing node as much as possible.

In one or more embodiments of the present disclosure, although the present node may actively attempt to contact the currently executing node to make the availability determination, since other nodes may also make the same attempt at the same time, the communication overhead between nodes is not cost-effective and increases the node pressure. In order to solve this problem, the current executing node may report the heartbeat signal at regular time in the available duration, for example, report the heartbeat signal to a unified management node or a node that is idle, and the node that receives the heartbeat signal records the report time of the heartbeat signal in the database, which may, of course, also report the heartbeat signal directly to the database server for storage.

Based on the thought, the node updates itself to the current execution node of the task, and after updating the task state to be in execution, when judging whether the current execution node of the task corresponding to the task state is available, determining the current execution node of the task corresponding to the task state and the reporting time of the latest report heartbeat signal thereof through the database, judging whether the reporting time of the latest report heartbeat signal is overtime from the current time, if so, determining that the current execution node is unavailable, otherwise, considering that the task is in normal execution, and can be ignored, or adopting other measures to continuously analyze the node availability.

In light of the foregoing, one or more embodiments of the present disclosure provide a practical implementation of the method of fig. 1, the flow of which is shown in fig. 2.

The scheme in fig. 2 includes the following steps:

the tasks of the graph platform for the graph database, such as point-edge statistics tasks, are split into multiple sub-steps. For example, the sub-steps of starting point edge statistics task, inquiring task progress, obtaining point edge statistics results and the like are split and called.

The method comprises the steps of storing tasks into a task table in a database, storing sub-steps into a sub-step table in the database, recording the correct execution sequence of each sub-step, and establishing the correct corresponding relation between the tasks and the sub-steps in the table. Initially, the state of tasks and sub-steps is set to wait for execution (e.g., denoted as waiting run).

The node (for example, a certain cluster server of the graph platform) scans the idle thread quantity of the own thread pool through timer training, and if the idle thread exists, the idle thread scans the task state in the database to determine the state as the task waiting for execution and/or in execution (for example, as running), thereby being beneficial to finding out abnormality in time and continuing, and being capable of more fully utilizing own thread resources.

If the task state in the task table is the task waiting for execution, the lock is robbed to execute, the current task execution node of the task is updated in the database to be self (for example, the host information of the task is recorded), the next sub-step of the task is started to be acquired, the state of the updated task is the state of the current sub-step in execution, the sub-step is started to be executed, and after the execution is finished, the state of the updated sub-step is successful or failed according to the result of success or failure of the execution. And so on, attempting to acquire the next sub-step for processing, and if all the sub-steps of the task are successfully executed in the correct sequence, updating the task state to be successful.

If the task state in the task table is the task in execution, checking to record the current task execution node for the task, and inquiring whether the reporting time of the latest heartbeat signal is overtime. If the time-out is not over, skipping, and ignoring the task; if the time is out, the lock is robbed to be executed, the current task executing node is updated to be self, and the task progress is started to be tracked.

Sequentially searching the states of the corresponding sub-steps from the database according to the sequence among the sub-steps of the task; if the currently searched sub-step state is failure, the task state of the update task is failure; if the state of the currently searched sub-step is successful, continuing to track the next sub-step; if the currently found sub-step state is in execution, the last execution is described to the breakpoint of the sub-step, tracking is finished, a connection point is found, the sub-step corresponding to the sub-step state is continuously executed, and the execution is continued from the connection point as much as possible. If the sub-step fails to be executed, updating the sub-step state to be failed, and attempting to re-execute the sub-step; if the sub-step is successfully executed, the state of the sub-step is updated successfully, and if the sub-step is the last sub-step of the task, the state of the task is updated successfully, and if the sub-step is not the last sub-step of the task, the next sub-step of the task is continuously acquired for processing.

The applicant has mentioned above that using the above scheme in the scenario of the graph platform, the characteristics of the scenario can be utilized to improve the global efficiency in the case where there may be multiple node tracking states to attempt to continue a task or sub-step. The idea is as follows: in practical applications, clusters are often distributed, nodes may be scattered in different machine rooms or even different urban areas, overheads such as various communications are differentiated, when a currently executed node of a certain task is not available, a plurality of other nodes may track to try to execute continuously based on the scheme, and how the plurality of other nodes are reasonably matched is considered to improve global efficiency and reduce unnecessary competition; the long task in the graph platform scene is usually aimed at large-scale graph data, the graph data is composed of nodes, edges and related attributes, the whole long task is required to be performed in the graph data, namely, the nodes go to another node through the connected edges, corresponding statistics is performed on each node and/or each edge, in this case, work on each area of the graph data can be considered as distributed in the whole graph data, the work distributed condition in the graph data is considered to be matched with the node distributed condition in the cluster, so that the nodes perform task connection according to the matched area as much as possible, on one hand, cost reduction and competition reduction are facilitated, and on the other hand, after the nodes finish the current sub-step, other sub-steps related to the surrounding area are also facilitated to be explored nearby.

Based on the idea of the previous paragraph, one or more embodiments of the present disclosure provide a scheme for executing a long task of graph data based on topology matching in a scenario of a graph platform, see fig. 3. In this scenario, the tasks are tasks of the graph platform for the graph database, for example long tasks.

The scheme in fig. 3 includes the following steps:

s302: and obtaining a similar regional distribution diagram obtained by matching the node topological graph of the cluster with the topological graph of the graph data related by the long task in the graph database.

The node topology map may reflect the distribution of nodes in the cluster in terms of communication connections or in terms of machine room and regional locations. The topology graph of the graph data can reflect the distribution of graph nodes and the connecting edges between the graph nodes. In the matching, in addition to matching in terms of the degree of similarity of graphics, the connection weight may be considered. The similar region distribution map may be a topological map having a similarity in a part extracted from the node topological map and the topological map of the map data, respectively, and may be represented separately or may be represented by merging.

S304: judging whether the user is in the similar region distribution diagram or not, and obtaining the judging result to be yes.

If the nodes are in the similar region distribution diagram, the nodes are more suitable for task connection in the similar region distribution diagram, otherwise, the nodes in the similar region distribution diagram are preferentially connected with the tasks in the region, so that the competition and the expenditure are reduced.

S306: updating the current execution node to the current execution node of the long task.

S308: executing the sub-step state is the regional sub-step that is executing and matches itself in the similar regional distribution map.

In one or more embodiments of the present disclosure, the multiple sub-steps of long task splitting include a regional sub-step of processing a specified local area of the related graph data, for example, by executing the regional sub-step for performing dot edge statistics on the specified local area. The node performs task connection in the similar area to which the node belongs as much as possible.

In one or more embodiments of the present disclosure, if, for the present node, there is no task available for connection in the similar area to which the present node belongs, the present node continues to explore and track other possible tasks available for connection from the edge of the similar area to the adjacent area, so as to avoid node resource limitation, and the exploration cost is relatively smaller.

Based on the same thought, one or more embodiments of the present disclosure further provide apparatuses and devices corresponding to the above method, see fig. 4 and fig. 5, respectively. The apparatus and device are capable of performing the above method and related alternatives accordingly.

Fig. 4 is a schematic structural diagram of a task execution device according to one or more embodiments of the present disclosure, where the task execution device is applied to a node in a cluster, and the device includes:

a status database determining module 402, configured to determine a database of maintenance task status and task sub-step status, the task status representing a task execution-related status, the task sub-step status representing respective execution-related status of a plurality of sub-steps split by the task;

a task state acquisition module 404, configured to acquire the task state from the database;

a node availability judging module 406, configured to judge, when the task status is in execution, whether a current execution node of the task is available, where the current execution node belongs to the cluster;

a task continuing execution module 408, configured to update itself to the current execution node of the task and keep the task state as executing if the current execution node is unavailable;

A next sub-step of acquiring the task;

Optionally, after the task state is obtained from the database, the task continuing execution module 408 updates itself to a current execution node of the task and updates the task state to be in execution when the task state is waiting for execution;

a next sub-step of acquiring the task;

Optionally, the task continuation execution module 408 updates the sub-step status to failed if the sub-step fails to execute;

if the sub-step is successfully executed, updating the state of the sub-step to be successful;

updating the task state of the task to be successful if the sub-step is the last sub-step of the task; in the case that the sub-step is not the last sub-step of the task, the next sub-step of the task is obtained for processing.

Optionally, the task continuing execution module 408 sequentially searches the database for the corresponding sub-step states according to the sequence between the sub-steps of the task after updating itself to the current execution node of the task;

If the currently searched sub-step state is failure, updating the task state of the task to be failure;

if the currently searched sub-step state is successful, continuing to track the next sub-step;

if the currently searched sub-step state is in execution, tracking is finished, and continuing to execute the sub-step corresponding to the sub-step state.

Optionally, the method further comprises:

the heartbeat signal reporting recording module 410 is configured to report, in a period when the heartbeat signal is still the current executing node of the task after updating the heartbeat signal to the current executing node of the task and updating the task state to the executing state, so that the reporting time of the heartbeat signal is recorded in the database;

the task continuing execution module 408 determines, by the database, a current execution node of a task corresponding to the task state and a reporting time of a last report of a heartbeat signal of the current execution node before updating itself to the current execution node of the task if the current execution node is unavailable;

judging whether the reporting time of the last reported heartbeat signal is overtime from the current time;

If yes, determining that the current executing node is not available.

Optionally, the task state obtaining module 404 scans the number of idle threads in its own thread pool periodically;

and if the idle thread exists, scanning the task state in the database through the idle thread to determine the state as a task waiting to be executed or being executed.

Optionally, the tasks and the sub-steps are also saved in the database, associated with corresponding execution-related states for execution;

the task continuation execution module 408 associates its identity with the task or the sub-step in the database to represent itself as the current executing node of the task.

Optionally, the task is a task of the graph platform aiming at a graph database;

the task includes at least one of:

dot edge statistics, large dot statistics, and large-scale graph data compression.

the task continuing execution module 408 obtains a similar regional distribution diagram obtained by matching a node topology map of the cluster with a topology map of map data related to the task in the map database before the update of the task itself to a current execution node of the task;

Judging whether the user is in the similar region distribution diagram or not, and obtaining the judging result to be yes.

Optionally, the task splitting sub-step includes a regional sub-step of processing a designated local region of the related graph data;

the task continuation execution module 408 executes the substep state as the regional substep that is executing and matches itself in the similar regional distribution map.

Fig. 5 is a schematic structural diagram of a task execution device according to one or more embodiments of the present disclosure, where the task execution device is applied to a node in a cluster, and the device includes:

at least one processor; the method comprises the steps of,

a memory communicatively coupled to the at least one processor; wherein,,

Acquiring the task state from the database;

a next sub-step of acquiring the task;

Based on the same considerations, one or more embodiments of the present specification further provide a non-volatile computer storage medium storing computer-executable instructions configured to:

acquiring the task state from the database;

a next sub-step of acquiring the task;

In the 90 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable Gate Array, FPGA)) is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented by using "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before the compiling is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but not just one of the hdds, but a plurality of kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), lava, lola, myHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.

The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present specification.

It will be appreciated by those skilled in the art that the present description may be provided as a method, system, or computer program product. Accordingly, the present specification embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description embodiments may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.

The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus, devices, non-volatile computer storage medium embodiments, the description is relatively simple, as it is substantially similar to method embodiments, with reference to the section of the method embodiments being relevant.

The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The foregoing is merely one or more embodiments of the present description and is not intended to limit the present description. Various modifications and alterations to one or more embodiments of this description will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, or the like, which is within the spirit and principles of one or more embodiments of the present description, is intended to be included within the scope of the claims of the present description.

Claims

1. A task execution method applied to nodes in a cluster, wherein the task is a task of a graph platform aiming at a graph database, the method comprising:

acquiring the task state from the database;

A next sub-step of acquiring the task;

updating the state of the next sub-step into execution, and executing the next sub-step;

before updating itself to the current executing node of the task, the method further includes:

obtaining a similar regional distribution diagram obtained by matching a node topological graph of the cluster with a topological graph of graph data related to the task in the graph database;

2. The method of claim 1, after said retrieving said task state from said database, further comprising:

when the task state is waiting for execution, updating the task state to be the current execution node of the task, and updating the task state to be executing;

a next sub-step of acquiring the task;

3. The method of claim 1, further comprising:

if the sub-step fails to be executed, updating the state of the sub-step to be failed;

4. The method according to claim 1 or 2, after said updating itself to the currently executing node of the task, the method further comprising:

sequentially searching the states of the corresponding sub-steps from the database according to the sequence among the sub-steps of the task;

5. The method of claim 2, wherein after updating itself to the currently executing node of the task and updating the task state to be executing, the method further comprises:

the method comprises the steps of reporting heartbeat signals at regular time in a period that the heartbeat signals are still current executing nodes of the task, so that the reporting time of the heartbeat signals is recorded in a database;

Before updating itself to the current execution node of the task if the current execution node is not available, the method further includes:

determining a current execution node of a task corresponding to the task state and the reporting time of the latest heartbeat signal;

if yes, determining that the current executing node is not available.

6. The method of claim 2, further comprising:

the number of idle threads in a thread pool is scanned regularly;

7. The method of claim 1, said tasks and said sub-steps also being stored in said database in association with corresponding execution-related states for execution;

the updating the self to the current execution node of the task specifically comprises the following steps:

in the database, an identification of itself is associated with the task or the sub-step to represent itself as the currently executing node of the task.

8. A method according to any one of claims 1 to 3, wherein the task is a long task of a graph platform for a graph database;

the long task includes at least one of:

9. The method of claim 1, wherein the task splitting includes a regional sub-step of processing a specified local region of the map data;

the executing the next substep specifically includes:

executing the sub-step state is the regional sub-step that is executing and matches itself in the similar regional distribution map.

10. A task execution device applied to nodes in a cluster, the task being a task of a graph platform for a graph database, the device comprising:

the task state acquisition module acquires the task state from the database;

a next sub-step of acquiring the task;

the task continuing execution module acquires a similar regional distribution diagram obtained by matching a node topological diagram of the cluster with a topological diagram of map data related to the task in the map database before updating the task to a current execution node of the task;

11. The apparatus of claim 10, wherein the task continuation execution module updates itself to a current execution node of the task and updates the task state to executing when the task state is waiting for execution after the task state is obtained from the database;

a next sub-step of acquiring the task;

12. The apparatus of claim 10, the task continuation execution module to update the sub-step status as failed if the sub-step execution fails;

13. The apparatus according to claim 10 or 11, wherein the task continuation execution module, after updating itself to a current execution node of the task, sequentially searches for a corresponding sub-step state from the database in order between sub-steps of the task;

14. The apparatus of claim 11, further comprising:

the heartbeat signal reporting and recording module is used for reporting heartbeat signals at regular time in a period that the heartbeat signals are still the current executing node of the task after the current executing node of the task is updated to the current executing node of the task and the task state is updated to be in execution, so that the reporting time of the heartbeat signals is recorded in a database;

the task continuing execution module is used for determining the current execution node of the task corresponding to the task state and the reporting time of the latest heartbeat signal reporting by the current execution node through the database before updating the current execution node into the current execution node of the task if the current execution node is unavailable;

if yes, determining that the current executing node is not available.

15. The apparatus of claim 11, wherein the task state acquisition module periodically scans for a number of threads that are free from its own thread pool;

16. The apparatus of claim 10, the tasks and the sub-steps also stored in the database in association with corresponding execution-related states for execution;

the task continuing execution module associates an own identification with the task or the substep in the database to represent itself as a current execution node of the task.

17. The apparatus according to any one of claims 10 to 12, wherein the task is a task of a graph platform for a graph database;

the task includes at least one of:

18. The apparatus of claim 10, wherein the task splitting sub-step includes a regional sub-step of processing a specified local region of the map data;

the task continuing execution module executes the substep state as the regional substep which is in execution and matched with the task in the similar regional distribution diagram.

19. A task execution device applied to nodes in a cluster, the task being a task of a graph platform for a graph database, the device comprising:

at least one processor; the method comprises the steps of,

A memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform:

acquiring the task state from the database;

a next sub-step of acquiring the task;

before updating itself to the current executing node of the task, the method further comprises: