WO2022105531A1 - Distributed multi-task management method and apparatus, and computer device and storage medium - Google Patents

Abstract

Disclosed is a distributed multi-task management method. The method comprises: acquiring a task execution request, and traversing, on the basis of the task execution request, a set of tasks that are created on distributed middleware; acquiring at least two target arrays in the set of tasks, wherein each of the target arrays comprises a task ID and a task sequence number of a task to be processed; acquiring an execution node quantity corresponding to idle execution nodes, and determining, according to the task sequence number and the execution node quantity, a target execution node corresponding to each task to be processed; and calling threads corresponding to the execution node quantity to execute at least two tasks to be processed, and acquiring execution results corresponding to the at least two tasks to be processed. By means of the distributed multi-task management method provided by the present application, tasks to be processed are averagely distributed to idle execution nodes, the processing logic is simple and efficient, and the execution speed for the tasks to be processed is accelerated.

Description

Distributed multitasking management method, device, computer equipment and storage medium

This application is based on the Chinese invention application with the application number 202011283749.3 filed on November 17, 2020, entitled "Distributed Multitasking Management Method, Apparatus, Computer Equipment and Storage Medium", and claims its priority.

technical field

The present application relates to the field of computer technologies, and in particular, to a distributed multitasking management method, apparatus, computer equipment, and storage medium.

Background technique

At present, to process multiple tasks to be executed in a distributed environment, a task is usually automatically allocated to one or more execution nodes for processing through a lock mechanism. However, the inventor found in the actual operation that this method has centralized processing on some nodes. There are multiple pending tasks, and some nodes only process a few or no pending tasks, resulting in heavy node load during automatic allocation, slow task execution, and excessive pressure. CPU resources and even downtime. For example, currently there are 6 tasks to be executed, each task to be executed corresponds to a lock, if the node A To grab five locks, you need to execute five tasks to be executed. The load of this node is much larger than that of other nodes, resulting in the problem of node load imbalance. Moreover, the invention also realizes that there are two tasks (forward task and backward task) with execution dependencies in the prior art, and the backward task may be executed first. Backward tasks are repeatedly executed, wasting resources.

technical problem

Embodiments of the present application provide a distributed multitasking management method, apparatus, computer equipment, and storage medium, so as to solve the problems of unbalanced node load and waste of resources.

technical solutions

A distributed multitasking management method, comprising:

Obtain a task execution request, and traverse the task collection created on the distributed middleware based on the task execution request;

Obtain at least two target arrays in the task set, each of which includes the task ID and the task sequence number of the task to be processed;

Obtain the number of execution nodes corresponding to the idle execution nodes, and determine the target execution node corresponding to each of the tasks to be processed according to the task sequence number and the number of execution nodes;

A thread corresponding to the number of the execution nodes is called to execute at least two of the tasks to be processed, and execution results corresponding to the at least two tasks to be processed are obtained.

A distributed multitasking management device, comprising:

a task set traversal module, configured to obtain a task execution request, and traverse the task set created on the distributed middleware based on the task execution request;

A target array acquisition module, used to obtain at least two target arrays in the task set, each of the target arrays including the task ID and the task sequence number of the task to be processed;

a target execution node determination module, configured to obtain the number of execution nodes corresponding to the idle execution nodes, and determine the target execution node corresponding to each of the to-be-processed tasks according to the task sequence number and the number of execution nodes;

The execution result obtaining module is configured to call threads corresponding to the number of execution nodes, execute at least two tasks to be processed, and obtain execution results corresponding to at least two tasks to be processed.

A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer-readable instructions:

Obtain a task execution request, and traverse the task collection created on the distributed middleware based on the task execution request;

Obtain at least two target arrays in the task set, each of which includes the task ID and the task sequence number of the task to be processed;

Obtain the number of execution nodes corresponding to the idle execution nodes, and determine the target execution node corresponding to each of the tasks to be processed according to the task sequence number and the number of execution nodes;

A thread corresponding to the number of the execution nodes is called to execute at least two of the tasks to be processed, and execution results corresponding to the at least two tasks to be processed are obtained.

One or more readable storage media having computer-readable instructions stored thereon, the computer-readable storage media having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by one or more processors, cause all The one or more processors perform the following steps:

Obtain a task execution request, and traverse the task collection created on the distributed middleware based on the task execution request;

Obtain at least two target arrays in the task set, each of which includes the task ID and the task sequence number of the task to be processed;

Obtain the number of execution nodes corresponding to the idle execution nodes, and determine the target execution node corresponding to each of the tasks to be processed according to the task sequence number and the number of execution nodes;

A thread corresponding to the number of the execution nodes is called to execute at least two of the tasks to be processed, and execution results corresponding to the at least two tasks to be processed are obtained.

beneficial effect

The above-mentioned distributed multi-task management method, device, computer equipment and storage medium obtains a task execution request, traverses the task set created on the distributed middleware based on the task execution request, and provides technical support for the subsequent realization of the equal distribution of pending tasks . Obtain at least two target arrays in the task set, and each target array includes the task ID and task sequence number of the task to be processed. When the server reads the target array, it can intelligently assign the task to be processed to In the execution node, an even distribution is achieved. Obtain the number of execution nodes corresponding to the idle execution nodes, determine the target execution node corresponding to each of the pending tasks according to the task sequence number and the number of execution nodes, and evenly distribute the pending tasks to the idle execution nodes. The processing logic is simple and efficient. Efficient, speeding up the execution of pending tasks. Solve the problem of uneven distribution in the lock mechanism, resulting in excessive load on some execution nodes and small load on some execution nodes. The threads corresponding to the number of execution nodes are called to execute at least two of the to-be-processed tasks, and execution results corresponding to at least two of the to-be-processed tasks are acquired, so as to process the to-be-processed tasks.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. , for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

1 is a schematic diagram of an application environment of a distributed multitasking management method in an embodiment of the present application;

2 is a flowchart of a distributed multitasking management method in an embodiment of the present application;

3 is another flowchart of a distributed multitasking management method in an embodiment of the present application;

4 is another flowchart of a distributed multitasking management method in an embodiment of the present application;

5 is another flowchart of a distributed multitasking management method in an embodiment of the present application;

6 is another flowchart of a distributed multitasking management method in an embodiment of the present application;

7 is another flowchart of a distributed multitasking management method in an embodiment of the present application;

8 is a schematic diagram of a distributed multitasking management device in an embodiment of the present application;

FIG. 9 is a schematic diagram of a computer device in an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The distributed multi-task management method provided by the embodiment of the present application can be applied in the application environment shown in FIG. 1 . Specifically, the distributed multitasking management method is applied in a distributed multitasking management system, and the distributed multitasking management system includes N clients and servers as shown in FIG. Communication is used to evenly distribute tasks to be processed to idle execution nodes, the processing logic is simple and efficient, and the execution speed of tasks to be processed is accelerated. Among them, the client is also called the client, which refers to the program corresponding to the server and providing local services for the client. Clients can be installed on, but not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented as an independent server or a server cluster composed of multiple servers.

In one embodiment, as shown in FIG. 2 , a distributed multi-task management method is provided, and the method is applied to the server in FIG. 1 as an example for description, including the following steps:

S201: Obtain a task execution request, and traverse the task set created on the distributed middleware based on the task execution request.

The task execution request is a request for a task to be processed.

Distributed middleware can extend the communication between processes while reducing the degree of coupling between multiple systems. Understandably, creating a task set on a distributed intermediate can ensure that tasks to be processed by multiple systems are processed at the same time.

The task collection is the set collection of redis. The set collection is an unordered collection of String type. The collection members of the set collection are unique, which means that duplicate data cannot appear in the collection.

In this embodiment, when a task to be processed is acquired, a task set is generated according to the to-be-processed task, so that the to-be-processed task can be quickly allocated to the task set, and when a task execution request is acquired, a return result command is used to read the task set , to provide technical support for the subsequent realization of the equal distribution of pending tasks, so as to solve the problem of relying on the lock mechanism to process pending tasks in the prior art, resulting in a large number of pending tasks on some execution nodes and a small number of pending tasks, resulting in Execution node load imbalance problem.

S202: Acquire at least two target arrays in the task set, where each target array includes a task ID and a task sequence number of a task to be processed.

The task ID is an ID that uniquely identifies the task to be processed. For example, the task ID may be T0, T1, T2, and so on.

The task sequence number indicates the order of the tasks to be processed in the task set, and the task sequence number corresponds to the task ID one-to-one, for example, (T0 1) (T1 2) Distributed multitasking management

The target array indicates the array of the corresponding relationship between the task number and the task ID, for example, {(T0 1)(T1 ) 2) Distributed multi-task management}, when the server reads the target array, it can intelligently allocate the tasks to be processed to the execution nodes to achieve even distribution.

S203: Obtain the number of execution nodes corresponding to the idle execution nodes, and determine the target execution node corresponding to each task to be processed according to the task sequence number and the number of execution nodes.

The number of execution nodes refers to the number of idle execution nodes, so that the idle execution nodes are subsequently used to process the task to be processed, and node resources are allocated reasonably.

Specifically, the server monitors all execution nodes to obtain the number of execution nodes corresponding to the idle execution nodes. Each idle execution node carries a node sequence number. According to the remainder of dividing the task sequence number by the number of execution nodes, the corresponding number of execution nodes is searched according to the remainder. The node sequence number, and the idle execution node corresponding to the node sequence number is used as the target execution node of the pending task corresponding to the task sequence number, and the pending tasks are evenly distributed among the idle execution nodes. The processing logic is simple and efficient, and the execution speed of the pending tasks is accelerated. . Solve the problem of uneven distribution in the lock mechanism, resulting in excessive load on some execution nodes and small load on some execution nodes.

For example, the pending tasks are T0, T1, T2, T3, T4 and T5, and the target array is {(T0 1)(T1 2)(T2 3) (T3 4) (T4 5) (T5 6)}, the number of execution nodes is 2, including execution node 0 and execution node 1, then the remainder of the task sequence number 1 divided by the execution node number 2 is 1. Therefore, the pending task corresponding to the task sequence number 1 is assigned to Execute node 1 to process; the remainder of task number 2 divided by the number of execution nodes 2 is 0. Therefore, the task to be processed corresponding to task number 2 is allocated to execution node 0 for processing, and distributed multitasking management. In order to evenly distribute the tasks to be processed among idle execution nodes, the processing logic is simple and efficient.

S204: Invoke threads corresponding to the number of execution nodes, execute at least two tasks to be processed, and obtain execution results corresponding to the at least two tasks to be processed.

The execution result refers to the result obtained after processing the task to be processed.

Specifically, the tasks to be processed on each target execution node are sorted according to the task priority, the task priority order of all the tasks to be processed on each target execution node is obtained, the threads corresponding to the number of execution nodes are called, and the tasks are prioritized according to the task priority. All tasks to be processed are executed sequentially to realize parallel processing of tasks to be processed, so as to improve the execution efficiency of tasks to be processed. Among them, the task priority order indicates the execution order of the tasks to be processed. If the priority is high, it will be processed first; if the priority is low, it will be processed slowly to ensure that the tasks to be processed are processed in a timely manner.

The distributed multi-task management method provided by this embodiment obtains a task execution request, traverses the task set created on the distributed middleware based on the task execution request, and provides technical support for the subsequent realization of the equal distribution of tasks to be processed. Obtain at least two target arrays in the task set. Each target array includes the task ID and task sequence number of the task to be processed. When the server reads the target array, it can intelligently assign the to-be-processed task to the execution node. achieve an even distribution. Obtain the number of execution nodes corresponding to the idle execution nodes, determine the target execution node corresponding to each pending task according to the task sequence number and the number of execution nodes, and evenly distribute the pending tasks to the idle execution nodes. The processing logic is simple and efficient, speeding up the waiting period. The execution speed of the processing task. Solve the problem of uneven distribution in the lock mechanism, resulting in excessive load on some execution nodes and small load on some execution nodes. A thread corresponding to the number of execution nodes is called, at least two tasks to be processed are executed, and execution results corresponding to the at least two tasks to be processed are obtained, so as to realize the processing of the tasks to be processed.

In one embodiment, as shown in FIG. 3 , before step S201, that is, before obtaining a task execution request and traversing the task set created on the distributed middleware based on the task execution request, the distributed multitasking management method includes:

S301 : Acquire at least two task requests to be assigned, and each task request to be assigned includes a task ID and a task sequence number of the task to be processed.

A request for a task to be assigned refers to a request for assigning a task to be processed to a task set. It can be understood that when there are multiple tasks to be processed, multiple requests for tasks to be allocated are obtained, so as to realize the processing of the multiple tasks to be processed, and to ensure that the pending tasks are processed in time.

S302: Obtain a target array based on the task ID and task sequence number corresponding to each task to be processed.

In this embodiment, a target array corresponding to a to-be-processed task is acquired according to the task ID and the task sequence number of the to-be-processed task.

S303: Insert the target array into the original collection created on the distributed middleware to form a task collection.

Among them, the original collection refers to the set collection of redis, and the original collection is an empty collection. Create an original set on the distributed middleware, insert the target array formed based on the task ID and task sequence number into the original set to form a task set, and provide technical support for the subsequent even distribution of pending tasks.

In this embodiment, when the original collection is created on the distributed middleware, and the target array is inserted into the original collection created on the distributed middleware, the type identifier of the task to be processed is obtained to determine the task type of the task to be processed, and the same The task numbers of the pending tasks of the task type are linked together. It should be noted that the pending tasks of the same task type refer to the tasks with the same execution logic. Therefore, the execution times of the pending tasks of the same task type are similar. The remainder of dividing the task number of the task to be processed by the number of execution nodes allocates the task to be processed to the execution node corresponding to the node number corresponding to the remainder for processing. Therefore, the task numbers of the tasks to be processed of the same task type are linked together to ensure that Each subsequent execution node can be assigned to a similar number of pending tasks of the same task type to ensure that each target execution node processes the pending tasks for a similar length of time, making full use of the node resources of the distributed environment to ensure that the target execution node can be processed in time. Complete pending tasks. For example, for task type 1, there are 3 tasks to be processed, the task numbers of the tasks to be processed are 1, 2, and 3, the number of tasks to be processed for task type 2 is 5, and the task numbers of the tasks to be processed are 4, 5, and 6. , 7 and 8, there are 2 execution nodes. Since the task numbers of the pending tasks of the same task type are connected together, the pending tasks of execution node 1 are 1, 3, 5 and 7, and the pending tasks of execution node 0 are The tasks are 2, 4, 6, and 8. It can be seen that the execution nodes process the pending tasks for a similar length of time, making full use of the node resources of the distributed environment.

The distributed multi-task management method provided by this embodiment acquires at least two task requests to be allocated, and each task request to be allocated includes a task ID and a task sequence number of the tasks to be processed, so as to realize the processing of multiple tasks to be processed, And it can be guaranteed to be processed in a timely manner. Based on the task ID and task sequence number corresponding to each task to be processed, the target array is obtained; the target array is inserted into the original collection created on the distributed middleware to form a task collection, and the target array formed by the task ID and task sequence number is inserted into the original collection to form A collection of tasks to provide technical support for subsequent equal distribution of pending tasks.

In one embodiment, as shown in FIG. 4 , step S301, namely acquiring at least two task requests to be assigned, includes:

S401: When the current time of the system reaches a preset time, traverse the to-be-processed queue, and obtain task IDs and traversal orders corresponding to at least two to-be-processed tasks.

Wherein, the current system time is the current time of the system. The preset time is a preset time. When the current time of the system reaches the preset time, it will automatically traverse the pending queue, so that the pending tasks can be processed subsequently, and the pending tasks can be automatically processed.

The pending queue refers to a queue that stores pending tasks so that the server can determine the number of pending tasks that need to be processed at a preset time.

The traversal order is the order in which the tasks in the queue to be processed are traversed. The traversal order can be from the beginning of the queue to the end of the queue, or from the end of the queue to the beginning of the queue, which is not limited here.

In this embodiment, a timing trigger mechanism is adopted, and when the current time of the system reaches the preset time, the traversal of the pending queue is triggered to determine the number of pending tasks to be processed and the task ID corresponding to the pending tasks.

S402: Acquire a task sequence number corresponding to the to-be-processed task according to the traversal sequence corresponding to the to-be-processed task.

In this embodiment, when the server traverses a task to be processed, it marks the task to be processed to form a task sequence number of the task to be processed, so that the tasks to be processed can be equally distributed to idle execution nodes according to the task sequence number.

S403: Generate a task request to be allocated according to the task ID and task sequence number corresponding to the task to be processed.

In this embodiment, the to-be-allocated task request is automatically generated according to the to-be-processed task and the task ID corresponding to the to-be-processed task, so as to realize the automatic processing of the to-be-processed task and improve the intelligent processing efficiency.

In the distributed multi-task management method provided by this embodiment, when the current time of the system reaches the preset time, the queue to be processed is traversed, and the task IDs and the traversal order corresponding to at least two tasks to be processed are obtained; Traverse the sequence to obtain the task sequence number corresponding to the task to be processed, so that the tasks to be processed can be equally distributed to the idle execution nodes according to the task sequence number subsequently. According to the task ID and task sequence number corresponding to the to-be-processed task, a to-be-allocated task request is generated, so as to realize the automatic processing of the to-be-processed task and improve the intelligent processing efficiency.

In one embodiment, as shown in FIG. 5, the target execution node includes a node sequence number. Step S203, namely determining the target execution node corresponding to each task to be processed according to the task sequence number and the number of execution nodes, including:

S501: The remainder is obtained by dividing the task sequence number of the task to be processed by the number of execution nodes.

S502: Determine the idle execution node corresponding to the node sequence number with the same remainder as the target execution node of the to-be-processed task corresponding to the task sequence number.

In this embodiment, the idle execution node corresponding to the node sequence number with the same remainder is determined as the target execution node corresponding to the task sequence number corresponding to the pending task, so that the pending tasks are evenly distributed to the target execution nodes for execution, and the redis lock mechanism is eliminated. , to improve the efficiency of program operation.

For example, if the number of execution nodes is 2, including execution node 0 and execution node 1, and the task sequence number is 1-6, the remainder of the task sequence number 1 divided by the execution node number 2 is 1. Therefore, the task to be processed corresponding to the task sequence number 1 is allocated. Process the execution node 1; the remainder of the task sequence number 2 divided by the execution node number 2 is 0. Therefore, the to-be-processed task corresponding to the task sequence number 2 is allocated to the execution node 0 for processing.

In the distributed multi-task management method provided by this embodiment, the remainder is obtained by dividing the task sequence number of the task to be processed by the number of executing nodes. Determine the idle execution node corresponding to the node sequence number equal to the remainder as the target execution node of the pending task corresponding to the task sequence number, so as to realize the even distribution of the pending tasks to the target execution node, eliminate the redis lock mechanism, and improve the program operation. efficiency.

In one embodiment, as shown in FIG. 6 , the task to be processed further includes a task order number. Step S204, that is, executing at least two tasks to be processed, and obtaining execution results corresponding to the at least two tasks to be processed, including:

S601: According to the task order number, determine whether there is an execution dependency relationship between at least two tasks to be processed.

The task order number is a pre-defined order number by the user, and the task order number indicates whether different tasks to be processed have execution dependencies, the number of pending tasks with execution dependencies, and the pending tasks with execution dependencies. In this embodiment, the task order number is analyzed according to a preset rule to obtain the meaning of the order number corresponding to the task order number, so as to determine whether there is an execution dependency relationship between at least two tasks to be processed according to the meaning of the order number. Automation processes pending tasks for which there are execution dependencies. Among them, the preset rule is the rule of the meaning of the order corresponding to the preset task order number. For example, the first digit of the task order number of the task to be processed indicates whether the to-be-processed task has an execution dependency, and 1 indicates that there is an execution dependency. Dependency, 0 indicates that there is no execution dependency; the second digit indicates the pending tasks under the same execution dependency that need to be processed at the same time; the third digit indicates the execution order of the pending tasks with execution dependencies. For example, the task order numbers are 011, 111, 112, 122, and 123. At this time, 011 does not have an execution dependency, and the pending tasks corresponding to the four task order numbers 111, 112, 122, and 123 all have execution dependencies. Since the second digits of 111 and 112 are the same, 111 and 112 have the same execution dependency, and 111 in 111 and 112 is smaller than 112. Therefore, the pending task with the task order number of 111 is executed first (that is, the forward task) , the pending task with the task order number of 112 is executed (ie, the backward task). Since the second digits of 122 and 123 are the same, 122 and 123 have the same execution dependency, and 122 in 122 and 123 is smaller than 123. Therefore, the pending task with the task order number 122 is executed first, and the task order number is 123 Executed after the pending task.

As an example, one pending task is an interview order task, and the other pending task is an interview record task, wherein the interview order task is an order task created when a user applies for a loan, and the interview record task is used to record an interview The task of the execution result of the order task.

Execution dependencies are relationships that indicate the order in which tasks to be processed are executed.

S602: If at least two tasks to be processed have execution dependencies, determine at least two tasks to be processed as forward tasks and backward tasks according to the execution dependencies.

Among them, the forward task refers to the pending task that must be executed first; the backward task is the task that is completed after the execution of the forward task and is processed according to the execution result of the forward task. It can be understood that the number of backward tasks can be One, or multiple, when the execution of the forward task is completed, the backward task that depends on the forward task is updated to the forward task. For example, for the pending tasks A, B and C with execution dependencies, when A When not executed, B and C are backward tasks; when A is executed, B is forward task, and C is backward task.

Specifically, when the server obtains the task order number of the task to be processed, it analyzes the task order number of the task to be processed according to the preset rule, so as to determine the execution sequence of the to-be-processed tasks with execution dependencies. The pending tasks of execution dependencies are determined as forward tasks and backward tasks to ensure that pending tasks are executed in sequence, improve the processing efficiency of pending tasks, and avoid repeated execution of pending tasks due to unclear sequence of pending tasks. .

S603: Execute the forward task and the backward task in sequence, and obtain the execution result.

In this embodiment, the sequential execution of the forward task and the backward task can ensure accurate execution of the to-be-processed task and speed up the completion of the to-be-processed task.

The distributed multi-task management method provided by this embodiment determines whether any two tasks to be processed have an execution dependency according to the task order number, so as to realize automatic processing of the pending tasks with execution dependencies. If there is an execution dependency between the two pending tasks, the two pending tasks are determined as a forward task and a backward task according to the execution dependency, so as to ensure that the pending tasks are executed in sequence, improve the processing efficiency of the pending tasks, and avoid Due to the unclear sequence of the pending tasks, the pending tasks are repeatedly executed. Executing the forward task and the backward task in sequence and obtaining the execution result can ensure the accurate execution of the pending tasks and speed up the completion of the pending tasks.

In one embodiment, as shown in FIG. 7 , step S603, that is, executing the forward task and the backward task in sequence, and obtaining the execution result, includes:

S701: Determine whether the forward task and the backward task are on the same target execution node.

Specifically, when a forward task and a backward task with execution dependencies are obtained, check the task sequence numbers of the forward task and the backward task, so as to quickly determine the target execution node corresponding to the forward task and the backward task, so as to Determine whether the target execution nodes corresponding to the forward task and the backward task are the same target execution node, and perform corresponding processing according to different situations to ensure that the pending tasks with execution dependencies are processed in sequence.

S702: If the forward task and the backward task are on the same target execution node, execute the forward task and the backward task in sequence, and obtain an execution result.

If the current task and the backward task are on the same target execution node, the forward task and the backward task are executed in sequence, and the execution result is obtained to ensure that the pending task is executed according to the execution dependency. The current task and the backward task are executed at the same target node. At this time, the forward task is executed first, and then the backward task is executed.

S703: If the forward task and the backward task are not on the same target execution node, a blocking queue is established between the target execution node corresponding to the forward task and the target execution node corresponding to the backward task.

Among them, the blocking queue is a queue used to store the execution result of the forward task, realizes the communication between the forward task and the backward task, and ensures that the forward task and the backward task that are not in the same target execution node can be executed according to the forward direction first. Tasks are executed in the order of the backward tasks.

Specifically, when the forward task and the backward task are not in the same target execution node, in order to ensure that the pending task is completed at one time, a blocking queue is created between the target execution node corresponding to the forward task and the target execution node corresponding to the backward task. , to ensure the communication between the forward task and the backward task, and provide support for the subsequent execution of the forward task and the backward task in sequence.

S704: Execute the forward task, obtain the execution result corresponding to the forward task, and send the execution result corresponding to the forward task to a blocking queue, where the blocking queue is used to send the execution result to the target execution node corresponding to the backward task The execution instruction including the execution result corresponding to the forward task.

The execution instruction is an instruction for triggering the execution of the backward task.

Specifically, the forward task is executed first, the execution result corresponding to the forward task is obtained, and the execution result of the forward task is sent to the blocking queue, so as to execute the backward task according to the execution result, thereby ensuring the processing efficiency of the pending task. Understandably, if the current task is not completed and there is no execution result, the execution instruction is not triggered, but an exception reminder message is generated, which ensures the processing efficiency of the pending task and avoids the failure of the backward task execution process and waste of resources.

S705: Execute the backward task according to the execution instruction, and obtain an execution result corresponding to the backward task.

The distributed multi-task management method provided by this embodiment determines whether the forward task and the backward task are on the same target execution node, so as to perform corresponding processing according to different situations, so as to ensure that the pending tasks with execution dependencies are processed in sequence. . If the forward task and the backward task are on the same target execution node, execute the forward task and the backward task in sequence, obtain the execution result, and ensure that the pending task is executed according to the execution dependency. If the forward task and the backward task are not on the same target execution node, a blocking queue is established between the target execution node corresponding to the forward task and the target execution node corresponding to the backward task to ensure the communication between the forward task and the backward task Contact to provide support for subsequent sequential execution of forward and backward tasks. Execute the forward task, obtain the execution result corresponding to the forward task, and send the execution result corresponding to the forward task to the blocking queue, wherein the blocking queue is used to send the target execution node corresponding to the backward task, including all The execution instruction of the execution result corresponding to the forward task ensures the processing efficiency of the to-be-processed task. The backward task is executed according to the execution instruction, and the execution result corresponding to the backward task is obtained.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

In one embodiment, a distributed multitasking management apparatus is provided, and the distributed multitasking management apparatus corresponds one-to-one with the distributed multitasking management method in the above embodiment. As shown in FIG. 8 , the distributed multi-task management apparatus includes a task set traversal module 801 , a target array acquisition module 802 , a target execution node determination module 803 and an execution result acquisition module 804 . The detailed description of each functional module is as follows:

The task set traversing module 801 is configured to obtain a task execution request, and traverse the task set created on the distributed middleware based on the task execution request.

A target array obtaining module 802, configured to obtain at least two target arrays in the task set, each of the target arrays includes a task ID and a task sequence number of a task to be processed.

The target execution node determination module 803 is configured to acquire the number of execution nodes corresponding to the idle execution nodes, and determine the target execution node corresponding to each of the tasks to be processed according to the task sequence number and the number of execution nodes.

The execution result obtaining module 804 is configured to call threads corresponding to the number of execution nodes, execute at least two tasks to be processed, and obtain execution results corresponding to at least two tasks to be processed.

In an embodiment, before the task set traversing module 801, the distributed multi-task management apparatus further includes: a to-be-assigned task request acquisition unit, a target array acquisition unit, and a task set formation unit.

The to-be-allocated task request acquisition unit is configured to acquire at least two to-be-allocated task requests, each of the to-be-allocated task requests includes a task ID and a task sequence number of the to-be-processed task.

A target array obtaining unit, configured to obtain a target array based on the task ID and the task sequence number corresponding to each task to be processed.

The task set forming unit is used for inserting the target array into the original set created on the distributed middleware to form a task set.

In one embodiment, the to-be-assigned task request acquisition unit includes: a to-be-processed queue traversal subunit, a task sequence number acquisition subunit, and a to-be-allocated task request generation subunit.

The to-be-processed queue traversal subunit is used to traverse the to-be-processed queue when the current time of the system reaches a preset time, and obtain the task IDs and the traversal order corresponding to at least two of the to-be-processed tasks.

The task sequence number obtaining subunit is configured to obtain the task sequence number corresponding to the to-be-processed task according to the traversal sequence corresponding to the to-be-processed task.

The to-be-allocated task request generating subunit is configured to generate a to-be-allocated task request according to the task ID corresponding to the to-be-processed task and the task sequence number.

In an embodiment, the target execution node includes a node sequence number; the target execution node determination module 803 includes: a remainder obtaining unit and a target execution node determination unit.

A remainder obtaining unit, configured to divide the task sequence number of the to-be-processed task by the number of execution nodes to obtain a remainder.

The target execution node determination unit determines the idle execution node corresponding to the node sequence number with the same remainder as the target execution node of the to-be-processed task corresponding to the task sequence number.

In an embodiment, the task to be processed further includes a task order number; that is, the execution result obtaining module 804 includes: a judgment unit, an execution dependency relationship determination unit, and an execution result obtaining unit.

A judging unit, configured to judge whether there is an execution dependency relationship between at least two of the to-be-processed tasks according to the task order number.

An execution dependency determination unit, configured to determine at least two of the to-be-processed tasks as a forward task and a backward task according to the execution dependency if there is an execution dependency of the at least two to-be-processed tasks.

The execution result obtaining unit is configured to execute the forward task and the backward task in sequence, and obtain the execution result.

In one embodiment, the execution result obtaining unit includes: a judgment subunit, a first execution subunit, a blocking queue establishment subunit, a forward task execution subunit, and a second execution subunit.

A judging subunit, configured to judge whether the forward task and the backward task are on the same target execution node.

The first execution subunit is configured to execute the sequential execution of the forward task and the backward task if the forward task and the backward task are on the same target execution node, and obtain an execution result.

Blocking queue establishment subunit, for if the forward task and the backward task are not on the same target execution node, then the target execution node corresponding to the forward task and the target execution node corresponding to the backward task A blocking queue is established between.

The forward task execution subunit is used to execute the forward task, obtain the execution result corresponding to the forward task, and send the execution result corresponding to the forward task to the blocking queue, wherein the blocking queue uses and sending an execution instruction including the execution result corresponding to the forward task to the target execution node corresponding to the backward task.

The second execution subunit is configured to execute the backward task according to the execution instruction, and obtain an execution result corresponding to the backward task.

For specific limitations on the distributed multitasking management apparatus, reference may be made to the foregoing limitations on the distributed multitasking management method, which will not be repeated here. Each module in the above-mentioned distributed multitasking management apparatus may be implemented in whole or in part by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In one embodiment, a computer device is provided, and the computer device may be a server, and its internal structure diagram may be as shown in FIG. 9 . The computer device includes a processor, memory, a network interface, and a database connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions and a database. The internal memory provides an environment for the execution of the operating system and computer-readable instructions in the non-volatile storage medium. The database of the computer device is used to store pending tasks. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer-readable instructions, when executed by a processor, implement a distributed multitasking management method.

In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, and the processor implements the distribution in the above embodiment when the processor executes the computer-readable instructions The steps of the multitasking management method, such as steps S201-S204 shown in FIG. 2 , or steps shown in FIG. 3 to FIG. 7 , are not repeated here to avoid repetition. Alternatively, when the processor executes the computer-readable instructions, the functions of each module/unit in this embodiment of the distributed multitasking management apparatus are implemented, for example, the task set traversal module 801, the target array acquisition module 802, the target execution module shown in FIG. 8 The functions of the node determination module 803 and the execution result acquisition module 804 are not repeated here to avoid repetition.

In one embodiment, one or more readable storage media storing computer readable instructions are provided, the computer readable storage media having computer readable instructions stored thereon, the computer readable instructions being stored by one or more processors. During execution, the one or more processors implement the steps of the distributed multitasking management method in the foregoing embodiment, such as steps S201-S204 shown in FIG. 2 , or steps shown in FIG. 3 to FIG. 7 . , in order to avoid repetition, it will not be repeated here. Alternatively, when the processor executes the computer-readable instructions, the functions of each module/unit in this embodiment of the distributed multitasking management apparatus are implemented, for example, the task set traversal module 801, the target array acquisition module 802, the target execution module shown in FIG. 8 The functions of the node determination module 803 and the execution result acquisition module 804 are not repeated here to avoid repetition. The readable storage medium in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile computer. In the readable storage medium, the computer-readable instructions, when executed, may include the processes of the foregoing method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM) and so on.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated to different functional units, Module completion, that is, dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above-mentioned embodiments, those of ordinary skill in the art should understand that: it is still possible to implement the above-mentioned implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the application, and should be included in the within the scope of protection of this application. 

Claims (20)

1. A distributed multitasking management method, comprising:

   Obtain a task execution request, and traverse the task collection created on the distributed middleware based on the task execution request;

   Obtain at least two target arrays in the task set, each of which includes the task ID and the task sequence number of the task to be processed;

   Obtain the number of execution nodes corresponding to the idle execution nodes, and determine the target execution node corresponding to each of the tasks to be processed according to the task sequence number and the number of execution nodes;

   A thread corresponding to the number of the execution nodes is called to execute at least two of the tasks to be processed, and execution results corresponding to the at least two tasks to be processed are obtained.
2. The distributed multi-task management method according to claim 1, wherein before the task execution request is acquired and the task set created on the distributed middleware is traversed based on the task execution request, the distributed multi-task management Methods include:

   Obtain at least two task requests to be assigned, and each described task request to be assigned includes a task ID and a task sequence number of the task to be processed;

   Obtain a target array based on the task ID and the task sequence number corresponding to each of the tasks to be processed;

   Insert the target array into the original collection created on the distributed middleware to form a task collection.
3. The distributed multitasking management method according to claim 2, wherein the acquiring at least two task requests to be assigned comprises:

   When the current time of the system reaches the preset time, it traverses the queue to be processed, and obtains the task ID and the traversal order corresponding to at least two of the tasks to be processed;

   According to the traversal sequence corresponding to the to-be-processed task, obtain the task sequence number corresponding to the to-be-processed task;

   A task request to be allocated is generated according to the task ID corresponding to the task to be processed and the task sequence number.
4. The distributed multitasking management method of claim 1, wherein the target execution node includes a node sequence number;

   The determining the target execution node corresponding to each task to be processed according to the task sequence number and the number of execution nodes includes:

   The remainder is obtained by dividing the task sequence number of the task to be processed by the number of execution nodes;

   Determine the idle execution node corresponding to the node sequence number equal to the remainder as the target execution node of the to-be-processed task corresponding to the task sequence number.
5. The distributed multi-task management method according to claim 1, wherein the to-be-processed task further comprises a task order number;

   The executing at least two of the tasks to be processed, and obtaining the execution results corresponding to the at least two tasks to be processed, includes:

   According to the task order number, determine whether at least two of the to-be-processed tasks have an execution dependency;

   If at least two of the to-be-processed tasks have execution dependencies, determining at least two of the to-be-processed tasks as forward tasks and backward tasks according to the execution dependencies;

   Execute the forward task and the backward task in sequence to obtain an execution result.
6. The distributed multi-task management method according to claim 5, wherein the executing the forward task and the backward task in sequence, and obtaining the execution result, comprises:

   Determine whether the forward task and the backward task are on the same target execution node;

   If the forward task and the backward task are on the same target execution node, execute the sequential execution of the forward task and the backward task, and obtain an execution result;

   If the forward task and the backward task are not on the same target execution node, a blocking queue is established between the target execution node corresponding to the forward task and the target execution node corresponding to the backward task;

   Execute the forward task, obtain the execution result corresponding to the forward task, and send the execution result corresponding to the forward task to the blocking queue, where the blocking queue is used to send the corresponding execution result to the backward task. The target execution node sends an execution instruction including an execution result corresponding to the forward task;

   The backward task is executed according to the execution instruction, and an execution result corresponding to the backward task is acquired.
7. A distributed multitasking management device, comprising:

   a task set traversal module, configured to obtain a task execution request, and traverse the task set created on the distributed middleware based on the task execution request;

   A target array acquisition module, used to obtain at least two target arrays in the task set, each of the target arrays including the task ID and the task sequence number of the task to be processed;

   a target execution node determination module, configured to obtain the number of execution nodes corresponding to the idle execution nodes, and determine the target execution node corresponding to each of the to-be-processed tasks according to the task sequence number and the number of execution nodes;

   The execution result obtaining module is configured to call threads corresponding to the number of execution nodes, execute at least two tasks to be processed, and obtain execution results corresponding to at least two tasks to be processed.
8. The distributed multitasking management device according to claim 7, wherein the distributed multitasking management device further comprises:

   A task request acquisition unit to be assigned, for acquiring at least two task requests to be assigned, each of the task requests to be assigned includes a task ID and a task sequence number of the task to be processed;

   a target array obtaining unit, used for obtaining a target array based on the task ID and the task sequence number corresponding to each of the tasks to be processed;

   The task set forming unit is used for inserting the target array into the original set created on the distributed middleware to form a task set.
9. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer-readable instructions:

   Obtain a task execution request, and traverse the task collection created on the distributed middleware based on the task execution request;

   Obtain at least two target arrays in the task set, each of which includes the task ID and the task sequence number of the task to be processed;

   Obtain the number of execution nodes corresponding to the idle execution nodes, and determine the target execution node corresponding to each of the tasks to be processed according to the task sequence number and the number of execution nodes;

   A thread corresponding to the number of the execution nodes is called to execute at least two of the tasks to be processed, and execution results corresponding to the at least two tasks to be processed are obtained.
10. 10. The computer device of claim 9, wherein the processor executes the computer-readable instructions prior to the obtaining a task execution request and traversing a set of tasks created on the distributed middleware based on the task execution request Also implement the following steps:

    Obtain at least two task requests to be assigned, and each described task request to be assigned includes a task ID and a task sequence number of the task to be processed;

    Obtain a target array based on the task ID and the task sequence number corresponding to each of the tasks to be processed;

    Insert the target array into the original collection created on the distributed middleware to form a task collection.
11. The computer device of claim 10, wherein the acquiring at least two task requests to be assigned comprises:

    When the current time of the system reaches the preset time, it traverses the queue to be processed, and obtains the task ID and the traversal order corresponding to at least two of the tasks to be processed;

    According to the traversal sequence corresponding to the to-be-processed task, obtain the task sequence number corresponding to the to-be-processed task;

    A task request to be allocated is generated according to the task ID corresponding to the task to be processed and the task sequence number.
12. The computer device of claim 9, wherein the target execution node includes a node sequence number;

    The determining the target execution node corresponding to each task to be processed according to the task sequence number and the number of execution nodes includes:

    The remainder is obtained by dividing the task sequence number of the task to be processed by the number of execution nodes;

    Determine the idle execution node corresponding to the node sequence number equal to the remainder as the target execution node of the to-be-processed task corresponding to the task sequence number.
13. The computer device of claim 9, wherein the task to be processed further comprises a task order number;

    The executing at least two of the tasks to be processed, and obtaining the execution results corresponding to the at least two tasks to be processed, includes:

    According to the task order number, determine whether at least two of the to-be-processed tasks have an execution dependency;

    If at least two of the to-be-processed tasks have execution dependencies, determining at least two of the to-be-processed tasks as forward tasks and backward tasks according to the execution dependencies;

    Execute the forward task and the backward task in sequence to obtain an execution result.
14. The computer device according to claim 13, wherein, executing the forward task and the backward task in sequence, and obtaining the execution result, comprises:

    Determine whether the forward task and the backward task are on the same target execution node;

    If the forward task and the backward task are on the same target execution node, execute the sequential execution of the forward task and the backward task, and obtain an execution result;

    If the forward task and the backward task are not on the same target execution node, a blocking queue is established between the target execution node corresponding to the forward task and the target execution node corresponding to the backward task;

    Execute the forward task, obtain the execution result corresponding to the forward task, and send the execution result corresponding to the forward task to the blocking queue, where the blocking queue is used to send the corresponding execution result to the backward task. The target execution node sends an execution instruction including an execution result corresponding to the forward task;

    The backward task is executed according to the execution instruction, and an execution result corresponding to the backward task is acquired.
15. One or more readable storage media having computer-readable instructions stored thereon, the computer-readable storage media having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by one or more processors, cause all The one or more processors perform the following steps:

    Obtain a task execution request, and traverse the task collection created on the distributed middleware based on the task execution request;

    Obtain at least two target arrays in the task set, each of which includes the task ID and the task sequence number of the task to be processed;

    Obtain the number of execution nodes corresponding to the idle execution nodes, and determine the target execution node corresponding to each of the tasks to be processed according to the task sequence number and the number of execution nodes;

    The threads corresponding to the number of execution nodes are called to execute at least two of the tasks to be processed, and execution results corresponding to the at least two tasks to be processed are obtained.
16. 16. The readable storage medium of claim 15, wherein the computer-readable instructions are executed by one or more prior to obtaining the task execution request and traversing the task set created on the distributed middleware based on the task execution request. When executed by multiple processors, the one or more processors are caused to further perform the following steps:

    Obtain at least two task requests to be assigned, and each described task request to be assigned includes a task ID and a task sequence number of the task to be processed;

    Obtain a target array based on the task ID and the task sequence number corresponding to each of the tasks to be processed;

    Insert the target array into the original collection created on the distributed middleware to form a task collection.
17. The readable storage medium of claim 16, wherein the acquiring at least two task requests to be assigned comprises:

    When the current time of the system reaches the preset time, the queue to be processed is traversed, and the task ID and the traversal order corresponding to at least two of the tasks to be processed are obtained;

    According to the traversal sequence corresponding to the to-be-processed task, obtain the task sequence number corresponding to the to-be-processed task;

    A task request to be allocated is generated according to the task ID corresponding to the task to be processed and the task sequence number.
18. The readable storage medium of claim 15, wherein the target execution node includes a node sequence number;

    The determining the target execution node corresponding to each task to be processed according to the task sequence number and the number of execution nodes includes:

    The remainder is obtained by dividing the task sequence number of the task to be processed by the number of execution nodes;

    Determine the idle execution node corresponding to the node sequence number equal to the remainder as the target execution node of the to-be-processed task corresponding to the task sequence number.
19. The readable storage medium of claim 15, wherein the task to be processed further comprises a task ticket number;

    The executing at least two tasks to be processed and obtaining the execution results corresponding to the at least two tasks to be processed include:

    According to the task order number, determine whether at least two of the to-be-processed tasks have an execution dependency;

    If at least two of the to-be-processed tasks have execution dependencies, determining at least two of the to-be-processed tasks as forward tasks and backward tasks according to the execution dependencies;

    Execute the forward task and the backward task in sequence to obtain an execution result.
20. The readable storage medium according to claim 19, wherein the executing the forward task and the backward task in sequence, and obtaining the execution result, comprises:

    Determine whether the forward task and the backward task are on the same target execution node;

    If the forward task and the backward task are on the same target execution node, execute the sequential execution of the forward task and the backward task, and obtain an execution result;

    If the forward task and the backward task are not on the same target execution node, a blocking queue is established between the target execution node corresponding to the forward task and the target execution node corresponding to the backward task;

    Execute the forward task, obtain the execution result corresponding to the forward task, and send the execution result corresponding to the forward task to the blocking queue, wherein the blocking queue is used to send the corresponding execution result to the backward task. The target execution node sends an execution instruction including an execution result corresponding to the forward task;

    The backward task is executed according to the execution instruction, and an execution result corresponding to the backward task is acquired.