CN108319495B

CN108319495B - Task processing method and device

Info

Publication number: CN108319495B
Application number: CN201710029055.9A
Authority: CN
Inventors: 唐家勇
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2017-01-16
Filing date: 2017-01-16
Publication date: 2023-01-31
Anticipated expiration: 2037-01-16
Also published as: CN108319495A

Abstract

The embodiment of the application provides a task processing method and device. The method comprises the following steps: decomposing a parent task to be processed into one or more independent subtasks; adding the one or more independent subtasks obtained through decomposition into a task queue of a thread pool; sequentially taking out each subtask from the task queue by using the working thread in the thread pool and executing the subtask, obtaining an execution result corresponding to each subtask and adding the execution result into a result list corresponding to the parent task; wherein the execution result contains exception information when the subtask execution is abnormal; and when at least one execution result containing the abnormal information exists in the result list corresponding to the parent task, feeding the abnormal information existing in the result list back to the user side. According to the method and the device, the user can determine the abnormal reason according to the abnormal information.

Description

Task processing method and device

Technical Field

The present application relates to the field of computer technologies, and in particular, to a task processing method and apparatus.

Background

To fully utilize computing resources and reduce processing time of tasks, task processing may typically take a "parallel-synchronous" mode. In the parallel-synchronous mode, a parent task can be decomposed into a plurality of independent subtasks, the decomposed subtasks can be executed in parallel in a plurality of threads, and finally the execution result of each subtask is merged to obtain the result of the parent task.

In the related art, each subtask decomposed by a parent task is added to a thread pool containing a plurality of working threads for execution, and an exception is inevitable in the process of executing each subtask in parallel.

Disclosure of Invention

In view of this, the present application provides a task processing method and apparatus.

In order to achieve the above purpose, the present application provides the following technical solutions:

according to a first aspect of the present application, a task processing method is provided, including:

decomposing a parent task to be processed into one or more independent subtasks;

adding the one or more independent subtasks obtained by decomposition into a task queue of a thread pool;

sequentially taking out each subtask from the task queue by using the working threads in the thread pool and executing the subtasks, obtaining an execution result corresponding to each subtask and adding the execution result to a result list corresponding to the parent task; wherein the execution result contains exception information when the subtask execution is abnormal;

and when at least one execution result containing the abnormal information exists in the result list corresponding to the parent task, feeding the abnormal information existing in the result list back to the user side.

According to a second aspect of the present application, there is provided a task processing apparatus including:

the decomposition unit is used for decomposing the parent task to be processed into one or more independent subtasks;

a first adding unit, configured to add the one or more decomposed sub-tasks that are independent of each other to a task queue of a thread pool;

the execution unit is used for taking out each subtask from the task queue in sequence by using the working threads in the thread pool and executing the subtask, obtaining an execution result corresponding to each subtask and adding the execution result into a result list corresponding to the parent task; wherein the execution result contains exception information when the subtask execution is abnormal;

and the exception feedback unit is used for feeding back the exception information existing in the result list to the user side when at least one execution result containing the exception information exists in the result list corresponding to the parent task.

According to the technical scheme, each subtask obtained by decomposing the parent task is added into the task queue of the thread pool, each subtask is executed by utilizing the working thread in the thread pool, and the execution result corresponding to each subtask is obtained and added into the result list corresponding to the parent task. And when at least one execution result containing the abnormal information exists in a result list corresponding to the parent task, indicating that the execution process of the parent task is abnormal, and feeding the abnormal information existing in the result list back to the user side. In the process, when the task execution process is abnormal, the abnormal information corresponding to the subtasks can be fed back to the user side, so that the task manager can determine the abnormal reason according to the abnormal information, and the use experience of the task manager is improved.

Drawings

FIG. 1 is a schematic diagram of a system architecture for implementing task execution and task management in one embodiment of the present application;

FIG. 2 is a block diagram of a parallel-synchronous task scheduler in an embodiment of the present application;

FIG. 3 is a diagram illustrating a data structure corresponding to a parallel-synchronous task scheduler according to an embodiment of the present application;

FIG. 4 is a flow chart of a method of task processing in an embodiment of the present application;

FIG. 5 is a flow diagram of performing subtasks and obtaining results of the performance in one embodiment of the present application;

FIG. 6 is a flowchart illustrating saving of normal results to a normal results list upon normal completion of execution of subtasks in an embodiment of the present application;

FIG. 7 is a flowchart illustrating an exemplary embodiment of saving an exception result to an exception result list when an exception occurs in an execution subtask;

fig. 8 is a flowchart of waiting for the execution of the parent task to be completed and feeding back the execution result to the user side in an embodiment of the present application;

FIG. 9 is a flow diagram of bot task termination using a bot task cleanup thread in an embodiment of the present application;

fig. 10 and fig. 11 are flowcharts of a task query process provided by an embodiment of the present application;

FIG. 12 is a schematic diagram of an electronic device according to an exemplary embodiment of the present application;

FIG. 13 is a block diagram of a task processing device according to an embodiment of the present application;

FIG. 14 is a block diagram of a task processing device according to another embodiment of the present application;

FIG. 15 is a block diagram of a task processing device according to yet another embodiment of the present application;

fig. 16 is a block diagram of a task processing device according to still another embodiment of the present application.

Detailed Description

The Parallel-synchronous (Parallel-Sync) task processing mode is widely applied to various fields so as to fully utilize computing resources and improve the processing efficiency. The term "parallel" refers to that all the subtasks are executed in parallel, and the term "synchronous" refers to that all the subtasks are completed before merging of the results. For convenience of explanation, the "parent Task" is denoted by "Job" and the "child Task" is denoted by "Task". The "parallel-synchronous" mode essentially comprises the following steps:

(1) A Job is decomposed into several interdependent tasks.

(2) And each Task is executed in parallel in different threads, and the result is respectively calculated.

(3) The results of all tasks are merged into the execution results of Job.

Generally, step (1) and step (3) can be executed in "user threads", and step (2) can be executed in several "worker threads". Wherein the number of the working threads is not necessarily the same as the number of the tasks.

For example, assume that a Job is: calculating the highest price of a stock in three days of Monday, tuesday and Wednesday, a possible 'parallel-synchronous' execution mode comprises the following steps:

(1) And (3) decomposition: this Job is decomposed into 3 interdependent tasks:

task1: calculating the maximum value of the stock in Monday;

task2: calculating the maximum value of the stock on the Tuesday;

task3: the maximum value of the stock on wednesday is calculated.

(2) And (3) asynchronous execution: and 3 tasks are executed in parallel in different threads, and the results are respectively calculated.

(3) Merging: the maximum of 3 Task results was taken as the maximum price in three days.

The "parallel-sync" mode may be applied in a Web service scenario, where the "parallel-sync" mode may include the following steps:

(a) The main thread (or user thread) of the Web server receives the Job request from the user terminal.

(b) The main thread decomposes Job into several mutually independent tasks.

(c) The main thread submits all tasks to the worker thread scheduler.

(d) And the scheduler coordinates all the working threads and asynchronously executes Task.

(e) The main thread waits for all Task executions to complete.

(f) The main thread merges the results of all tasks into the Job's results.

(g) The main thread returns the Job's results to the user side.

Based on the basic concept of the parallel-synchronous mode, the embodiment of the application provides a technology which can manage the task execution process, can correctly cope with various exceptions in the task processing process, and can feed back various exception information like a user.

Fig. 1 is a schematic diagram of a system architecture for implementing task execution and task management in an embodiment of the present application. The system may include: the task management system comprises a first terminal device 20 used for a task initiator, a second terminal device 30 used for a task manager, and a server 10 which is respectively communicated with the first terminal device 20 and the second terminal device 30 and is used for realizing a parallel-synchronous task processing mode. Each Task initiator can initiate a parent Task (Job) request to the server, after receiving the Job request, the server 10 can decompose the Job according to a preset rule by using a server main thread (user thread) to obtain one or more independent tasks, and add at least two independent tasks to a Task queue of the thread pool, so that each Task is executed in parallel by using each working thread of the thread pool, and finally, a corresponding execution result is obtained and fed back to the Task initiator. In this scenario, the role of a task manager is further included, which is used to monitor and manage the task execution process of the server 10, and the task manager can check the state of each Job that has not been executed and completed on the server 10, cancel one or more jobs (for example, a Job consumes too much resources or consumes too much time) as needed, obtain exception information of each Job, and so on. It should be noted that, in some scenarios, the task initiator and the task manager may be the same person, that is, the task initiator may also have the right to manage the task, and the task manager may also have the right to initiate the task, which is not limited herein.

Fig. 2 is a schematic diagram of a parallel-synchronous task scheduler according to an embodiment of the present application. In this embodiment, the parallel-synchronous task scheduler may exist in the server, and may include: a Thread Pool (Task Thread Pool), a Task container (JobContainer), a zombie Task cleanup Thread (Dead JobClear Thread), and a zombie Task ID Queue (Dead Job ID Queue). The thread pool is a consumer thread pool for executing Task, and the number of threads in the thread pool can be set by a user or can be changed within a certain range according to requirements. The task container is used to store task data for all Jobs that are not currently executing and have not timed out. The zombie task cleaning thread is used for cleaning the zombie tasks corresponding to the IDs in the zombie task ID queue. Of course, in other embodiments, the parallel-synchronous task scheduler may further include, according to actual needs: the Scheduler's global Lock (Scheduler Lock), the current maximum Job ID (Max Job ID), a Lock to prevent concurrent access to the zombie task ID Queue (Dead Job ID Queue Lock), and so on.

Fig. 3 is a diagram illustrating a data structure corresponding to a parallel-synchronous task scheduler according to an embodiment of the present application. In the embodiment of the present application, according to the "object-oriented" design method, one data object may include two major types of structures: memory data structures (i.e., individual data fields) and interfaces (i.e., functions or methods that implement certain functions). As shown in fig. 3, all "Job" objects that have not been executed and timed out are stored in the task container, with the Job ID as an index.

Each "Job" object contains the following data members: (1) job ID: job's unique number; (2) subtask List (Task List): indexed by "Task ID". (3) Job Data Holder pointer: a pointer (or reference) to a data object that holds the parameter, result.

Each Task object includes the following data members: (1) the Task ID: may be the number of the Task in Job's "Task List". (2) And Future: and after the Task is submitted to the thread pool, returning objects for inquiring the Task state and controlling the Task thread. Since different programming languages and function libraries return different objects, they are collectively denoted by "Future". (3) The Task Executor pointer: a pointer to the interface (code block) that actually executes the Task. (4) Job Data Holder pointer: a pointer (or reference) to a data object that holds the parameter, result.

"Job Data Holder" corresponds to one Job, and is a Data object for storing parameters and execution results of the Job, and "Job Data Holder" includes the following Data members: (1) job ID; (2) task cancel Flag (Job Cancelled Flag): a boolean value that marks whether Job has been cancelled, the initial value may be "FALSE". (3) A Task Parameter List (Task Parameter List): job decomposes the generated parameter lists of all tasks, indexed by Task ID, in a one-to-one correspondence with each Task, which remains unchanged once generated. (4) Normal results List (Task Result List): a list of the results of normal execution of each Task is stored. (5) Exception result List (Task Exception List): and storing an abnormal information list when the Task execution is abnormal. (6) Data Lock (Data Lock): a lock to restrict concurrent access to the normal result list and the abnormal result list. (7) And Semaphore: the semaphore waiting for the end of Job execution (success or failure) synchronizes blocking while waiting for "grant". (8) And a task listener: a listener to notify the Scheduler that a Job's execution is complete (success or failure). In the embodiment of the present application, there are two types of data objects that need to be customized by a user (code developer): (1) input parameters of Task: and (2) outputting the result of the Task.

Based on the above, the following will describe the procedure of the task processing method provided by the embodiment of the present application.

Fig. 4 is a flowchart of a task processing method in an embodiment of the present application, which can be applied to a server, and includes the following steps 101 to 104, where:

in step 101, the user thread of the server decomposes the Job to be processed into one or more Task independent tasks.

In step 102, the user thread of the server adds the one or more independent tasks obtained by decomposition to a Task queue of a thread pool.

In step 103, the Task queues are sequentially taken out and executed by the working threads in the thread pool, and the execution result corresponding to each Task is obtained and added to the result list corresponding to the Job; wherein the execution result contains exception information when the Task executes an exception.

The Thread pool is a container having a task queue and a certain number of Work threads (Work threads), and the length of the task queue and the number of the threads can be changed according to requirements. The Task to be executed is firstly put into a Task queue to be executed, an idle working thread in a thread pool takes out one Task for execution according to a certain strategy (Task sequence or priority and the like) each time, and then takes out the next Task for execution after the execution of one Task is finished, and the steps are repeated. The thread pool technology can fully utilize multi-core computing resources and reduce the cost of frequent creation and destruction of threads. Meanwhile, the number of the working threads is limited, so that excessive occupation of CPU resources can be prevented.

In an embodiment of the present application, the Result List may include a normal Result List and an abnormal Result List, where the execution Result is added to an abnormal Result List (Task Exception List) when the Task execution is abnormal, and the execution Result is added to a normal Result List (Task Result List) when the Task execution does not find abnormality. When an exception occurs in the execution of a Task, exception information of the Task is generated, for example: the exception of the read-write data, the exception of the calculation logic, and the like, and the exception information is written into the specified position for storing the Task result in the exception result list. The exception result list and the normal result list are empty before the Task has not started execution. The working thread is responsible for actually executing the corresponding Task and capturing the exception occurring in the execution process. Examples of anomalies: the file accessed by the Task does not exist, or the divisor is 0 when the Task calculates the division. In one embodiment, the exception execution result corresponding to the Task ID may also include the input parameters of the Task, etc.

In an alternative embodiment, the results of the abnormal execution and the results of the normal execution may be added to the same result list, where each Task is assigned a storage location for storing the execution results, and the result list is empty when Job is not executing.

In step 104, when at least one execution result including the abnormal information exists in the result list corresponding to the Job, the abnormal information existing in the result list is fed back to the user side.

The user thread of the server can determine whether the execution process of a certain Job is abnormal by checking whether at least one execution result containing abnormal information exists in the result list of the Job, and when the execution process of the Job is abnormal, the abnormal information existing in the result list is fed back to a user side (namely, a terminal device used by a task manager) for the user to check. Wherein the abnormal Job is automatically terminated to avoid unnecessary consumption of computing resources.

Through the process, when the Task execution process is abnormal, the abnormal information corresponding to the Task can be fed back to the user side, so that a Task manager can determine the abnormal reason (such as the Task ID of the abnormal Task, the abnormal reason of the Task, the input parameter of the Task and the like) according to the abnormal information, and the use experience of the Task manager is improved.

In addition, in the related art, when the Task is abnormal in the execution process or the Job execution times out, a programmer needs to deeply participate, and a more complex processing code is written to deal with the problem. The method provided by the embodiment of the application can provide the interface for blocking waiting for the user, and can feed back abnormal information such as Job overtime, task execution abnormity and the like in the Task execution process to the user, so that programming of more complex processing codes by programmers can be avoided, the workload of the programmers is greatly reduced, and the efficiency is improved.

Fig. 5 is a flow of executing Task and obtaining an execution result in an embodiment of the present application, and corresponds to step 103 in fig. 4, in an embodiment, step 103 may specifically include:

step 131: and the working thread acquires a corresponding Task parameter from a Task Data set (Job Data Holder) according to the Task ID, wherein the Task parameter is an input value of one or more parameters of the Task.

Step 132: and the working thread executes the corresponding Task according to the Task parameter and obtains an execution result.

Step 133: and when the Task execution process is not abnormal, the working thread saves the obtained normal execution result to the corresponding position of the normal result list.

Step 134: and when the Task execution process of the working thread is abnormal, storing the obtained abnormal execution result to a corresponding position of the abnormal result list.

Fig. 6 is a flowchart of saving a normal execution result to a normal result list in an embodiment of the present application, and corresponding to the step 133, the method may include the following steps:

step 1331: and storing the normal execution result in a corresponding position in a normal result list according to the Task ID.

Step 1332: the number of normal results (denoted as NR) in the normal result list is obtained. In this embodiment, to avoid concurrent access to Data, a Data Lock (Data Lock) is locked before beginning at step 1332 and unlocked after completion of step 1332.

Step 1333: it is checked whether NR equals N (N is the number of tasks decomposed by Job). When NR = N, step 1334 is entered.

Step 1334: and according to the Job ID corresponding to the Task ID, removing corresponding Job data (comprising a Task parameter, the Job ID, a Task list, a stored execution result and the like) from the Task container. In this embodiment, to avoid concurrent access to data, the "Scheduler Lock" may be locked before the start of the step 1334 and unlocked after the completion of the step 1334.

Fig. 7 is a flowchart of saving an abnormal execution result to an abnormal result list according to an embodiment of the present application, and corresponding to step 134, the method may include the following steps:

step 1341: and saving the abnormal execution result in a corresponding position in the abnormal result list according to the Task ID.

Step 1342: and acquiring the number (recorded as NE) of non-empty exception results in the exception result list. In this embodiment, to avoid concurrent access of Data, the Data Lock (Data Lock) needs to be locked and unlocked before and after step 1342.

Step 1343: it is checked whether NE is equal to 1, i.e. whether the Task is the first exception-taking Task. When NE =1, step 1344 is entered.

Step 1344: the corresponding Job ID is added to the zombie task ID queue to wait for cleanup. In this embodiment, to avoid concurrent access to the data, the "Scheduler Lock" may be locked before the start of the step 1344 and unlocked after the completion of the step 1344.

Fig. 8 is a flowchart of waiting for the Task to end and feeding back the execution result to the user side in an embodiment of the present application. In an embodiment of the present application, based on the embodiment shown in fig. 4, the method may further include:

step 1051: the user thread blocks the return value waiting for the end of Job execution with a semaphore (semamphere). After N tasks obtained by decomposing a Job are submitted to a Task queue of a thread pool, a user thread needs to use semamphere to block and wait for N permissions, and meanwhile, the waiting Timeout duration is set as a Timeout parameter input by a user. The process of waiting for a return value by a user thread using semaphore blocking is roughly as follows: wait (N, timeout) is called by the user thread to block waiting for N permissions, wherein if the waiting for N permissions does not time out, the return value is "TRUE", and if the waiting times out, the return value is "FALSE". Wherein, for each Task: (1) If normal execution is completed, call the mapping. Add (1) to add 1 permission; wait (N, timeout) may return "TRUE" when N tasks are completed normally, completing N permissions; (2) If the execution process of a Task is abnormal, the Task calls semaphore.add (N) to add N permissions, which causes semaphore.wait (N, timeout) to immediately return to "TRUE". In addition, once the duration of semamphore congestion wait exceeds the Timeout parameter entered by the user, "FALSE" is returned. It should be noted that the return values of different programming languages or function libraries are different and can be adjusted according to specific requirements.

Step 1052: after receiving the return value, judging whether the execution process of the Job exceeds a preset time length according to the return value; when it is determined that the execution process of the Job does not exceed the preset wait timeout duration (the return value is "TRUE"), the process proceeds to step 104. As can be seen, in this embodiment, the triggering time in step 104 is that the parallel execution process of Task is not overtime, and when the obtained return value is "TRUE", it is checked whether the exception result list is empty, so that the operating pressure of the computer can be relieved to a certain extent. In other embodiments, after the Task is submitted to the Task queue of the thread pool, the exception result list may also be checked periodically or aperiodically to determine if an exception has occurred during Task execution.

With continued reference to fig. 8, in this embodiment, the method further includes:

step 1054: when the return value is "FALSE," Job ID is added to the zombie task ID queue to wait for cleanup.

Step 1055: indication information indicating "Job execution time-out" is returned.

The aforementioned step 104 may include steps 1041, 1042, 1043, wherein:

step 1041: and acquiring the number of the non-empty abnormal execution results in the abnormal result list, and recording the number as NE.

Step 1042: and (4) judging whether the NE is 0 or not, if so, entering the step 1053, and if not, entering the step 1043.

Step 1043: and returning an abnormal execution result comprising the abnormal information to a user side (task manager).

Step 1053: it is determined whether the value of the task cancel flag is FALSE (indicating that it is not canceled), if so, the process proceeds to step 1056, and if not, the process proceeds to step 1057.

Step 1056: an indication message indicating that "Job is executed successfully" is returned to the user side.

Step 1057: an indication message indicating that "Job is terminated" is returned to the user side.

Fig. 9 is a flowchart of cleaning bot tasks by using a bot task cleaning thread in an embodiment of the present application, and in this embodiment, the present application includes the following steps:

step 1061: it is checked whether the terminating task ID list is empty. If yes, wait for a certain period of time (e.g., 3 seconds), and then enter step 1061. If not, go to step 1062.

Step 1062: the first Job ID in the zombie task ID queue is fetched.

Step 1063: the Scheduler Lock (Scheduler Lock) is locked.

Step 1064: according to the Job ID, corresponding Job data (including a Task list and corresponding parameters) is taken out from the Task container, and the parent Task is deleted from the Task container.

Step 1065: traversing the Task list of Job, and for each Task, interrupting the thread executing the Task through the Future object stored inside the Task.

Step 1066: the value of the Job cancel flag is set to "TRUE" indicating that it has been cancelled.

Step 1067: n permissions are added to "Semaphore", where N is the number of tasks, so that a user thread that is waiting for N permissions returns from a blocked state, resuming execution.

Step 1068: the Scheduler Lock is unlocked. The steps 1063 and 1068 are used to prevent the concurrent access of the data.

Fig. 10 and fig. 11 are flowcharts of a task query process provided by an embodiment of the present application. In an embodiment of the present application, an interface (or function) for a user to query information or status of a task may be provided, and the user may query a specific Job or query all jobs that are executing without timeout.

As shown in fig. 10, in an embodiment, the process may be executed in a user thread, and the method includes:

step 1071: receiving a first query instruction containing the Job ID to be checked from a user side;

step 1072: responding to the first query instruction, and querying whether the Job ID to be queried exists in the task container; if so, proceed to step 1073. In one embodiment, to prevent concurrent access to data, the Scheduler Lock needs to be locked and unlocked before and after step 1072.

Step 1073: returning information indicating that the Job corresponding to the Job ID has not been executed and ended to the user side, wherein the returned information may further include, but is not limited to: job's execution state, job's Task input parameters, and so on.

In another embodiment, as shown in fig. 11, the process may be executed in a user thread, and the method includes:

step 1081: receiving a second query instruction from the user side for querying all Job in the task container;

step 1082: querying an object set consisting of task object IDs of all parent tasks existing in the task container in response to the second query instruction;

step 1083: and returning the object set obtained by query to the user side.

In summary, with the Task processing method provided in this embodiment of the present application, each Task obtained by decomposing the Job is added to the Task queue of the thread pool, and the work threads in the thread pool are used to execute each Task, so as to obtain an execution result corresponding to each Task and add the execution result to the result list corresponding to the Job. And when at least one execution result containing the abnormal information exists in the result list corresponding to the Job, the execution process of the Job is abnormal, and the abnormal information existing in the result list is fed back to the user side. In the process, when the Task execution process is abnormal, the abnormal information corresponding to the Task can be fed back to the user side, so that the Task manager can determine the abnormal reason according to the abnormal information, and the use experience of the Task manager is improved. In addition, the method can correctly process the conditions of exception of the Task, overtime of a parent Task and the like, and can reduce the workload of the software development process to a certain extent because fewer interfaces need to be realized by a user. In addition, the user is provided with an interface for blocking waiting, and the complexity of programming by the user is further reduced. In addition, the task manager is provided with an interface for inquiring the task and actively stopping the task, so that the user can be better involved in the monitoring and controlling of the task. Situations in which a task manager may actively terminate Job include: the number of tasks for a certain Job is too large, or each Task is too time-consuming, cannot be completed for a long time (the set timeout is long), affects the execution of other Jobs, and so on.

Fig. 12 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present application. The electronic device may be a server, and referring to fig. 12, at a hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required by other services. The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form the task processing device on the logic level. Of course, besides the software implementation, the present application does not exclude other implementations, such as logic devices or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, and may also be hardware or logic devices.

Fig. 13 is a block diagram of a task processing device according to an embodiment of the present application, and in a software implementation, the task processing device may include:

a decomposition unit 201, decomposing Job to be processed into one or more mutually independent tasks;

a first adding unit 202, configured to add the one or more mutually independent tasks obtained through decomposition to a Task queue of a thread pool;

the execution unit 203 is configured to take out and execute each Task from the Task queue in sequence by using the work thread in the thread pool, obtain an execution result corresponding to each Task, and add the execution result to the result list corresponding to the Job; wherein the execution result contains exception information when the Task is executing an exception;

an exception feedback unit 204, configured to feed back, to the user side, the exception information existing in the result list when at least one execution result including the exception information exists in the result list corresponding to the Job.

In one embodiment, the results list may include an exception results list, the execution results being added to the exception results list corresponding to Job when the Task executes an exception;

the anomaly feedback unit 204 is configured to:

and when at least one execution result containing the abnormal information exists in the abnormal result list corresponding to the Job, the Job feeds the abnormal information existing in the abnormal result list back to the user side.

Fig. 14 is a block diagram of a task processing device according to another embodiment of the present application, and in a software implementation, based on the embodiment shown in fig. 13, the task processing device may further include:

a waiting unit 205 for waiting for a return value of the Semaphore;

the anomaly feedback unit 204 may be configured to:

and when the obtained return value indicates that the execution process of the Job does not exceed the preset time length and at least one execution result containing the abnormal information exists in a result list corresponding to the Job, feeding the abnormal information existing in the result list back to the user side.

In another embodiment, the anomaly feedback unit 204 may further be configured to:

and when the parent task is overtime, the overtime information is fed back to the user side.

Fig. 15 is a block diagram of a task processing device according to another embodiment of the present application, and in a software implementation, the task processing device may further include, based on the embodiment shown in fig. 13:

a second adding unit 207, configured to add the Job ID to the zombie task ID queue when at least one execution result including abnormal information exists in the result list corresponding to the Job, or the Job is not executed within a preset time period and is finished, or a cancel instruction for canceling the Job is received from a user side.

And the Task terminating unit 208 is configured to terminate the work threads of each Task, which are obtained by decomposing the Task corresponding to the Job ID, one by one according to the Job ID existing in the zombie Task ID queue by using the zombie Task cleaning thread.

Fig. 16 is a block diagram of a task processing device according to still another embodiment of the present application, and in a software implementation, the task processing device may further include, based on the embodiment shown in fig. 13:

a third adding unit 209, configured to add the task object of the parent task into a task container, where the task container is used to store, using the parent task ID as an index, the task object corresponding to the parent task that has not been executed and has not exceeded a preset time length.

And the query unit 210 is configured to obtain a corresponding task query result according to the query instruction of the user and return the task query result to the user.

In an embodiment, the querying unit 210 specifically includes:

the first receiving unit is used for receiving a first query instruction containing the Job ID to be queried from the user side;

a first query unit, configured to query whether the Job ID to be queried exists in the task container in response to the first query instruction;

and the first query feedback unit is used for returning information used for indicating that the Job corresponding to the Job ID is not executed and finished to the user side when the Job ID to be queried exists in the task container.

In another embodiment, the querying unit 210 specifically includes:

the second receiving unit is used for receiving a second query instruction which is from the user side and used for querying all Jobs in the task container;

a second query unit, configured to query, in response to the second query instruction, an object set composed of task objects of all jobs existing in the task container; the set of objects includes: job ID and its parameters (including parameters for each Task).

And the second query feedback unit is used for returning the object set obtained by query to the user side.

The contents of the method embodiments and the contents of the apparatus embodiments described herein may be complementary to each other, unless they conflict with each other.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being divided into various units by function, respectively. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of other like elements in a process, method, article, or apparatus comprising the element.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art to which the present application pertains. Any modification, equivalent replacement, improvement or the like made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for processing a task, comprising:

adding the one or more independent subtasks obtained through decomposition into a task queue of a thread pool;

sequentially taking out each subtask from the task queue by using a plurality of working threads in the thread pool, executing each subtask in parallel through the plurality of working threads, obtaining an execution result corresponding to each subtask, and adding the execution result to a result list corresponding to the parent task; wherein the execution result contains exception information when the subtask execution is abnormal;

checking whether at least one execution result containing abnormal information exists in a result list corresponding to the parent task through a user thread so as to determine whether the execution process of the parent task is abnormal;

when at least one execution result containing the abnormal information exists in a result list corresponding to the parent task, feeding the abnormal information existing in the result list back to a user side;

when at least one execution result containing the abnormal information exists in the result list corresponding to the parent task, the parent task is abnormal, and the abnormal parent task is automatically terminated.

2. The method of claim 1, wherein the result list comprises an exception result list, wherein the execution result is added to an exception result list corresponding to the parent task when the child task executes an exception;

when at least one execution result containing the abnormal information exists in the result list corresponding to the parent task, the step of feeding back the abnormal information existing in the result list to the user side comprises the following steps:

and when at least one execution result containing the abnormal information exists in the abnormal result list corresponding to the parent task, feeding the abnormal information existing in the abnormal result list back to the user side.

3. The method according to claim 1, wherein after adding the one or more decomposed independent subtasks to a task queue of a thread pool, before feeding back the exception information existing in the result list to the user side, the method further comprises:

waiting for a return value of the Semaphore;

and when the obtained return value indicates that the execution process of the parent task does not exceed the preset time length and at least one execution result containing the abnormal information exists in a result list corresponding to the parent task, feeding the abnormal information existing in the result list back to the user side.

4. The method of claim 1, further comprising:

when at least one execution result containing abnormal information exists in a result list corresponding to the parent task, or the parent task is not executed and finished within a preset time length, or a cancel instruction for canceling the parent task is received from a user side, the parent task ID is added into a zombie task ID queue;

and terminating the working threads of all sub tasks which are obtained by decomposing the parent task corresponding to the parent task ID by utilizing a zombie task cleaning thread according to the parent task ID existing in the zombie task ID queue.

5. The method of claim 1, wherein after decomposing the pending parent task into one or more interdependent subtasks, the method further comprises:

adding the task object of the parent task into a task container, wherein the task container is used for storing the task object corresponding to the parent task which is not executed and does not exceed the preset time length by taking the ID of the parent task as an index;

receiving a first query instruction containing a parent task ID to be searched from a user side;

responding to the first query instruction, and querying whether the parent task ID to be queried exists in the task container;

and if so, returning information for indicating that the parent task corresponding to the parent task ID is not executed and ended to the user side.

6. The method of claim 1, wherein after decomposing the pending parent task into one or more interdependent child tasks, the method further comprises:

adding the task object of the parent task into a task container, wherein the task container is used for storing the task object corresponding to the parent task which is not executed and exceeds a preset time length by taking the ID of the parent task as an index;

receiving a second query instruction from the user side for querying all parent tasks in the task container;

querying a set of objects consisting of task objects of all parent tasks present within the task container in response to the second query instruction;

and returning the object set obtained by query to the user side.

7. A task processing apparatus, comprising:

a first adding unit, configured to add the one or more interdependent subtasks obtained through decomposition to a task queue of a thread pool;

the execution unit is used for taking out each subtask from the task queue in sequence by utilizing a plurality of working threads in the thread pool, executing each subtask in parallel through the plurality of working threads, obtaining an execution result corresponding to each subtask and adding the execution result into a result list corresponding to the parent task; wherein the execution result contains exception information when the subtask execution is abnormal;

the exception feedback unit is used for checking whether at least one execution result containing exception information exists in a result list corresponding to the parent task through a user thread so as to determine whether the execution process of the parent task is abnormal; when at least one execution result containing the abnormal information exists in a result list corresponding to the parent task, feeding the abnormal information existing in the result list back to a user side;

8. The apparatus of claim 7, wherein the result list comprises an exception result list, and wherein the execution result is added to an exception result list corresponding to the parent task when the child task executes an exception;

the anomaly feedback unit is used for:

9. The apparatus of claim 7, further comprising:

a waiting unit for waiting for a return value of the Semaphore;

the anomaly feedback unit is used for:

10. The apparatus of claim 7, further comprising:

the second adding unit is used for adding the parent task ID into a zombie task ID queue when at least one execution result containing abnormal information exists in a result list corresponding to the parent task, or the parent task is not executed and finished within a preset time length, or a cancel instruction for canceling the parent task is received from a user side;

and the task termination unit is used for utilizing a zombie task cleaning thread to terminate the working threads of all the sub tasks which are obtained by decomposing the parent task corresponding to the parent task ID according to the parent task ID existing in the zombie task ID queue one by one.

11. The apparatus of claim 7, further comprising:

a third adding unit, configured to add a task object of the parent task into a task container, where the task container is used to store, with the parent task ID as an index, a task object corresponding to the parent task that has not been executed and has not exceeded a preset duration;

the first receiving unit is used for receiving a first query instruction containing a parent task ID to be queried from a user side;

a first query unit, configured to query whether the parent task ID to be queried exists in the task container in response to the first query instruction;

and the first query feedback unit is used for returning information for indicating that the parent task corresponding to the parent task ID is not executed and finished to the user side when the parent task ID to be queried exists in the task container.

12. The apparatus of claim 7, further comprising:

the second receiving unit is used for receiving a second query instruction which is used for querying all parent tasks in the task container from the user side;

a second query unit, configured to query, in response to the second query instruction, an object set composed of task objects of all parent tasks existing in the task container;