CN113238843A - Task execution method, device, equipment and storage medium - Google Patents

Task execution method, device, equipment and storage medium Download PDF

Info

Publication number
CN113238843A
CN113238843A CN202110524439.4A CN202110524439A CN113238843A CN 113238843 A CN113238843 A CN 113238843A CN 202110524439 A CN202110524439 A CN 202110524439A CN 113238843 A CN113238843 A CN 113238843A
Authority
CN
China
Prior art keywords
executed
tasks
task
queue
annotation information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110524439.4A
Other languages
Chinese (zh)
Inventor
王学理
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Zhenshi Information Technology Co Ltd
Original Assignee
Beijing Jingdong Zhenshi Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Zhenshi Information Technology Co Ltd filed Critical Beijing Jingdong Zhenshi Information Technology Co Ltd
Priority to CN202110524439.4A priority Critical patent/CN113238843A/en
Publication of CN113238843A publication Critical patent/CN113238843A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G06F9/5066—Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • G06F9/524—Deadlock detection or avoidance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/48—Indexing scheme relating to G06F9/48
    • G06F2209/484—Precedence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5011—Pool
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5018—Thread allocation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/5021—Priority

Abstract

The invention discloses a task execution method, a task execution device, a task execution equipment and a task execution storage medium. The method comprises the following steps: acquiring at least two tasks to be executed based on a distributed timing task framework, wherein the at least two tasks to be executed carry annotation information, and the annotation information comprises: identifying a task; querying a database according to the annotation information to obtain thread pools corresponding to the at least two tasks to be executed, wherein the database comprises: task identification of at least one type of task and a thread pool corresponding to the task identification; based on the thread pool, the at least two tasks to be executed are executed in parallel, and the technical scheme of the invention can realize the parallel execution of the tasks and improve the task execution efficiency.

Description

Task execution method, device, equipment and storage medium
Technical Field
The embodiment of the invention relates to the technical field of computers, in particular to a task execution method, a task execution device, a task execution equipment and a task execution storage medium.
Background
The streaming operation of the distributed timing task framework firstly acquires tasks when scheduling is triggered every time, if the tasks are acquired, then the tasks are executed serially one by one, the task acquisition operation and the task execution operation of the distributed timing task framework are executed serially, and the distributed timing task framework executes a plurality of tasks serially one by one after the acquired tasks are executed.
In the process of implementing the invention, at least the following technical problems are found in the prior art:
when the logic of task execution is complex, network delay or time-consuming situation occurs, a certain task is blocked, and then the execution of the subsequent task and the next data acquisition are blocked. Resulting in slow task processing and affecting the efficiency of business operations.
Disclosure of Invention
Embodiments of the present invention provide a method, an apparatus, a device, and a storage medium for task execution, so as to implement parallel execution of tasks and improve task execution efficiency.
In a first aspect, an embodiment of the present invention provides a task execution method, including:
acquiring at least two tasks to be executed based on a distributed timing task framework, wherein the at least two tasks to be executed carry annotation information, and the annotation information comprises: identifying a task;
querying a database according to the annotation information to obtain thread pools corresponding to the at least two tasks to be executed, wherein the database comprises: task identification of at least one type of task and a thread pool corresponding to the task identification;
and executing the at least two tasks to be executed in parallel based on the thread pool.
In a second aspect, an embodiment of the present invention further provides a task execution device, where the task execution device includes:
the task obtaining module is configured to obtain at least two tasks to be executed based on a distributed timing task framework, where the at least two tasks to be executed carry annotation information, and the annotation information includes: identifying a task;
a query module, configured to query a database according to the annotation information to obtain thread pools corresponding to the at least two tasks to be executed, where the database includes: task identification of at least one type of task and a thread pool corresponding to the task identification;
and the execution module is used for executing the at least two tasks to be executed in parallel based on the thread pool.
In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the task execution method according to any one of the embodiments of the present invention.
In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the task execution method according to any one of the embodiments of the present invention.
The embodiment of the invention acquires at least two to-be-executed tasks carrying annotation information based on a distributed timing task framework, queries a database according to the annotation information to obtain thread pools corresponding to the at least two to-be-executed tasks, and executes the at least two to-be-executed tasks in parallel based on the thread pools, so as to solve the problems that after the distributed timing task framework acquires the at least two to-be-executed tasks, the at least two to-be-executed tasks are executed serially, if the processing logic of any task is complex, the network is delayed or the time is consumed, a certain task is blocked, the execution of the subsequent task is further blocked, the data is acquired next time, the task processing is slow, and the business operation efficiency is affected.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a flowchart of a task execution method according to a first embodiment of the present invention;
FIG. 2 is a flowchart of a task execution method according to a second embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a task execution device according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device in the fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Example one
Fig. 1 is a flowchart of a task execution method according to an embodiment of the present invention, where this embodiment is applicable to a case of task execution based on a distributed timing task framework, and the method may be executed by a task execution device in an embodiment of the present invention, where the device may be implemented in a software and/or hardware manner, as shown in fig. 1, the method specifically includes the following steps:
s110, acquiring at least two tasks to be executed based on a distributed timing task framework, wherein the at least two tasks to be executed carry annotation information, and the annotation information comprises: and identifying the task.
The distributed timing task framework may be an early-joba framework, or may be other distributed timing task frameworks, which is not limited in this embodiment of the present invention.
The task identifier may be composed of at least one of numbers, letters, and symbols, and the task identifier may also be in other forms, which is not limited in this embodiment of the present invention.
For example, the task identifier corresponding to the first type of task is a task identifier a, the task identifier of the second type of task is a task identifier B, and the task identifier of the third type of task is a task identifier C.
The annotation information may be annotation information added for the task when the task is established. The annotation information may include: the task identification can further comprise: the maximum number of threads may also include: the type identifier is executed in parallel, which is not limited in this embodiment of the present invention.
For example, the manner of obtaining at least two tasks to be executed based on the distributed timed task framework may be: and acquiring task grabbing logic based on the distributed timing task framework, and grabbing at least two tasks to be executed through the task grabbing logic. Wherein the number of tasks to be performed is related to the fetch logic.
S120, querying a database according to the annotation information to obtain thread pools corresponding to the at least two tasks to be executed, wherein the database comprises: task identification of at least one type of task and a thread pool corresponding to the task identification.
Wherein the database comprises: the task identifier of at least one type of task and the thread pool corresponding to the task identifier may be, for example, that the database includes: the task identifier A corresponds to a thread pool P, the task identifier B corresponds to a thread pool O, the task identifier C corresponds to a thread pool Q and the like.
For example, the manner of obtaining the thread pools corresponding to the at least two tasks to be executed according to the annotation information query database may be: a database about the corresponding relation between the task identifier and the thread pool is established in advance, and the annotation information comprises: and the task identifier queries the database according to the task identifier to obtain a thread pool corresponding to the task identifier, namely the thread pools corresponding to at least two tasks to be executed. For example, it may be: pre-establishing a database, wherein the database comprises: the task identifier A corresponds to a thread pool P, the task identifier B corresponds to a thread pool O, the task identifier C corresponds to a thread pool Q, a first task to be executed and a second task to be executed are obtained based on a distributed timing task frame, and if annotation information carried by at least two tasks to be executed is: and the task identifier A queries the database according to the task identifier A to obtain a thread pool P corresponding to the task identifier A.
S130, executing the at least two tasks to be executed in parallel based on the thread pool.
For example, the way of executing at least two tasks to be executed in parallel based on the thread pool may be: and executing the at least two acquired tasks to be executed in parallel based on the thread pool, and acquiring the at least two tasks to be executed based on the distributed timing task framework after all the tasks are executed. For example, the first to-be-executed task and the second to-be-executed task are acquired based on the thread pool, and the first to-be-executed task and the second to-be-executed task are executed in parallel based on the thread pool. In the prior art, if a first task to be executed and a second task to be executed are obtained based on a thread pool, the second task to be executed must be executed after the first task to be executed is executed. If the situation of logic complexity, network delay or time consumption occurs when the first task to be executed is executed, the second task to be executed is caused to be blocked, and further the execution of the subsequent task and the next data acquisition are blocked, so that the task processing is slow, and the business operation efficiency is influenced, in the embodiment of the invention, the first task to be executed and the second task to be executed are executed in parallel based on the thread pool, so that the situation that the second task to be executed is executed after the first task to be executed is executed in the prior art can be avoided, and the situation that the task processing is slow, the business operation efficiency is influenced, and the task can be executed in parallel, because the second task to be executed is caused to be blocked, and further the execution of the subsequent task and the next data acquisition are blocked, when the situation that the logic complexity, the network delay or the time consumption occurs when the first task to be executed, and the task execution efficiency is improved.
For example, the manner for executing at least two tasks to be executed in parallel based on the thread pool may also be: and determining a parallel execution type according to the annotation information, and executing at least two tasks to be executed in parallel according to the thread pool and the parallel execution type. For example, it may be: and if the parallel execution type is A, writing at least two tasks to be executed into the queue, executing the tasks to be executed in the queue in parallel based on the thread pool until the tasks to be executed in the queue are all executed, and returning to the step of executing and capturing the at least two tasks to be executed. If the parallel execution type is B, writing at least two tasks to be executed which are obtained for the first time into a queue, and executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed; when the tasks to be executed in the queue are all executed, returning to execute the step of grabbing at least two tasks to be executed; determining the maximum number of queue executions according to the annotation information, determining the number of queue idleness according to the maximum number of queue executions, and determining the number of tasks to be executed to be written into the queue according to the number of queue idleness if the number of the tasks to be executed which are not obtained for the first time is less than or equal to the maximum number of queue executions; determining a target task to be executed according to the number of the tasks to be executed written into the queue; writing the target tasks to be executed into the queue, executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed, ignoring the non-target tasks to be executed in the tasks to be executed which are not acquired for the first time, returning to execute the step of acquiring at least two tasks to be executed based on the distributed timing task framework, ignoring the tasks to be executed which are not acquired for the first time if the number of the tasks to be executed which are not acquired for the first time is larger than the maximum number of the tasks to be executed in the queue, and returning to execute the step of acquiring at least two tasks to be executed based on the distributed timing task framework. For example, if the parallel execution type is B, writing the first obtained 8 tasks to be executed into the queue, and executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed; and determining that the maximum number of queue executions is 5 according to the annotation information, if 6 to-be-executed tasks are acquired for the second time, ignoring the 6 to-be-executed tasks, and returning to the step of executing the to-be-executed tasks acquired based on the distributed timing task framework. If 4 to-be-executed tasks are acquired for the third time, determining the queue idle number to be 2 according to the maximum queue execution number to be 5, determining the number of to-be-executed tasks to be written into the queue to be 2 according to the queue idle number 2, determining target to-be-executed tasks to be a to-be-executed task X and a to-be-executed task Y according to the number of to-be-executed tasks to be written into the queue to be 2, writing the to-be-executed task X and the to-be-executed task Y into the queue, executing the to-be-executed tasks in the queue based on the thread pool until the to-be-executed tasks in the queue are all executed, ignoring the to-be-executed tasks except the to-be-executed task X and the to-be-executed task Y in the 3 rd acquired to-be-executed tasks, acquiring the to-be-executed tasks for the fourth time based on the distributed timing task framework, and repeatedly executing the steps.
Optionally, before querying the database according to the annotation information, the method further includes:
obtaining annotation information of at least one type of task, wherein the annotation information further comprises: a maximum number of threads;
creating a thread pool for the at least one type of task according to the maximum thread quantity;
and correspondingly storing the thread pool and the task identifier into a database.
The maximum number of threads may be set in advance according to a task type, or may be set by a system, which is not limited in this embodiment of the present invention.
For example, the manner of obtaining the annotation information of at least one task may be: a first table of a corresponding relationship between the feature information of the task and the annotation information is pre-established, and the first table is searched according to the feature information of the task to obtain corresponding annotation information, which may be, for example: the first table has stored therein: the feature information a corresponds to annotation information P, the feature information b corresponds to annotation information O, and the feature information c corresponds to annotation information Q. And if the characteristic information of the first type task is a, querying a first table to obtain annotation information P corresponding to the characteristic information a. The manner of obtaining the annotation information of at least one task may also be: and pre-establishing a second table of the corresponding relation between the task identifier and the maximum thread number, and searching the second table according to the task identifier to obtain the maximum thread number corresponding to the task identifier. For example, it may be: the second table stores: the task identifier A corresponds to the maximum thread number M, the task identifier B corresponds to the maximum thread number N, and the task identifier C corresponds to the maximum thread number W. And if the task identifier of the first type of task is A, querying a second table to obtain the maximum thread number M corresponding to the task identifier A.
For example, the thread pool is created for the at least one type of task according to the maximum thread number, and for example, the thread pool P is created for the first type of task, the thread pool O is created for the second type of task, and the thread pool Q is created for the third type of task according to the maximum thread number corresponding to the first type of task.
Illustratively, the thread pool and the task identifier are correspondingly stored in a database, for example, the database may include: the task identifier A corresponds to a thread pool P, the task identifier B corresponds to a thread pool O, and the task identifier C corresponds to a thread pool Q.
Optionally, the obtaining at least two tasks to be executed based on the distributed timing task framework includes:
acquiring task grabbing logic based on a distributed timing task framework;
and grabbing at least two tasks to be executed through grabbing logic.
Wherein the number of the grabbed tasks to be executed is related to grabbing logic.
The technical scheme of the embodiment obtains at least two tasks to be executed carrying annotation information based on a distributed timing task framework, querying the database according to the annotation information to obtain thread pools corresponding to at least two tasks to be executed, executing the at least two tasks to be executed in parallel based on the thread pools, the method solves the problem that after a distributed timing task framework acquires at least two tasks to be executed, the at least two tasks to be executed are executed in series, if the processing logic of any task is complex, the network is delayed or time is consumed, a certain task is blocked, and then the execution of subsequent tasks is blocked, data is acquired next time, so that the problems that the task processing is slow and the service operation efficiency is influenced are caused.
Example two
Fig. 2 is a flowchart of a task execution method according to a second embodiment of the present invention, and the second embodiment is further optimized based on the first embodiment. The annotation information further includes: identifying the parallel execution type; correspondingly, the parallel execution of the at least two tasks to be executed based on the thread pool includes: determining a parallel execution type according to the parallel execution type identifier; and executing the at least two tasks to be executed in parallel based on the thread pool and the parallel execution type.
As shown in fig. 2, the method includes:
s210, acquiring at least two tasks to be executed based on a distributed timing task framework, wherein the at least two tasks to be executed carry annotation information, and the annotation information comprises: task identification and parallel execution type identification.
The parallel execution type identifier may be composed of at least one of a number, a letter, and a symbol, and may also be in other forms, and the parallel execution type identifier is different from the task identifier.
The parallel execution type identifier may be a parallel execution type identifier manually added when the task is established, or may be a parallel execution type identifier obtained by a system querying a correspondence table according to feature information of the task, which is not limited in this embodiment of the present invention.
S220, querying a database according to the annotation information to obtain at least two thread pools corresponding to the tasks to be executed, wherein the database comprises: and task identifications of at least one type of task and a thread pool corresponding to the task identifications.
And S230, determining the parallel execution type according to the parallel execution type identifier.
For example, the method for determining the parallel execution type according to the parallel execution type identifier may be: establishing a corresponding relationship table about the parallel execution type identifier and the parallel execution type in advance, obtaining the parallel execution type identifier in the annotation information, and querying the corresponding relationship table according to the parallel execution type identifier to obtain the parallel execution type, for example, the corresponding relationship table may include: the parallel execution type identifier R corresponds to a first type of parallel execution type, the parallel execution type identifier S corresponds to a second type of parallel execution type, the parallel execution type identifier T corresponds to a third type of parallel execution type, the parallel execution type identifier R in the annotation information is obtained, and the corresponding relation table is inquired according to the parallel execution type identifier R to obtain the first type of parallel execution type.
S240, executing the at least two tasks to be executed in parallel based on the thread pool and the parallel execution type.
For example, the parallel execution mode of the at least two tasks to be executed based on the thread pool and the parallel execution type may be: if the parallel execution type is the first type, writing at least two tasks to be executed into the queue, and executing the tasks to be executed in the queue in batch based on the thread pool; and returning to execute the step of grabbing at least two tasks to be executed until the tasks to be executed in the queue are all executed. The parallel execution mode of the at least two tasks to be executed based on the thread pool and the parallel execution type can also be as follows: if the parallel execution type is the second type, determining the maximum number of queue execution according to the annotation information, writing at least two tasks to be executed obtained for the first time into the queue, and executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed; and returning to execute the step of grabbing at least two tasks to be executed after the tasks to be executed in the queue are all executed. If the number of the tasks to be executed which are not acquired for the first time is less than or equal to the maximum number of the queue execution, determining the number of the tasks to be executed which are to be written into the queue according to the queue idle number; determining a target task to be executed according to the number of the tasks to be executed of the queue to be written; writing the target tasks to be executed into a queue, executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed, ignoring the non-target tasks to be executed in the tasks to be executed which are not acquired for the first time, returning to execute the steps of acquiring at least two tasks to be executed based on the distributed timing task framework, if the number of the tasks to be executed which are not acquired for the first time is larger than the maximum number of the tasks to be executed in the queue, ignoring the tasks to be executed which are not acquired for the first time, and returning to execute the steps of acquiring at least two tasks to be executed based on the distributed timing task framework.
Optionally, executing the at least two tasks to be executed in parallel based on the thread pool and the parallel execution type includes:
if the parallel execution type is determined to be the first type according to the parallel execution type identifier, writing the at least two tasks to be executed into a queue;
executing the tasks to be executed in the queue in batch based on the thread pool;
and returning to execute the step of grabbing at least two tasks to be executed until the tasks to be executed in the queue are all executed.
Illustratively, if the parallel execution type is the first type, directly writing the captured at least two tasks to be executed into the queue, executing the tasks to be executed in the queue in batch based on the thread pool until the tasks to be executed in the queue are all executed, returning to the step of capturing the at least two tasks to be executed, for example, 8 to-be-executed tasks are acquired based on a distributed timing task framework, a parallel execution type identifier in annotation information is acquired, if the parallel execution type identifier is R, a corresponding relation table is queried, the parallel execution type corresponding to the parallel execution type identifier R is a first type, 8 to-be-processed tasks are all written into a queue, the 8 to-be-executed tasks in the queue are executed in parallel based on a thread pool, and the step of executing and capturing at least two to-be-executed tasks is returned after the 8 to-be-executed tasks in the queue are all executed. In the prior art, if 8 to-be-executed tasks are acquired based on a thread pool, a second to-be-executed task must be executed after a first to-be-executed task is executed, a third to-be-executed task must be executed after the second to-be-executed task is executed, and so on. If the first task to be executed is complex in logic, delayed in network or time-consuming, the rest tasks to be executed are blocked, and further the execution of the subsequent tasks and the next data acquisition are blocked, so that the task processing is slow, and the business operation efficiency is affected. If the first task to be executed is complex in logic, delayed in network or time-consuming, the rest tasks to be executed are blocked, and then the execution of the subsequent tasks and the next data acquisition are blocked, so that the conditions that the task processing is slow and the service operation efficiency is affected can be caused, the tasks can be executed in parallel, and the task execution efficiency is improved.
Optionally, executing the at least two tasks to be executed in parallel based on the thread pool and the parallel execution type includes:
if the parallel execution type is determined to be the second type according to the parallel execution type identifier, determining the maximum number of queue executions according to annotation information;
writing at least two tasks to be executed obtained for the first time into a queue, and executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed;
and returning to execute the step of grabbing at least two tasks to be executed after the tasks to be executed in the queue are all executed.
Illustratively, if the parallel execution type is determined to be the second type according to the parallel execution type identifier, determining the maximum number of queue executions according to the annotation information; writing at least two tasks to be executed which are obtained for the first time into a queue, and executing the tasks to be executed in the queue based on a thread pool until the tasks to be executed in the queue are all executed; and returning to execute the step of grabbing at least two tasks to be executed after the tasks to be executed in the queue are all executed. For example, if 8 to-be-executed tasks are acquired for the first time based on the distributed timing task framework, if the parallel execution type is the second type, the 8 to-be-executed tasks acquired for the first time are written into the queue, and the to-be-executed tasks in the queue are executed based on the thread pool until the to-be-executed tasks in the queue are all executed. If 20 to-be-executed tasks are acquired for the first time based on the distributed timing task framework, if the parallel execution type is the second type, the 20 to-be-executed tasks acquired for the first time are written into the queue, the to-be-executed tasks in the queue are executed based on the thread pool until the to-be-executed tasks in the queue are all executed, and after the to-be-executed tasks in the queue are all executed, the steps of executing and capturing at least two to-be-executed tasks are returned.
Optionally, executing the at least two tasks to be executed in parallel based on the thread pool and the parallel execution type includes:
if the parallel execution type is determined to be the second type according to the parallel execution type identifier, determining the maximum number of queue executions according to annotation information;
determining the queue idle number according to the maximum queue execution number;
if the number of the tasks to be executed which are not acquired for the first time is less than or equal to the maximum number of the queue execution, determining the number of the tasks to be executed which are to be written into the queue according to the queue idle number;
determining a target task to be executed according to the number of the tasks to be executed of the queue to be written;
and writing the target tasks to be executed into a queue, and executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed.
The method for determining the maximum number of queue executions according to the annotation information may be as follows: the annotation information comprises queue execution maximum data, and the queue execution maximum data in the annotation information is directly obtained; the way of determining the maximum number of queue executions according to the annotation information may also be: a corresponding relation list of the task identification and the maximum queue execution quantity is established in advance, and the corresponding relation list is inquired according to the task identification to obtain the maximum queue execution quantity corresponding to the task identification; the embodiments of the present invention are not limited in this regard.
For example, if the maximum number of queue executions is 5, the number of queue idleness in the 5 queues is obtained according to the maximum number of queue executions being 5.
If the number of the tasks to be executed, which are not acquired for the first time, is less than or equal to the maximum number of the queue executions, the number of the tasks to be executed, which are to be written into the queue, is determined according to the queue idle number, for example, if the number of the tasks to be executed, which are acquired for the second time, is 4, the maximum number of the queue executions is 5, and 4 is less than 5, the number of the tasks to be executed, which are to be written into the queue, is determined to be 2 according to the queue idle number, and optionally 2 tasks to be executed, which are to be written into the queue, are determined to be the target tasks to be executed.
For example, if the parallel execution type is determined to be the second type according to the parallel execution type identifier, determining the maximum number of queue executions according to annotation information; determining the queue idle number according to the maximum queue execution number; if the number of the tasks to be executed which are not acquired for the first time is less than or equal to the maximum number of the queue execution, determining the number of the tasks to be executed which are to be written into the queue according to the queue idle number; determining a target task to be executed according to the number of the tasks to be executed of the queue to be written; and writing the target tasks to be executed into a queue, and executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed. For example, at least two tasks to be executed are acquired based on a distributed timing task framework, where the at least two tasks to be executed carry annotation information, and the annotation information includes: task identification and parallel execution type identification; determining the parallel execution type to be a second type according to the parallel execution type identifier, determining the maximum number of queue execution to be 5 according to the annotation information, determining the queue idle number to be 2 according to the maximum number of queue execution to be 5, if 4 to-be-executed tasks are acquired for the second time, determining the number of to-be-executed tasks to be written into the queue to be 2 according to the queue idle number because 4 is less than 5, determining 2 to-be-executed tasks to be target to-be-executed tasks from any 2 to-be-executed tasks to be 2 according to the number of to-be-executed tasks to be written into the queue, writing the 2 target to-be-executed tasks into the queue, and executing the to-be-executed tasks in the queue based on the thread pool until the to-be-executed tasks in the queue are all.
Optionally, the method further includes:
and ignoring the non-target tasks to be executed in the tasks to be executed which are not acquired for the first time, and returning to execute the step of acquiring at least two tasks to be executed based on the distributed timing task framework.
For example, if the parallel execution type is determined to be the second type according to the parallel execution type identifier, determining the maximum number of queue executions according to annotation information; determining the queue idle number according to the maximum queue execution number; if the number of the tasks to be executed which are not acquired for the first time is less than or equal to the maximum number of the queue execution, determining the number of the tasks to be executed which are to be written into the queue according to the queue idle number; determining a target task to be executed according to the number of the tasks to be executed of the queue to be written; and ignoring the non-target tasks to be executed in the tasks to be executed which are not acquired for the first time, and returning to execute the step of acquiring at least two tasks to be executed based on the distributed timing task framework. For example, at least two tasks to be executed are acquired based on a distributed timing task framework, where the at least two tasks to be executed carry annotation information, and the annotation information includes: task identification and parallel execution type identification; determining the parallel execution type as a second type according to the parallel execution type identifier, determining the maximum number of queue executions as 5 according to the annotation information, determining the queue idle number to be 2 according to the maximum number of the executed queues to be 5, if 4 tasks to be executed are acquired for the second time, since 4 is less than 5, the number of the tasks to be executed which are to be written into the queue is determined to be 2 according to the queue free number, and according to the number of the tasks to be executed written into the queue being 2, optionally selecting 2 tasks to be executed from 4 tasks to be executed, determining the tasks to be executed as target tasks, writing the 2 target tasks to be executed into the queue, ignoring other tasks except the 2 target tasks to be executed, returning to execute the steps of obtaining at least two tasks to be executed based on the distributed timing task framework and executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed and complete synchronous execution.
In the embodiment of the invention, the two steps of acquiring at least two tasks to be executed based on the distributed timing task framework and executing the tasks to be executed in the queue based on the thread pool are executed in parallel, in the prior art, the tasks to be executed are acquired first, and then the acquired tasks to be executed are executed, that is, the two steps of acquiring the tasks to be executed and executing the acquired tasks to be executed are executed in series.
Optionally, the method further includes:
and if the number of the tasks to be executed which are not obtained for the first time is larger than the maximum number of the tasks to be executed in the queue, ignoring the tasks to be executed which are not obtained for the first time, and returning to the step of obtaining at least two tasks to be executed based on the distributed timing task framework.
Illustratively, if the number of the tasks to be executed, which are not acquired for the first time, is greater than the maximum number of the queues to be executed, the tasks to be executed, which are not acquired for the first time, are ignored, and the step of acquiring at least two tasks to be executed based on the distributed timing task framework is returned to be executed, for example, the step of acquiring at least two tasks to be executed based on the distributed timing task framework may be executed, where the at least two tasks to be executed carry annotation information, and the annotation information includes: task identification and parallel execution type identification; and determining the parallel execution type to be a second type according to the parallel execution type identifier, determining the maximum number of queue executions to be 5 according to the annotation information, if 8 to-be-executed tasks are acquired for the second time, and if 8 is larger than 5, ignoring the 8 acquired execution tasks for the second time, and returning to execute the step of acquiring at least two to-be-executed tasks based on the distributed timing task framework.
The technical scheme of this embodiment includes acquiring at least two to-be-executed tasks carrying annotation information based on a distributed timing task framework, querying a database according to the annotation information to obtain at least two thread pools corresponding to the to-be-executed tasks, determining a parallel execution type according to a parallel execution type identifier in the annotation information, executing the at least two to-be-executed tasks in parallel based on the thread pools and the parallel execution type, so as to solve the problems that after the distributed timing task framework acquires the at least two to-be-executed tasks, the at least two to-be-executed tasks are executed serially, if any task is complex in processing logic, network delay or time consuming, a certain task is blocked, further the execution of subsequent tasks is blocked, and data is acquired next time, so that task processing is slow and business operation efficiency is affected, and after the distributed timing task framework acquires the at least two to-be-executed tasks, at least two tasks to be executed can be executed in parallel according to the parallel execution type and the thread pool, or two steps of acquiring the tasks and executing the tasks are executed in parallel, and the task execution efficiency is improved.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a task execution device according to a third embodiment of the present invention. The embodiment may be applicable to the case of task execution based on a distributed timed task framework, where the apparatus may be implemented in a software and/or hardware manner, and the apparatus may be integrated in any device that provides a task execution function, as shown in fig. 3, where the task execution apparatus specifically includes: a task obtaining module 310, a query module 320, and an execution module 330.
The task obtaining module is configured to obtain at least two tasks to be executed based on a distributed timing task framework, where the at least two tasks to be executed carry annotation information, and the annotation information includes: identifying a task;
a query module, configured to query a database according to the annotation information to obtain thread pools corresponding to the at least two tasks to be executed, where the database includes: task identification of at least one type of task and a thread pool corresponding to the task identification;
and the execution module is used for executing the at least two tasks to be executed in parallel based on the thread pool.
Optionally, the method further includes:
an information obtaining module, configured to obtain annotation information of at least one type of task before querying a database according to the annotation information, where the annotation information further includes: a maximum number of threads;
a thread pool creating module for creating a thread pool for the at least one type of task according to the maximum thread number;
and the storage module is used for correspondingly storing the thread pool and the task identifier into a database.
Optionally, the annotation information further includes: identifying the parallel execution type;
correspondingly, the execution module is specifically configured to:
determining a parallel execution type according to the parallel execution type identifier;
and executing the at least two tasks to be executed in parallel based on the thread pool and the parallel execution type.
Optionally, the execution module is specifically configured to:
if the parallel execution type is determined to be the first type according to the parallel execution type identifier, writing the at least two tasks to be executed into a queue;
executing the tasks to be executed in the queue in batch based on the thread pool;
and returning to execute the step of grabbing at least two tasks to be executed until the tasks to be executed in the queue are all executed.
Optionally, the execution module is specifically configured to:
if the parallel execution type is determined to be the second type according to the parallel execution type identifier, determining the maximum number of queue executions according to annotation information;
writing at least two tasks to be executed obtained for the first time into a queue, and executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed;
and returning to execute the step of grabbing at least two tasks to be executed after the tasks to be executed in the queue are all executed.
Optionally, the execution module is specifically configured to:
if the parallel execution type is determined to be the second type according to the parallel execution type identifier, determining the maximum number of queue executions according to annotation information;
determining the queue idle number according to the maximum queue execution number;
if the number of the tasks to be executed which are not acquired for the first time is less than or equal to the maximum number of the queue execution, determining the number of the tasks to be executed which are to be written into the queue according to the queue idle number;
determining a target task to be executed according to the number of the tasks to be executed of the queue to be written;
and writing the target tasks to be executed into a queue, and executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed.
Optionally, the execution module is specifically configured to:
and ignoring the non-target tasks to be executed in the tasks to be executed which are not acquired for the first time, and returning to execute the step of acquiring at least two tasks to be executed based on the distributed timing task framework.
Optionally, the execution module is specifically configured to:
and if the number of the tasks to be executed which are not obtained for the first time is larger than the maximum number of the tasks to be executed in the queue, ignoring the tasks to be executed which are not obtained for the first time, and returning to the step of obtaining at least two tasks to be executed based on the distributed timing task framework.
Optionally, the task obtaining module is specifically configured to:
acquiring task grabbing logic based on a distributed timing task framework;
and grabbing at least two tasks to be executed through grabbing logic.
The product can execute the method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
The technical scheme of the embodiment obtains at least two tasks to be executed carrying annotation information based on a distributed timing task framework, querying the database according to the annotation information to obtain thread pools corresponding to at least two tasks to be executed, executing the at least two tasks to be executed in parallel based on the thread pools, the method solves the problem that after a distributed timing task framework acquires at least two tasks to be executed, the at least two tasks to be executed are executed in series, if the processing logic of any task is complex, the network is delayed or time is consumed, a certain task is blocked, and then the execution of subsequent tasks is blocked, data is acquired next time, so that the problems that the task processing is slow and the service operation efficiency is influenced are caused.
Example four
Fig. 4 is a schematic structural diagram of a computer device in the fourth embodiment of the present invention. FIG. 4 illustrates a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention. The computer device 12 shown in FIG. 4 is only one example and should not bring any limitations to the functionality or scope of use of embodiments of the present invention.
As shown in FIG. 4, computer device 12 is in the form of a general purpose computing device. The components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16.
Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
Computer device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The system Memory 28 may include computer system readable media in the form of volatile Memory, such as Random Access Memory (RAM) 30 and/or cache Memory 32. Computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (a Compact disk-Read Only Memory (CD-ROM)), Digital Video disk (DVD-ROM), or other optical media may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
Computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with computer device 12, and/or with any devices (e.g., network card, modem, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface 22. In the computer device 12 of the present embodiment, the display 24 is not provided as a separate body but is embedded in the mirror surface, and when the display surface of the display 24 is not displayed, the display surface of the display 24 and the mirror surface are visually integrated. Moreover, computer device 12 may also communicate with one or more networks (e.g., a Local Area Network (LAN), Wide Area Network (WAN)) and/or a public Network (e.g., the Internet) via Network adapter 20. As shown, network adapter 20 communicates with the other modules of computer device 12 via bus 18. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, Redundant processing units, external disk drive Arrays, disk array (RAID) systems, tape drives, and data backup storage systems, to name a few.
The processing unit 16 executes various functional applications and data processing by executing programs stored in the system memory 28, for example, to implement a task execution method provided by an embodiment of the present invention:
acquiring at least two tasks to be executed based on a distributed timing task framework, wherein the at least two tasks to be executed carry annotation information, and the annotation information comprises: identifying a task;
querying a database according to the annotation information to obtain thread pools corresponding to the at least two tasks to be executed, wherein the database comprises: task identification of at least one type of task and a thread pool corresponding to the task identification;
and executing the at least two tasks to be executed in parallel based on the thread pool.
EXAMPLE five
Fifth embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the task execution method provided in all the inventive embodiments of this application:
acquiring at least two tasks to be executed based on a distributed timing task framework, wherein the at least two tasks to be executed carry annotation information, and the annotation information comprises: identifying a task;
querying a database according to the annotation information to obtain thread pools corresponding to the at least two tasks to be executed, wherein the database comprises: task identification of at least one type of task and a thread pool corresponding to the task identification;
and executing the at least two tasks to be executed in parallel based on the thread pool.
Any combination of one or more computer-readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (Hyper Text Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, or the like, as well as conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of an element does not in some cases constitute a limitation on the element itself.
The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims (12)

1. A method of task execution, comprising:
acquiring at least two tasks to be executed based on a distributed timing task framework, wherein the at least two tasks to be executed carry annotation information, and the annotation information comprises: identifying a task;
querying a database according to the annotation information to obtain thread pools corresponding to the at least two tasks to be executed, wherein the database comprises: task identification of at least one type of task and a thread pool corresponding to the task identification;
and executing the at least two tasks to be executed in parallel based on the thread pool.
2. The method of claim 1, prior to querying a database based on the annotation information, further comprising:
obtaining annotation information of at least one type of task, wherein the annotation information further comprises: a maximum number of threads;
creating a thread pool for the at least one type of task according to the maximum thread quantity;
and correspondingly storing the thread pool and the task identifier into a database.
3. The method according to claim 1 or 2, wherein the annotation information further comprises: identifying the parallel execution type;
correspondingly, the parallel execution of the at least two tasks to be executed based on the thread pool includes:
determining a parallel execution type according to the parallel execution type identifier;
and executing the at least two tasks to be executed in parallel based on the thread pool and the parallel execution type.
4. The method of claim 3, wherein executing the at least two tasks to be executed in parallel based on the thread pool and the parallel execution type comprises:
if the parallel execution type is determined to be the first type according to the parallel execution type identifier, writing the at least two tasks to be executed into a queue;
executing the tasks to be executed in the queue in batch based on the thread pool;
and returning to execute the step of grabbing at least two tasks to be executed until the tasks to be executed in the queue are all executed.
5. The method of claim 3, wherein executing the at least two tasks to be executed in parallel based on the thread pool and the parallel execution type comprises:
if the parallel execution type is determined to be the second type according to the parallel execution type identifier, determining the maximum number of queue executions according to annotation information;
writing at least two tasks to be executed obtained for the first time into a queue, and executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed;
and returning to execute the step of grabbing at least two tasks to be executed after the tasks to be executed in the queue are all executed.
6. The method of claim 3, wherein executing the at least two tasks to be executed in parallel based on the thread pool and the parallel execution type comprises:
if the parallel execution type is determined to be the second type according to the parallel execution type identifier, determining the maximum number of queue executions according to annotation information;
determining the queue idle number according to the maximum queue execution number;
if the number of the tasks to be executed which are not acquired for the first time is less than or equal to the maximum number of the queue execution, determining the number of the tasks to be executed which are to be written into the queue according to the queue idle number;
determining a target task to be executed according to the number of the tasks to be executed of the queue to be written;
and writing the target tasks to be executed into a queue, and executing the tasks to be executed in the queue based on the thread pool until the tasks to be executed in the queue are all executed.
7. The method of claim 6, further comprising:
and ignoring the non-target tasks to be executed in the tasks to be executed which are not acquired for the first time, and returning to execute the step of acquiring at least two tasks to be executed based on the distributed timing task framework.
8. The method of claim 6, further comprising:
and if the number of the tasks to be executed which are not obtained for the first time is larger than the maximum number of the tasks to be executed in the queue, ignoring the tasks to be executed which are not obtained for the first time, and returning to the step of obtaining at least two tasks to be executed based on the distributed timing task framework.
9. The method of claim 1, wherein obtaining at least two tasks to be performed based on a distributed timed task framework comprises:
acquiring task grabbing logic based on a distributed timing task framework;
and grabbing at least two tasks to be executed through grabbing logic.
10. A task execution apparatus, comprising:
the task obtaining module is configured to obtain at least two tasks to be executed based on a distributed timing task framework, where the at least two tasks to be executed carry annotation information, and the annotation information includes: identifying a task;
a query module, configured to query a database according to the annotation information to obtain thread pools corresponding to the at least two tasks to be executed, where the database includes: task identification of at least one type of task and a thread pool corresponding to the task identification;
and the execution module is used for executing the at least two tasks to be executed in parallel based on the thread pool.
11. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1-9 when executing the program.
12. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-9.
CN202110524439.4A 2021-05-13 2021-05-13 Task execution method, device, equipment and storage medium Pending CN113238843A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110524439.4A CN113238843A (en) 2021-05-13 2021-05-13 Task execution method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110524439.4A CN113238843A (en) 2021-05-13 2021-05-13 Task execution method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113238843A true CN113238843A (en) 2021-08-10

Family

ID=77134144

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110524439.4A Pending CN113238843A (en) 2021-05-13 2021-05-13 Task execution method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113238843A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113849292A (en) * 2021-11-30 2021-12-28 天聚地合(苏州)数据股份有限公司 Timed task execution method and device, storage medium and equipment
CN114327872A (en) * 2021-12-14 2022-04-12 特赞（上海）信息科技有限公司 Multimedia asynchronous processing method and device
CN115601195A (en) * 2022-10-17 2023-01-13 桂林电子科技大学（Cn） Transaction bidirectional recommendation system and method based on real-time label of power user

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150135183A1 (en) * 2013-11-12 2015-05-14 Oxide Interactive, LLC Method and system of a hierarchical task scheduler for a multi-thread system
CN107341054A (en) * 2017-06-29 2017-11-10 广州市百果园信息技术有限公司 Task executing method, device and computer-readable recording medium
CN110806933A (en) * 2019-11-05 2020-02-18 中国建设银行股份有限公司 Batch task processing method, device, equipment and storage medium
CN111625329A (en) * 2020-05-18 2020-09-04 北京达佳互联信息技术有限公司 Task allocation method and device, electronic equipment, server and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150135183A1 (en) * 2013-11-12 2015-05-14 Oxide Interactive, LLC Method and system of a hierarchical task scheduler for a multi-thread system
CN107341054A (en) * 2017-06-29 2017-11-10 广州市百果园信息技术有限公司 Task executing method, device and computer-readable recording medium
CN110806933A (en) * 2019-11-05 2020-02-18 中国建设银行股份有限公司 Batch task processing method, device, equipment and storage medium
CN111625329A (en) * 2020-05-18 2020-09-04 北京达佳互联信息技术有限公司 Task allocation method and device, electronic equipment, server and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
高翔;张金登;: "基于线程池的多任务并行处理模型", 指挥信息系统与技术, no. 04 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113849292A (en) * 2021-11-30 2021-12-28 天聚地合(苏州)数据股份有限公司 Timed task execution method and device, storage medium and equipment
CN114327872A (en) * 2021-12-14 2022-04-12 特赞（上海）信息科技有限公司 Multimedia asynchronous processing method and device
CN115601195A (en) * 2022-10-17 2023-01-13 桂林电子科技大学（Cn） Transaction bidirectional recommendation system and method based on real-time label of power user
CN115601195B (en) * 2022-10-17 2023-09-08 桂林电子科技大学 Transaction bidirectional recommendation system and method based on real-time label of power user

Similar Documents

Publication Publication Date Title
CN113238843A (en) Task execution method, device, equipment and storage medium
CN108965355B (en) Method, apparatus and computer readable storage medium for data transmission
CN105787077B (en) Data synchronization method and device
EP3916630A1 (en) Method and apparatus for identifying video
CN112104734B (en) Method, device, equipment and storage medium for pushing information
CN109033814B (en) Intelligent contract triggering method, device, equipment and storage medium
CN107589990B (en) Data communication method and system based on thread pool
EP3825865A2 (en) Method and apparatus for processing data
CN110851276A (en) Service request processing method, device, server and storage medium
CN112416338A (en) Code warehouse system based on label
CN105786917B (en) Method and device for concurrent warehousing of time series data
US20230401215A1 (en) Event Sequences Search
CN112613964A (en) Account checking method, account checking device, account checking equipment and storage medium
EP3889770A1 (en) Mini program material processing method and apparatus, electronic device, and medium
CN110837412B (en) Method, device, equipment and storage medium for judging operation ready state
CN114168607A (en) Global serial number generation method, device, equipment, medium and product
CN110879818B (en) Method, device, medium and electronic equipment for acquiring data
CN112818204A (en) Service processing method, device, equipment and storage medium
CN112506490A (en) Interface generation method and device, electronic equipment and storage medium
CN113779117A (en) Data monitoring method and device, storage medium and electronic equipment
CN112784195A (en) Page data publishing method and system
CN111339776A (en) Resume parsing method and device, electronic equipment and computer-readable storage medium
CN112148705A (en) Data migration method and device
CN111143464A (en) Data acquisition method and device and electronic equipment
CN110727666A (en) Cache assembly, method, equipment and storage medium for industrial internet platform

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination