CN113641493A - Task pushing method, device and equipment and computer readable storage medium - Google Patents
Task pushing method, device and equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN113641493A CN113641493A CN202110919401.7A CN202110919401A CN113641493A CN 113641493 A CN113641493 A CN 113641493A CN 202110919401 A CN202110919401 A CN 202110919401A CN 113641493 A CN113641493 A CN 113641493A
- Authority
- CN
- China
- Prior art keywords
- task
- executed
- execution node
- task execution
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 230000000977 initiatory effect Effects 0.000 claims abstract description 26
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012544 monitoring process Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000004883 computer application Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 1
- 239000003999 initiator Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/508—Monitor
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The invention discloses a task pushing method, which is applied to a task center and comprises the following steps: receiving a task to be executed registered by a task initiating node; judging whether an idle target task execution node exists in the task execution node set; the task execution node set comprises at least one task execution node which is subscribed to a task center in advance; if so, pushing the task to be executed to the target task execution node so as to execute the task to be executed by using the target task execution node. By applying the task pushing method provided by the invention, the risks of memory overflow and system downtime are greatly reduced, the reliability of the system is improved, and the task execution time is shortened. The invention also discloses a task pushing device, equipment and a storage medium, and has corresponding technical effects.
Description
Technical Field
The present invention relates to the field of computer application technologies, and in particular, to a task pushing method, device, and apparatus, and a computer-readable storage medium.
Background
Under distributed task scheduling, a common processing scheme is that a task initiator directly distributes tasks to a plurality of task execution nodes by using a consistent hash (hash) algorithm, so that the task execution nodes have the same number of tasks, and the task processing number of each node is within a reasonable range.
However, the processing time consumption of the task is not simply determined by the number of tasks, but also determined by the performance of the task execution node (CPU, memory, network IO, etc.) and the amount of calculation of the specific task. Therefore, the task dispatching method using the consistent hash algorithm results in different task amounts of the plurality of task execution nodes executing the computation, and it may happen that a task before some task execution nodes with a large running computation amount is not executed yet, and a next distributed task arrives, and occupies a part of memory and a time slice of the CPU to process the task just pushed, thereby resulting in long time consumption of task execution, that is, the total time of task execution is the final time after the last task is executed-the initial execution time. If more and more tasks are pushed on the nodes and the execution time is longer, the pressure Of some task execution nodes with high calculation pressure is high, and the risk Of Memory overflow (OOM) and even system downtime exists, which is only to cause the overall task scheduling time to be long.
In summary, how to effectively solve the problems that the existing task pushing mode easily causes memory overflow and even system downtime, and the task scheduling time is long, is a problem that needs to be solved urgently by those skilled in the art at present.
Disclosure of Invention
The invention aims to provide a task pushing method which greatly reduces risks of memory overflow and system downtime, improves system reliability and shortens task execution time; another object of the present invention is to provide a task pushing apparatus, a device and a computer readable storage medium.
In order to solve the technical problems, the invention provides the following technical scheme:
a task pushing method is applied to a task center and comprises the following steps:
receiving a task to be executed registered by a task initiating node;
judging whether an idle target task execution node exists in the task execution node set; the task execution node set consists of at least one task execution node which is subscribed to the task center in advance;
if so, pushing the task to be executed to the target task execution node so as to execute the task to be executed by using the target task execution node.
In a specific embodiment of the present invention, determining whether there is an idle target task execution node in a task execution node set includes:
and judging whether an idle target task execution node with a normal node state exists in the task execution node set.
In a specific embodiment of the present invention, after determining that there is an idle target task execution node in a task execution node set, before pushing the task to be executed to the target task execution node, the method further includes:
and locking the task to be executed according to the target task execution node.
In one embodiment of the present invention, the method further comprises:
when a task execution node subscription request is received, acquiring a newly added task execution node;
and adding the newly added task execution node to the task execution node set.
In a specific embodiment of the present invention, when there are a plurality of tasks to be executed, pushing the tasks to be executed to the target task execution node includes:
determining the task type corresponding to each task to be executed; wherein the task types include sequential tasks and unordered tasks;
and pushing each task to be executed to a corresponding target task execution node according to the sequence of the first sequential task and the second sequential task.
In a specific embodiment of the present invention, when each of the tasks to be executed includes an unordered task, the method further includes:
performing task splitting operation on the tasks to be executed belonging to the unordered task type to obtain each subtask;
pushing each task to be executed to a corresponding target task execution node according to the sequence of the first sequential task and the second sequential task, including:
pushing the tasks to be executed of each subtask and sequential task type to corresponding target task execution nodes respectively according to the sequence of the sequential tasks and the unordered tasks;
utilizing the target task execution node to execute the task to be executed, wherein the executing operation comprises the following steps:
and sequentially executing the tasks to be executed of the sequential task type by using the corresponding target task execution node, and executing each subtask in parallel.
In one embodiment of the present invention, the method further comprises:
monitoring the task state of each task to be executed to obtain a real-time task state corresponding to each task to be executed;
and displaying the real-time task state corresponding to each task to be executed in a file tree mode.
A task pushing device is applied to a task center and comprises:
the task receiving module is used for receiving the task to be executed registered by the task initiating node;
the judging module is used for judging whether idle target task execution nodes exist in the task execution node set or not; the task execution node set consists of at least one task execution node which is subscribed to the task center in advance;
and the task execution module is used for pushing the task to be executed to the target task execution node when determining that an idle target task execution node exists in the task execution node set, so as to execute the task to be executed by using the target task execution node.
A task pushing device comprising:
a memory for storing a computer program;
a processor for implementing the steps of the task pushing method when executing the computer program.
A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the task pushing method as set forth above.
The task pushing method provided by the invention is applied to a task center and used for receiving the task to be executed registered by a task initiating node; judging whether an idle target task execution node exists in the task execution node set; the task execution node set comprises at least one task execution node which is subscribed to a task center in advance; if so, pushing the task to be executed to the target task execution node so as to execute the task to be executed by using the target task execution node.
According to the technical scheme, the task center for task scheduling management is preset, and when the task initiating node needs to initiate the task to be executed, the task to be executed is registered to the task center. The task execution node set used for task execution is subscribed to the task center in advance, and the task center can acquire the running state of each task execution node in the task execution node set. When the task center receives the task to be executed, whether an idle target task execution node exists in the task execution node set or not is searched, if yes, the task to be executed is pushed to the target task execution node, and the target task execution node is utilized to execute the task to be executed. Compared with the existing mode of task distribution by adopting a consistent hash algorithm, the invention realizes the decoupling of the task initiating node and the task executing nodes, avoids the pressure inclination among the task executing nodes by the modes of task registration and node subscription, lightens the pressure of a single task executing node, greatly reduces the risks of memory overflow and system downtime, improves the reliability of the system and shortens the task executing time.
Correspondingly, the invention also provides a task pushing device, equipment and a computer readable storage medium corresponding to the task pushing method, which have the technical effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart of an implementation of a task pushing method according to an embodiment of the present invention;
FIG. 2 is a flowchart of another implementation of a task pushing method according to an embodiment of the present invention;
FIG. 3 is a block diagram of a task pushing system according to an embodiment of the present invention;
FIG. 4 is a block diagram of a task pushing apparatus according to an embodiment of the present invention;
FIG. 5 is a block diagram illustrating a task pushing apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a task pushing device according to this embodiment.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart of an implementation of a task pushing method in an embodiment of the present invention, which is applied to a task center, and the method may include:
s101: and receiving the task to be executed registered by the task initiating node.
The method comprises the steps that communication connection between a task initiating node and a task center is established in advance, after the task initiating node generates a task to be executed, the task initiating node registers the task to be executed to the task center, and the task center receives the task to be executed registered by the task initiating node and manages the task to be executed.
S102: and judging whether an idle target task execution node exists in the task execution node set, if so, executing the step S103, and if not, not processing.
The task execution node set is composed of at least one task execution node which is subscribed to the task center in advance.
The method comprises the steps that a task execution node set is preset, the task execution node set comprises at least one task execution node which is subscribed to a task center in advance, and communication connection between the task execution node and the task center in the task execution node set is established in advance. After receiving a task to be executed registered by a task initiating node, a task center judges whether an idle target task executing node exists in a task executing node set, namely judges whether a task executing node which does not execute the task currently exists in the task executing node set, if so, indicates that a task executing node which can be used for executing the task currently exists, and if not, indicates that a task executing node which can be used for executing the task does not exist currently and does not process the task.
S103: and pushing the task to be executed to the target task execution node so as to execute the task to be executed by using the target task execution node.
When the idle target task execution node exists in the task execution node set, the task execution node which can be used for executing the task currently exists is indicated, the task to be executed is pushed to the target task execution node, and therefore the target task execution node is utilized to execute the task to be executed. The task is published and executed based on the task center by means of task registration and task execution node subscription, decoupling of a task initiating node and a task executing node is achieved, and node pressure inclination caused by different computing resources consumed by a single task in a traditional task scheduling scheme is solved. The method for publishing the subscription task reduces the pressure of the task execution node, the task execution node acquires the task to be executed from the task center after the task execution node successfully executes the task, and supports the parallel acquisition of a plurality of execution tasks, thereby reducing the pressure of a single node, avoiding the conditions of memory overflow, downtime and the like, shortening the execution time of the task, and improving the service performance and reliability.
According to the technical scheme, the task center for task scheduling management is preset, and when the task initiating node needs to initiate the task to be executed, the task to be executed is registered to the task center. The task execution node set used for task execution is subscribed to the task center in advance, and the task center can acquire the running state of each task execution node in the task execution node set. When the task center receives the task to be executed, whether an idle target task execution node exists in the task execution node set or not is searched, if yes, the task to be executed is pushed to the target task execution node, and the target task execution node is utilized to execute the task to be executed. Compared with the existing mode of task distribution by adopting a consistent hash algorithm, the invention realizes the decoupling of the task initiating node and the task executing nodes, avoids the pressure inclination among the task executing nodes by the modes of task registration and node subscription, lightens the pressure of a single task executing node, greatly reduces the risks of memory overflow and system downtime, improves the reliability of the system and shortens the task executing time.
It should be noted that, based on the above embodiments, the embodiments of the present invention also provide corresponding improvements. In the following embodiments, steps that are the same as or correspond to those in the above embodiments may be referred to one another, and corresponding advantageous effects may also be referred to one another, which is not described in detail in the following modified embodiments.
Referring to fig. 2, fig. 2 is a flowchart of another implementation of a task pushing method in an embodiment of the present invention, where the method may include the following steps:
s201: and receiving the task to be executed registered by the task initiating node.
Referring to fig. 3, fig. 3 is a block diagram of a task pushing system according to an embodiment of the present invention. The task initiating node A initiates a task A and the task initiating node B initiates a task B in the task pushing system.
S202: and judging whether a target task execution node which is idle and has a normal node state exists in the task execution node set, if so, executing the step S203, and if not, not processing.
The task execution node set is composed of at least one task execution node which is subscribed to the task center in advance.
After receiving a task to be executed registered by a task initiating node, when querying a task executing node, judging whether an idle target task executing node with a normal node state exists in a task executing node set, thereby screening the target task executing node with a normal running state and not executing the task currently. For example, the task center may detect the operating state of the task execution nodes by detecting heartbeats of the task execution nodes, and when the task center finds that some task execution nodes are down or lost for a long time through a heartbeat mechanism, the task center actively releases the tasks executed by the task execution nodes, waits for other task execution nodes to acquire and execute the tasks, and maintains high availability of the tasks. By detecting the running state of the task execution node, the reliability of the task execution node is ensured, and the task scheduling efficiency is improved.
Referring to fig. 3, the task pushing system may include a task execution node a, a task execution node B, a task execution node C, a task execution node D, and a task execution node E subscribed to the task center.
S203: and locking the task to be executed according to the target task execution node.
After the target task execution node which is idle and has a normal node state is determined to exist, the task to be executed is locked according to the target task execution node, so that the task to be executed is locked to the target task execution node, repeated distribution of the task is avoided, and resources are saved.
S204: and pushing the task to be executed to the target task execution node so as to execute the task to be executed by using the target task execution node.
In one embodiment of the present invention, the method may further comprise the steps of:
the method comprises the following steps: when a task execution node subscription request is received, acquiring a newly added task execution node;
step two: and adding the newly added task execution node to the task execution node set.
For convenience of description, the above two steps may be combined for illustration.
And when a task execution node needs to be added, sending a task execution node subscription request to the task center, wherein the task execution node subscription request comprises the new task execution node to be added to the task execution node set. When the task center receives the task execution node subscription request, the task center acquires the newly added task execution nodes and adds the newly added task execution nodes to the task execution node set, so that the capacity expansion of the task execution nodes is realized.
In a specific embodiment of the present invention, when there are a plurality of tasks to be executed, pushing the tasks to be executed to the target task execution node may include the following steps:
the method comprises the following steps: and determining the task type corresponding to each task to be executed.
Wherein the task types include sequential tasks and unordered tasks.
Step two: and pushing each task to be executed to a corresponding target task execution node according to the sequence of the first sequential task and the second sequential task.
For convenience of description, the above two steps may be combined for illustration.
A task to be executed is generally composed of a plurality of subtasks, and the task is classified in advance according to the task execution order of each subtask included in the task to be executed, and the task type is divided into a sequential task and an unordered task. When the number of the tasks to be executed is multiple, the task type corresponding to each task to be executed is determined, namely, whether each task to be executed belongs to a sequential task or an unordered task is determined.
Because each subtask included in the sequential task needs to execute the task according to a preset execution sequence, when the sequential task and the unordered task exist at the same time, the sequential task is executed first, and then the unordered task is executed. After the task types corresponding to the tasks to be executed are determined, the tasks to be executed are pushed to the corresponding target task execution nodes according to the sequence of the first-order tasks and the second-order tasks.
In an embodiment of the present invention, when each task to be executed includes an unordered task, the method may further include the following steps:
performing task splitting operation on the tasks to be executed belonging to the unordered task type to obtain each subtask;
the method for pushing each task to be executed to a corresponding target task execution node according to the sequence of the first sequential task and the second sequential task may include the following steps:
pushing each subtask and the task to be executed of the sequential task type to a corresponding target task execution node according to the sequence of the sequential task and the unordered task;
the executing operation of the task to be executed by using the target task executing node may include the following steps:
and sequentially executing the tasks to be executed in the sequential task type by using the corresponding target task execution node, and executing all subtasks in parallel.
When each task to be executed comprises an unordered task, task splitting operation is carried out on the task to be executed which belongs to the unordered task type to obtain each subtask, each subtask and the task to be executed of the sequential task type are respectively pushed to corresponding target task execution nodes according to the sequence of the sequential task and the unordered task, the task to be executed of the sequential task type is sequentially executed by utilizing the corresponding target task execution nodes, and each subtask is executed in parallel. By ensuring the task execution sequence of the first-order task and the second-order task, the tasks to be executed in the unordered task type are executed in parallel, and the task execution efficiency is greatly improved.
For example, when the task center receives a sequential task a (10 subtasks) and an unordered task BC (20 subtasks each), the task execution process includes executing 10 subtasks in the task a in series, and then executing the task B and the task C in parallel, that is, after the task a is finished, the task B and the task C are executed.
In one embodiment of the present invention, the method may further comprise the following:
the method comprises the following steps: monitoring the task state of each task to be executed to obtain a real-time task state corresponding to each task to be executed;
step two: and displaying the real-time task states corresponding to the tasks to be executed respectively in a file tree mode.
For convenience of description, the above two steps may be combined for illustration.
After receiving the to-be-executed tasks registered by the task initiating node, monitoring the task state of each to-be-executed task to obtain the real-time task state corresponding to each to-be-executed task, and displaying the real-time task state corresponding to each to-be-executed task in a file tree mode, so that the real-time tracking of the to-be-executed task state is realized, the effective allocation of the tasks is facilitated, and the task scheduling efficiency is improved.
The task state can be divided into: init, process, done, fail.
Init: in the initialization, no subtask exists in the task to be executed and is sent to a specific task execution node for execution;
the Process: in the execution flow, part of subtasks of the task to be executed are already sent to a specific node for execution;
done: the execution is successful, and all subtasks of the task to be executed are successfully executed;
fail: and (4) executing the tasks in a failure mode, wherein the tasks to be executed have subtask execution failures in each subtask (after repeated retries, the tasks still fail).
When the subtask is not pushed to the specific execution node, the display result is as follows: task 1: ";
when the subtask is pushed to a specific node, the identifier is a specific executed Internet protocol address (Internet protocol, IP), and the display result is: task 1: xx.xx.xx.xx: xx ";
the subtask execution fails (fail + number of times of execution failure), and the display result is: task 1: fail 1;
the subtask retries execution failure (fail) multiple times, showing the result as: task 1: fail;
the subtask was successfully executed (done), showing the result as: task 1: done.
Corresponding to the above method embodiment, the present invention further provides a task pushing device, and the task pushing device described below and the task pushing method described above may be referred to correspondingly.
Referring to fig. 4, fig. 4 is a block diagram of a task pushing device in an embodiment of the present invention, which is applied to a task center, and the task pushing device may include:
a task receiving module 41, configured to receive a task to be executed registered by a task initiating node;
a judging module 42, configured to judge whether there is an idle target task execution node in the task execution node set; the task execution node set comprises at least one task execution node which is subscribed to a task center in advance;
and the task executing module 43 is configured to, when it is determined that an idle target task executing node exists in the task executing node set, push the to-be-executed task to the target task executing node, so as to perform an executing operation on the to-be-executed task by using the target task executing node.
According to the technical scheme, the task center for task scheduling management is preset, and when the task initiating node needs to initiate the task to be executed, the task to be executed is registered to the task center. The task execution node set used for task execution is subscribed to the task center in advance, and the task center can acquire the running state of each task execution node in the task execution node set. When the task center receives the task to be executed, whether an idle target task execution node exists in the task execution node set or not is searched, if yes, the task to be executed is pushed to the target task execution node, and the target task execution node is utilized to execute the task to be executed. Compared with the existing mode of task distribution by adopting a consistent hash algorithm, the invention realizes the decoupling of the task initiating node and the task executing nodes, avoids the pressure inclination among the task executing nodes by the modes of task registration and node subscription, lightens the pressure of a single task executing node, greatly reduces the risks of memory overflow and system downtime, improves the reliability of the system and shortens the task executing time.
In an embodiment of the present invention, the determining module 42 is a module that determines whether there is a target task execution node in the task execution node set that is idle and has a normal node state.
In one embodiment of the present invention, the apparatus may further include:
and the task locking module is used for locking the to-be-executed task according to the target task execution node before pushing the to-be-executed task to the target task execution node after determining that the idle target task execution node exists in the task execution node set.
In one embodiment of the present invention, the apparatus may further include:
the execution node acquisition module is used for acquiring a newly added task execution node when a task execution node subscription request is received;
and the node adding module is used for adding the newly added task execution node to the task execution node set.
In one embodiment of the present invention, the task execution module 43 includes:
the task type determining submodule is used for determining the task type corresponding to each task to be executed when the number of the tasks to be executed is multiple; the task types comprise sequential tasks and unordered tasks;
and the task pushing submodule is used for respectively pushing each task to be executed to the corresponding target task execution node according to the sequence of the first sequential task and the second unordered task.
In one embodiment of the present invention, the apparatus may further include:
the task splitting module is used for performing task splitting operation on the tasks to be executed which belong to the unordered task type to obtain each subtask;
the task pushing submodule is a module which respectively pushes each subtask and the task to be executed of the sequential task type to the corresponding target task execution node according to the sequence of the sequential task and the unordered task;
the task execution module 43 includes a task execution sub-module, and the task execution sub-module is specifically a module that sequentially executes tasks to be executed of sequential task types by using corresponding target task execution nodes, and executes each sub-task in parallel.
In one embodiment of the present invention, the apparatus may further include:
the task state acquisition module is used for monitoring the task state of each task to be executed to obtain the real-time task state corresponding to each task to be executed;
and the state display module is used for displaying the real-time task states corresponding to the tasks to be executed respectively in a file tree mode.
Corresponding to the above method embodiment, referring to fig. 5, fig. 5 is a schematic diagram of a task pushing device provided by the present invention, where the task pushing device may include:
a memory 332 for storing a computer program;
the processor 322 is configured to implement the steps of the task pushing method of the above method embodiments when executing the computer program.
Specifically, referring to fig. 6, fig. 6 is a schematic diagram of a specific structure of a task pushing device according to this embodiment, which may generate a relatively large difference due to different configurations or performances, and may include a processor (CPU) 322 (e.g., one or more processors) and a memory 332, where the memory 332 stores one or more computer applications 342 or data 344. Memory 332 may be, among other things, transient or persistent storage. The program stored in memory 332 may include one or more modules (not shown), each of which may include a sequence of instructions operating on a data processing device. Still further, the processor 322 may be configured to communicate with the memory 332 to execute a series of instruction operations in the memory 332 on the task pushing device 301.
The task push device 301 may also include one or more power sources 326, one or more wired or wireless network interfaces 350, one or more input-output interfaces 358, and/or one or more operating systems 341.
The steps in the task pushing method described above may be implemented by the structure of the task pushing device.
Corresponding to the above method embodiment, the present invention further provides a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps of:
receiving a task to be executed registered by a task initiating node; judging whether an idle target task execution node exists in the task execution node set; the task execution node set comprises at least one task execution node which is subscribed to a task center in advance; if so, pushing the task to be executed to the target task execution node so as to execute the task to be executed by using the target task execution node.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
For the introduction of the computer-readable storage medium provided by the present invention, please refer to the above method embodiments, which are not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device, the apparatus and the computer-readable storage medium disclosed in the embodiments correspond to the method disclosed in the embodiments, so that the description is simple, and the relevant points can be referred to the description of the method.
The principle and the implementation of the present invention are explained in the present application by using specific examples, and the above description of the embodiments is only used to help understanding the technical solution and the core idea of the present invention. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. A task pushing method is applied to a task center and comprises the following steps:
receiving a task to be executed registered by a task initiating node;
judging whether an idle target task execution node exists in the task execution node set; the task execution node set consists of at least one task execution node which is subscribed to the task center in advance;
if so, pushing the task to be executed to the target task execution node so as to execute the task to be executed by using the target task execution node.
2. The task pushing method according to claim 1, wherein determining whether there is an idle target task execution node in the set of task execution nodes comprises:
and judging whether an idle target task execution node with a normal node state exists in the task execution node set.
3. The task pushing method according to claim 1, wherein after determining that there is a target task execution node that is idle in the set of task execution nodes, before pushing the task to be executed to the target task execution node, further comprising:
and locking the task to be executed according to the target task execution node.
4. The task pushing method according to claim 1, further comprising:
when a task execution node subscription request is received, acquiring a newly added task execution node;
and adding the newly added task execution node to the task execution node set.
5. The task pushing method according to any one of claims 1 to 4, wherein when there are a plurality of tasks to be executed, pushing the task to be executed to the target task execution node includes:
determining the task type corresponding to each task to be executed; wherein the task types include sequential tasks and unordered tasks;
and pushing each task to be executed to a corresponding target task execution node according to the sequence of the first sequential task and the second sequential task.
6. The task pushing method according to claim 5, when each of the tasks to be executed includes an unordered task, further comprising:
performing task splitting operation on the tasks to be executed belonging to the unordered task type to obtain each subtask;
pushing each task to be executed to a corresponding target task execution node according to the sequence of the first sequential task and the second sequential task, including:
pushing the tasks to be executed of each subtask and sequential task type to corresponding target task execution nodes respectively according to the sequence of the sequential tasks and the unordered tasks;
utilizing the target task execution node to execute the task to be executed, wherein the executing operation comprises the following steps:
and sequentially executing the tasks to be executed of the sequential task type by using the corresponding target task execution node, and executing each subtask in parallel.
7. The task pushing method according to claim 5, further comprising:
monitoring the task state of each task to be executed to obtain a real-time task state corresponding to each task to be executed;
and displaying the real-time task state corresponding to each task to be executed in a file tree mode.
8. A task pushing device is applied to a task center and comprises:
the task receiving module is used for receiving the task to be executed registered by the task initiating node;
the judging module is used for judging whether idle target task execution nodes exist in the task execution node set or not; the task execution node set consists of at least one task execution node which is subscribed to the task center in advance;
and the task execution module is used for pushing the task to be executed to the target task execution node when determining that an idle target task execution node exists in the task execution node set, so as to execute the task to be executed by using the target task execution node.
9. A task pushing device, characterized by comprising:
a memory for storing a computer program;
a processor for implementing the steps of the task pushing method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the task pushing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110919401.7A CN113641493A (en) | 2021-08-11 | 2021-08-11 | Task pushing method, device and equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110919401.7A CN113641493A (en) | 2021-08-11 | 2021-08-11 | Task pushing method, device and equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113641493A true CN113641493A (en) | 2021-11-12 |
Family
ID=78420839
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110919401.7A Withdrawn CN113641493A (en) | 2021-08-11 | 2021-08-11 | Task pushing method, device and equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113641493A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115955319A (en) * | 2023-03-14 | 2023-04-11 | 季华实验室 | Data set generation system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170331867A1 (en) * | 2015-06-17 | 2017-11-16 | Tencent Technology (Shenzhen) Company Limited | Method, device and system for pushing file |
| CN108255589A (en) * | 2017-08-29 | 2018-07-06 | 平安科技（深圳）有限公司 | Method for scheduling task and server |
| CN109656699A (en) * | 2018-12-14 | 2019-04-19 | 平安医疗健康管理股份有限公司 | Distributed computing method, device, system, equipment and readable storage medium storing program for executing |
| CN112486648A (en) * | 2020-11-30 | 2021-03-12 | 北京百度网讯科技有限公司 | Task scheduling method, device, system, electronic equipment and storage medium |
| CN112527489A (en) * | 2020-12-22 | 2021-03-19 | 税友软件集团股份有限公司 | Task scheduling method, device and equipment and computer readable storage medium |
-
2021
- 2021-08-11 CN CN202110919401.7A patent/CN113641493A/en not_active Withdrawn
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170331867A1 (en) * | 2015-06-17 | 2017-11-16 | Tencent Technology (Shenzhen) Company Limited | Method, device and system for pushing file |
| CN108255589A (en) * | 2017-08-29 | 2018-07-06 | 平安科技（深圳）有限公司 | Method for scheduling task and server |
| CN109656699A (en) * | 2018-12-14 | 2019-04-19 | 平安医疗健康管理股份有限公司 | Distributed computing method, device, system, equipment and readable storage medium storing program for executing |
| CN112486648A (en) * | 2020-11-30 | 2021-03-12 | 北京百度网讯科技有限公司 | Task scheduling method, device, system, electronic equipment and storage medium |
| CN112527489A (en) * | 2020-12-22 | 2021-03-19 | 税友软件集团股份有限公司 | Task scheduling method, device and equipment and computer readable storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115955319A (en) * | 2023-03-14 | 2023-04-11 | 季华实验室 | Data set generation system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107087019B (en) | Task scheduling method and device based on end cloud cooperative computing architecture | |
| US10924538B2 (en) | Systems and methods of monitoring software application processes | |
| CN108810115B (en) | Load balancing method and device suitable for distributed database and server | |
| CN111897633A (en) | Task processing method and device | |
| CN106648900B (en) | Supercomputing method and system based on smart television | |
| CN111930493B (en) | NodeManager state management method and device in cluster and computing equipment | |
| CN102385536B (en) | Method and system for realization of parallel computing | |
| CN112231108A (en) | Task processing method and device, computer readable storage medium and server | |
| CN113641493A (en) | Task pushing method, device and equipment and computer readable storage medium | |
| CN111221653A (en) | Service processing method and device and computer readable storage medium | |
| CN112631756A (en) | Distributed regulation and control method and device applied to space flight measurement and control software | |
| CN111835809B (en) | Work order message distribution method, work order message distribution device, server and storage medium | |
| US10193744B1 (en) | Mass restoration of enterprise business services following service disruption | |
| CN112468310B (en) | Streaming media cluster node management method and device and storage medium | |
| CN111431951B (en) | Data processing method, node equipment, system and storage medium | |
| CN116599828A (en) | Task execution method and device, storage medium and electronic equipment | |
| US10789129B1 (en) | Rolling restoration of enterprise business services following service disruption | |
| CN111064636B (en) | Control method, device and system for connection of front-end processor and computer equipment | |
| CN111414206A (en) | Method and device for starting and loading metadata in parallel based on HDFS | |
| CN113485798A (en) | Kernel function generation method, apparatus, device and storage medium | |
| CN114710485A (en) | Processing method and processing apparatus | |
| CN114546631A (en) | Task scheduling method, control method, core, electronic device and readable medium | |
| CN113703930A (en) | Task scheduling method, device and system and computer readable storage medium | |
| CN111309467A (en) | Task distribution method and device, electronic equipment and storage medium | |
| CN110647440A (en) | Big data task processing method and system based on state machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20211112 |