CN114327805A

CN114327805A - Task scheduling system, method and device, electronic equipment and storage medium

Info

Publication number: CN114327805A
Application number: CN202011064445.8A
Authority: CN
Inventors: 曹平
Original assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Current assignee: Beijing Kingsoft Cloud Network Technology Co Ltd
Priority date: 2020-09-30
Filing date: 2020-09-30
Publication date: 2022-04-12

Abstract

The embodiment of the invention provides a task scheduling system, a task scheduling method, a task scheduling device, electronic equipment and a storage medium. The system comprises a dispatching node and a plurality of executing nodes, wherein each executing node is provided with a node label; the dispatching node is used for acquiring the task to be processed and the target node label; adding a target node label for the task to be processed, and sending the task to be processed added with the target node label to a task queue; each execution node is used for acquiring a target task from the task queue and processing the target task; wherein, the target task is as follows: the target node label is the same to-be-processed task as the node label of the executing node. Compared with the prior art, the scheme provided by the embodiment of the invention can ensure the safety of the task to be processed.

Description

Task scheduling system, method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of internet technologies, and in particular, to a task scheduling system, method, apparatus, electronic device, and storage medium.

Background

Currently, with the continuous development of internet technology, "cloud" plays an increasingly important role in various fields, such as cloud computing, cloud storage, and the like. The cloud is an abstraction of the internet and the underlying infrastructure, and can be divided into a public cloud and a private cloud.

Among them, a so-called Private cloud (Private cloud) is a cloud constructed for one customer to use alone. Specifically, the customer owns the underlying infrastructure and can control the manner in which applications are deployed on this underlying infrastructure, thereby effectively controlling data security and quality of service in the private cloud.

In the related art, a plurality of public execution nodes are arranged in a private cloud, and the plurality of public execution nodes are used for processing various types of tasks to be processed of a client, for example, the plurality of public execution nodes in a private cloud of a certain company can process various types of tasks to be processed initiated by various departments such as finance, administration, sales, research and development of the company, and the like. Specifically, a plurality of common execution nodes are managed by a management node in a unified manner, and the management node acquires a task to be processed sent by each department of a client and distributes the task to be processed to one of the plurality of common execution nodes so as to process the task to be processed.

However, in the above related art, the common execution node to which the to-be-processed task is allocated has an uncertainty characteristic, so that the security of the to-be-processed task cannot be guaranteed.

Disclosure of Invention

Embodiments of the present invention provide a task scheduling system, method, apparatus, electronic device, and storage medium, so as to ensure the security of a task to be processed. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present invention provides a task scheduling system, where the system includes a dispatch node and multiple execution nodes, and each execution node has a node tag;

the dispatching node is used for acquiring a task to be processed and a target node label; adding the target node label to the task to be processed, and sending the task to be processed added with the target node label to a task queue;

each execution node is configured to acquire a target task from the task queue and process the target task, where the target task is: the target node label is the same to-be-processed task as the node label of the executing node.

Optionally, in a specific implementation manner, the number of the task queues is multiple, and there are execution nodes with the same node label in the multiple execution nodes, where the node label is different from the execution node of any other execution node and corresponds to all the task queues, and in the execution nodes with the same node label, different execution nodes correspond to different task queues;

each executing node is specifically configured to: and acquiring a target task from the task queue corresponding to the execution node, and processing the target task.

Optionally, in a specific implementation manner, the obtaining a target task from a task queue corresponding to the execution node includes:

when a plurality of task queues corresponding to the execution node are available, circularly traversing each task queue corresponding to the execution node according to a preset queue traversing sequence;

for each traversed task queue, if the task queue comprises target tasks and the number of the target tasks is not less than the specified number, acquiring the specified number of target tasks from the task queue according to the sequence of the task dispatching time from early to late, and traversing the next task queue; if the task queue comprises the target tasks and the number of the included target tasks is smaller than the specified number, acquiring each target task included by the task queue according to the sequence of the task dispatching time from early to late, and traversing the next task queue; and if the target task is not included in the task queue, directly traversing the next task queue.

Optionally, in a specific implementation manner, the task queue is a rocktmq message queue; the information type of the task to be processed is a Rockettmq message; the adding the target node label for the task to be processed comprises:

and setting the property value of the property attribute of the task to be processed as the target node label.

Optionally, in a specific implementation manner, the obtaining a target task from the task queue includes:

and acquiring a target task from the task queue in a preset filtering mode.

In a second aspect, an embodiment of the present invention provides a task scheduling method, which is applied to a dispatch node in a task scheduling system, and the method includes:

acquiring a task to be processed and a target node label;

adding the target node label for the task to be processed;

sending the to-be-processed task added with the target node label to a task queue so that each execution node in the task scheduling system obtains a target task from the task queue and processes the target task; the target tasks acquired by each execution node are as follows: the target node label is the same to-be-processed task as the node label of the executing node.

In a third aspect, an embodiment of the present invention provides another task scheduling method, which is applied to each execution node in a task scheduling system, and the method includes:

acquiring a target task from a task queue; wherein each task to be processed in the task queue is: after acquiring a task to be processed and a target node label, a dispatch node in the task scheduling system dispatches the task to be processed added with the target node label to the task queue, wherein the target task is as follows: the target node label and the node label of the execution node are the same to-be-processed task;

and processing the target task.

In a fourth aspect, an embodiment of the present invention provides a task scheduling apparatus, which is applied to a dispatch node in a task scheduling system, and the apparatus includes:

the information acquisition module is used for acquiring the task to be processed and the target node label;

the label adding module is used for adding the target node label to the task to be processed;

the task dispatching module is used for dispatching the to-be-processed task added with the target node label to a task queue so that each execution node in the task scheduling system obtains a target task from the task queue and processes the target task; the target tasks acquired by each execution node are as follows: the target node label is the same to-be-processed task as the node label of the executing node.

In a fifth aspect, an embodiment of the present invention provides another task scheduling apparatus, applied to each execution node in a task scheduling system, where the apparatus includes:

the task obtaining module is used for obtaining a target task from the task queue; wherein each task to be processed in the task queue is: after acquiring a task to be processed and a target node label, a dispatch node in the task scheduling system dispatches the task to be processed added with the target node label to the task queue, wherein the target task is as follows: the target node label and the node label of the execution node are the same to-be-processed task;

and the task processing module is used for processing the target task.

In a sixth aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;

a memory for storing a computer program;

and the processor is configured to implement the steps of the task scheduling method according to the second aspect or the third aspect when executing the program stored in the memory.

In a seventh aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the task scheduling method provided in the second aspect or the third aspect are implemented.

In an eighth aspect, an embodiment of the present invention provides a computer program product containing instructions, which when run on a computer, causes the computer to perform the steps of a task scheduling method provided in the second or third aspect.

The embodiment of the invention has the following beneficial effects:

by applying the scheme provided by the embodiment of the invention, when the dispatching node in the task scheduling system acquires the task to be processed, the dispatching node can simultaneously acquire the target node label, wherein the target node label is a label designated for the task to be processed and can indicate an execution node for processing the task to be processed. Furthermore, when the to-be-processed task is dispatched to the task queue by the dispatching node, the target node label can be added to the to-be-processed task, so that the to-be-processed task dispatched to the task queue by the dispatching node is the to-be-processed task added with the target node label. Accordingly, each executing node in the task scheduling system has a node label indicating the pending task that the executing node can process. Furthermore, when the execution node acquires the to-be-processed task to be processed from the task queue, the acquired target task is the to-be-processed task with the same target node label as the node label of the execution node. The executing node may then process the retrieved target task.

Based on this, by applying the scheme provided by the embodiment of the present invention, the to-be-processed tasks and the respective execution nodes can be classified through the node labels, wherein the to-be-processed tasks having a certain node label are acquired by the execution nodes having the same node label and processed by the execution nodes. That is, each type of execution node will process a preset type of to-be-processed task, that is, the execution node that processes each type of to-be-processed task is determined, so that each type of to-be-processed task will be processed by the preset type of execution node, and the security of processing tasks can be improved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic structural diagram of a task scheduling system according to an embodiment of the present invention;

fig. 2 is a schematic flowchart of a task scheduling method applied to a dispatch node in a task scheduling system according to an embodiment of the present invention;

fig. 3 is a schematic flowchart of a task scheduling method applied to each execution node in a task scheduling system according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a task scheduling apparatus applied to a dispatch node in a task scheduling system according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of a task scheduling device applied to each execution node in a task scheduling system according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

fig. 7 is a schematic diagram of a task scheduling system according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In the related art, a plurality of public execution nodes are arranged in a private cloud, and the public execution nodes are used for processing various to-be-processed tasks of a client. However, in the above related art, the common execution node to which the to-be-processed task is allocated has an uncertainty characteristic, so that the security of the to-be-processed task cannot be guaranteed.

In order to solve the technical problem, an embodiment of the present invention provides a task scheduling system. The system comprises a dispatching node and a plurality of execution nodes, wherein each execution node is provided with a node label;

each execution node is configured to acquire a target task from the task queue and process the target task, where the target task is: the target node label is the same to-be-processed task as the node label the executing node has.

The task scheduling system can be an independent system for scheduling and processing various to-be-processed tasks, and can also be a subsystem for scheduling and processing various to-be-processed tasks included in a private cloud system and the like, which is reasonable. In addition, the dispatch node in the task scheduling system may be any device that needs to perform task scheduling, such as a desktop computer, a notebook computer, etc., and the execution node in the task scheduling system may be a physical machine that needs to acquire and process a task to be processed, such as a server, etc.; it may also be a virtual machine that needs to fetch and process the task to be processed, which is also reasonable.

As can be seen from the above, by applying the solution provided in the embodiment of the present invention, when obtaining a to-be-processed task, a dispatch node in a task scheduling system may simultaneously obtain a target node tag, where the target node tag is a tag specified for the to-be-processed task and may indicate an execution node for processing the to-be-processed task. Furthermore, when the to-be-processed task is dispatched to the task queue by the dispatching node, the target node label can be added to the to-be-processed task, so that the to-be-processed task dispatched to the task queue by the dispatching node is the to-be-processed task added with the target node label. Accordingly, each executing node in the task scheduling system has a node label indicating the pending task that the executing node can process. Furthermore, when the execution node acquires the to-be-processed task to be processed from the task queue, the acquired target task is the to-be-processed task with the same target node label as the node label of the execution node. The executing node may then process the retrieved target task.

Hereinafter, a task scheduling system according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

Fig. 1 is a schematic structural diagram of a task scheduling system according to an embodiment of the present invention. As shown in FIG. 1, the task scheduling system includes a dispatching node 110 and a plurality of execution nodes 120, and each execution node 120 has a node label;

the dispatching node 110 is configured to obtain a task to be processed and a target node label; adding a target node label for the task to be processed, and sending the task to be processed added with the target node label to a task queue;

each execution node 120 is configured to obtain a target task from the task queue and process the target task;

wherein, the target task is as follows: the target node label is the same to-be-processed task as the node label of the executing node.

The target node label obtained by the dispatch node 110 is: the target node tag may indicate an executing node for processing the pending task, that is, the pending task should be processed by an executing node having the same node tag as the target node tag.

Moreover, each executing node 120 has a node tag indicating a task to be processed that the executing node 120 can process, that is, the executing node 120 can process a task to be processed whose target node tag is the same as the node tag of the executing node 120, that is, the target node tag of the task to be processed that the executing node processes is the same as the transition of the executing node. .

The dispatch node 110 may obtain the task to be processed and the target node tag in various ways, which is not specific to the embodiment of the present invention.

For example, the dispatching node 110 may provide a human-machine interface for user operation, so that the user may send the pending task and the target node label to the dispatching node 110 through the operation of the human-machine interface.

The following are exemplary: the user can input the task to be processed, and selects a target node label appointed by the task to be processed from a plurality of node label options provided by the human-computer interaction interface.

For another example, the dispatching node 110 may obtain the pending tasks and the target node tags sent by other electronic devices.

For example, the dispatching node 110 may be communicatively connected to a client of a user, so that the user may publish a pending task and a target node tag specified for the pending task on the client used by the user, and the client may send the pending task and the target node tag to the dispatching node 110.

Further, after obtaining the to-be-processed task and the target node tag, the dispatch node 110 may add the target node tag to the to-be-processed task, and then dispatch the to-be-processed task added with the target node tag to the task queue.

In this way, a task queue including a plurality of to-be-processed tasks exists at the dispatching node 110, a target node tag is added to each to-be-processed task in the task queue, and the target node tag of each to-be-processed task is a target node tag that is specified for the to-be-processed task and is obtained when the dispatching node 110 obtains the to-be-processed task. For each executing node 120, the executing node 120 may read a target node tag of each to-be-processed task in the task queue, so as to obtain the to-be-processed task whose target node tag is the same as the node tag of the executing node 120, that is, obtain the target task. The executing node 120 may then process the retrieved target task.

Optionally, for each execution node 120, the execution node 120 may obtain the target task from the task queue according to a predetermined period. Specifically, the method comprises the following steps: after the executing node 120 acquires the target task from the task queue, the executing node 120 first processes the acquired target task, and further, after the acquired target task is processed, waits for a time interval specified by a predetermined period, and then acquires a new target task from the task queue again; when the executing node 120 does not acquire the target task from the task queue, the executing node 120 may wait for a time interval specified by a predetermined period before reading the task queue again to determine whether the target task can be acquired.

For example, the time interval specified by the predetermined period is 5s, the execution node 120 acquires the target task from the task queue, the execution node 120 first processes the acquired target task, and after the acquired target processing is completed, the execution node 120 waits for 5s and then reads the task queue again to determine whether a new target task can be acquired; at this time, if the execution node 120 does not acquire a new target task from the task queue, the execution node 120 waits for 5 seconds again and then reads the task queue again to determine whether the new target task can be acquired.

In addition, in order to improve the processing efficiency of the to-be-processed tasks and ensure that each of the to-be-processed tasks can be processed quickly, each of the execution nodes 120 may obtain a plurality of target tasks from the task queue each time the target task is obtained from the task queue.

Optionally, for each execution node, when the task queue includes the target tasks and the number of the included target tasks is not less than the specified number, the execution node 120 may obtain the specified number of target tasks from the task queue; when the task queue includes target tasks and the number of included target tasks is less than the specified number, each execution node 120 may obtain all target tasks from the task queue.

Optionally, in a specific implementation manner, the task queue may be a rocktmq message queue, and the information type of the task to be processed may be a rocktmq message;

the rocktmq is a message middleware in a queue model of an open-source distributed system, and has the characteristics of high performance, high reliability, high real-time performance and distribution.

Correspondingly, in this specific implementation manner, the manner in which the dispatching node 110 adds the target node tag to the task to be processed includes:

the dispatching node 110 sets the property value of the property attribute of the task to be processed as the target node tag.

Property is an attribute which each message has in a rocktmq message queue and is used for retrieving messages.

Further, on the basis of the foregoing specific implementation manner, optionally, in another specific implementation manner, a manner in which each execution node 120 acquires a target task from a task queue includes:

each execution node 120 obtains a target task from the task queue in a preset filtering manner.

Optionally, the preset filtering manner may be a filtering manner of bySql. The bySql is a screening method used by the message selector, and the message is screened in a manner similar to mysql.

Based on this, by applying the scheme provided by the embodiment of the present invention, the tasks to be processed and the execution nodes can be classified through the node labels, wherein the task having a certain node label is acquired by the execution node having the same node label and processed by the execution node. That is, each type of execution node will process a preset type of to-be-processed task, that is, the execution node that processes each type of to-be-processed task is determined, so that each type of to-be-processed task will be processed by the preset type of execution node, and the security of processing tasks can be improved.

It is understood that the dispatching node 110 may acquire more pending tasks in a short time, and there may be more pending tasks with the same target node label in the pending tasks, that is, the executing node 120 capable of processing the pending tasks with the same target node label needs to process more pending tasks in a short time.

Based on this, optionally, in a specific implementation manner, the number of the task queues may be multiple, and the execution nodes 120 with the same node label exist in the multiple execution nodes 120, where the node label is different from the execution node 120 of any other execution node and corresponds to all the task queues, and in the execution nodes 120 with the same node label, different execution nodes 120 correspond to different task queues;

each executing node 120 is specifically configured to: and acquiring the target task from the task queue corresponding to the execution node 120, and processing the acquired target task.

In this specific implementation manner, the dispatching node 110 may dispatch each to-be-processed task added with the target node tag to one of the task queues according to a preset arbitrary dispatching rule. For example, the to-be-processed tasks added with the target node labels can be dispatched to one of the plurality of task queues in turn according to an average dispatching rule; for another example, the to-be-processed task added with the target node label may be dispatched to one of the plurality of task queues including fewer to-be-processed tasks according to a queue balancing dispatch rule. This is all reasonable.

In addition, in this specific implementation manner, according to the node labels that each execution node 120 has, the multiple execution nodes 120 included in the task scheduling system provided in the embodiment of the present invention may be divided into multiple execution node groups, where each execution node group includes at least one execution node 120, and when one execution node group includes multiple execution nodes 120, the multiple execution nodes 120 included in the one execution node group have the same node label.

Furthermore, in order to ensure load balancing among each executing node 120 and ensure that the task scheduling system provided by the embodiment of the present invention has higher processing efficiency, so that each task to be processed can be processed as soon as possible, for an executing node group including a plurality of executing nodes 120, at least one task queue of the plurality of task queues may be allocated to each executing node 120 in the executing node group in advance. Thus, each task queue of the plurality of task queues corresponds to one executing node 120 of the executing node group, and the task queues allocated to different executing nodes 120 of the executing node group are different. That is, different execution nodes 120 correspond to different task queues in the execution nodes 120 with the same node label.

For an executing node group including a plurality of executing nodes 120, the plurality of task queues may be equally allocated to each executing node 120 included in the executing node group, or the task queues may be allocated to each executing node 120 according to the processing capability of each executing node 120 included in the executing node group. This is all reasonable.

Accordingly, for an executing node group including only one executing node 120, node labels of the executing nodes 120 included in the executing node group are different from node labels of any executing node 120 in other executing nodes 120 included in the scheduling system according to the embodiment of the present invention, and therefore, the executing nodes 120 included in the executing node group need to acquire target tasks from all task queues in the plurality of task queues. I.e., execution node 120 with a node tag different from any other execution node corresponds to all task queues.

Based on this, in this specific implementation, each executing node 120 may be specifically configured to: and acquiring a target task from the task queue corresponding to the execution node, and processing the acquired target task.

Further, on the basis of the foregoing specific implementation manner, optionally, in a specific implementation manner, a manner in which each execution node 120 acquires a target task from a task queue corresponding to the execution node 120 and processes the acquired target task may include:

when a plurality of task queues corresponding to the execution node 120 are provided, circularly traversing each task queue corresponding to the execution node 120 according to a predetermined queue traversal order;

Specifically, in the present embodiment, for an executing node 120 corresponding to a plurality of task queues, when allocating a task queue to the executing node 120, the traversing order of each task queue may be preset.

Thus, the executing node 120 can cycle through each task queue corresponding to the executing node 120 according to the predetermined queue traversal order.

Also, upon traversing to each task queue, the execution node 120 may retrieve a specified number of target tasks from the task queue. By circularly traversing each task queue, the execution node can acquire a specified number of target tasks from the corresponding plurality of task queues. Further, when the specified number of target tasks are acquired, the task acquisition process may be ended, and the execution node 120 stops acquiring new target tasks and starts processing the acquired specified number of target tasks.

It can be understood that, when traversing to each task queue, there may be a case where the number of target tasks included in the task queue does not satisfy the specified number. Further, the number of all the target tasks included in each task queue may not satisfy the predetermined number.

Based on this, in this particular implementation, for each task queue traversed:

if the task queue includes the target tasks and the number of the included target tasks is not less than the specified number, the execution node 120 may obtain the specified number of target tasks from the task queue according to the sequence of the task distribution time from early to late, and traverse the next task queue;

if the task queue includes target tasks and the number of the included target tasks is less than the specified number, the execution node 120 may obtain each target task included in the task queue according to the sequence of the task distribution time from early to late, and traverse the next task queue; that is, if the task queue includes target tasks and the number of included target tasks is less than the specified number, the execution node 120 may obtain all target tasks included in the task queue according to the order from early to late of the task dispatch time;

if the target task is not included in the task queue, the execution node 120 may directly traverse the next task queue.

Further, a predetermined number of tasks to be processed that each execution node 120 can process at one time may be set in advance in consideration of the processing capabilities of the respective execution nodes 120.

Thus, when the number of the target tasks acquired by the execution node 120 satisfies the predetermined number, the execution node 120 may stop traversing the corresponding task queue and start processing the acquired target tasks. Alternatively, when the number of the target tasks acquired by the execution node 120 does not satisfy the predetermined number, but the target node tags of all the to-be-processed tasks in each task queue that the execution node 120 has read are different from the node tags of the execution node 120, the execution node 120 may also stop traversing the corresponding task queue and start processing the acquired target tasks.

For example, the task queues corresponding to an execution node 120 are queue 1, queue 2, and queue 3, the predetermined queue traversal order is queue 1, queue 2, and queue 3 in turn, the designated number is 1, and the predetermined number is 32.

In this way, the executing node 120 may first read the target node tag of each to-be-processed task in the queue 1, and since the queue 1 includes a plurality of target tasks, the executing node 120 may obtain 1 target task from the queue 1 according to the sequence of the task dispatch time from early to late; then, the executing node 120 reads the target node tag of each to-be-processed task in the queue 2, because the queue 2 does not include the target task, the executing node 120 can directly read the target node tag of each to-be-processed task in the queue 3, and because the queue 3 includes 1 target task, the executing node can obtain the target task from the queue 3; then, the executing node 120 may read the target node tag of each to-be-processed task in the queue 1 again, and obtain 1 target task from the queue 1 according to the number of target tasks included in the queue 1 and the order of the task dispatch time from early to late.

Repeating the above steps until the execution node 120 acquires 32 target tasks, the execution node 120 may stop traversing the corresponding task queue and start processing the acquired target tasks; or, until the target node labels of all the to-be-processed tasks read from the queue 1, the queue 2, and the queue 3 by the execution node are different from the node label possessed by the execution node 120, the execution node 120 may stop traversing the corresponding task queue and start processing the acquired target task.

Accordingly, in this specific implementation manner, when the task queue corresponding to the execution node 120 is one, the execution node may obtain the target task from the task queue according to the order of the task dispatch time from early to late.

When the number of the target tasks acquired by the execution node 120 satisfies the predetermined number, the execution node 120 may stop acquiring the target tasks and start processing the acquired target tasks. Alternatively, when the number of the target tasks acquired by the execution node 120 does not satisfy the predetermined number, but the target node tags of all the to-be-processed tasks in the task queue that have been read by the execution node 120 are different from the node tags that the execution node 120 has, the execution node 120 may also stop acquiring the target tasks and start processing the acquired target tasks.

In order to ensure timely and smooth processing of the to-be-processed task, a backup node may be set for the execution node 120, so that when the execution node 120 fails and cannot normally acquire and process the to-be-processed task, the to-be-processed task acquisition and processing work may be completed by using the backup node.

Based on this, optionally, in a specific implementation manner, the number of the task queues is one, and multiple execution nodes 120 with the same node label exist in the multiple execution nodes 120.

The plurality of execution nodes 120 with the same node label include a main execution node and at least one backup execution node. That is, in this specific implementation manner, one execution node group may include one main execution node and at least one backup execution node.

Correspondingly, in this specific implementation manner, for an execution node group including one main execution node and at least one backup execution node, when the main execution node included in the execution node group is normally operated, the main execution node is configured to obtain a target task from a task queue and process the obtained target task.

And when the main execution node included in the execution node group can not normally run, starting a backup execution node included in the execution node group, wherein the backup execution node is used for acquiring the target task from the task queue and processing the acquired target task.

When the execution node group comprises a plurality of backup execution nodes, the priority of each backup execution node can be preset, so that when the main execution node cannot normally run, the backup execution node with the highest priority in each backup execution node which can normally run is started, and the started execution node is used for acquiring the target task from the task queue and processing the acquired target task.

For a better understanding of a task scheduling system provided by the embodiment of the present invention, a specific embodiment shown in fig. 7 is used for illustration.

The task message in fig. 7 is a to-be-processed task acquired by a dispatch node, the node group attribute is a target node tag specified for the to-be-processed task, Queue is a task Queue, and a job execution machine is an execution node. The node tag of each work execution machine in the node group G1 is id 1, and the node tag of each work execution machine in the node group G2 is id 2.

The number of the task queues is 4, and the JOBs 1 to 4 are the tasks to be processed belonging to the task queues 1 to 4 respectively. The target node labels of JOB1-JOB4 are ("groupID", 1), ("groupID", 2), ("groupID", 1), and ("groupID", 2), respectively.

The JOB execution machines 1 to 4 can respectively acquire JOB1 to JOB4 from the corresponding JOB queues 1 to 4.

Corresponding to the task scheduling system provided by the embodiment of the invention, the embodiment of the invention also provides a task scheduling method. The method is applied to the dispatching node in the task scheduling system.

Fig. 2 is a schematic flowchart of a task scheduling method applied to a dispatch node according to an embodiment of the present invention, and as shown in fig. 2, the method may include the following steps:

s201: acquiring a task to be processed and a target node label;

s202: adding a target node label for a task to be processed;

s203: sending the to-be-processed task added with the target node label to a task queue so that each execution node in the task scheduling system obtains a target task from the task queue and processes the target task;

the target tasks acquired by each execution node are as follows: the target node label is the same to-be-processed task as the node label of the executing node.

Based on this, by applying the scheme provided by the embodiment of the present invention, the to-be-processed tasks and the respective execution nodes can be classified through the node labels, wherein the to-be-processed tasks having a certain node label are acquired by the execution nodes having the same node label and processed by the execution nodes. That is, each type of execution node will process a preset type of tasks to be processed, that is, the execution node that processes each type of tasks to be processed is determined, so that each type of tasks to be processed will be processed by the preset type of execution node, and thus, the security of processing tasks can be improved.

Optionally, in a specific implementation manner, the task queue is a rocktmq message queue; the information type of the task to be processed is a Rockettmq message; in the step S202, adding the target node label to the task to be processed may include the following steps:

and setting the property value of the property attribute of the task to be processed as a target node label.

Corresponding to the task scheduling system provided by the embodiment of the invention, the embodiment of the invention also provides a task scheduling method. The method is applied to each execution node in the task scheduling system.

Fig. 3 is a flowchart illustrating a task scheduling method applied to each execution node according to an embodiment of the present invention, as shown in fig. 3, the method may include the following steps:

s301: acquiring a target task from a task queue;

wherein, each task to be processed in the task queue is: after acquiring a task to be processed and a target node label, a dispatching node in the task scheduling system dispatches the task to be processed added with the target node label to a task queue, wherein the target task is as follows: the target node label and the node label of the execution node are the same to-be-processed task;

s302: and processing the target task.

the step S301 of obtaining the target task from the task queue may include the following steps:

and acquiring a target task from the task queue corresponding to the execution node.

Optionally, in a specific implementation manner, the obtaining of the target task from the task queue corresponding to the execution node may include the following steps:

Optionally, in a specific implementation manner, in the step S301, obtaining, from the task queue, the to-be-processed task whose added target node label is the node label that the executing node has, may include the following steps:

and acquiring the target task from the task queue in a preset filtering mode.

Corresponding to the task scheduling system provided by the embodiment of the invention, the embodiment of the invention also provides a task scheduling device. The device is applied to the dispatching node in the task scheduling system.

Fig. 4 is a schematic structural diagram of a task scheduling apparatus applied to a dispatch node according to an embodiment of the present invention, and as shown in fig. 4, the apparatus may include the following modules:

an information obtaining module 410, configured to obtain a task to be processed and a target node tag;

a tag adding module 420, configured to add a target node tag to the task to be processed;

and the task dispatching module 430 is configured to dispatch the to-be-processed task added with the target node tag to a task queue, so that each execution node in the task scheduling system obtains the target task from the task queue and processes the target task.

Optionally, in a specific implementation manner, the task queue is a rocktmq message queue; the information type of the task to be processed is a Rockettmq message; the tag adding module 420 is specifically configured to set a property value of the property attribute of the task to be processed as a target node tag.

Corresponding to the task scheduling system provided by the embodiment of the invention, the embodiment of the invention also provides a task scheduling device. The device is applied to each execution node in the task scheduling system.

Fig. 5 is a schematic structural diagram of a task scheduling apparatus applied to each execution node according to an embodiment of the present invention, and as shown in fig. 5, the apparatus may include the following modules:

a task obtaining module 510, configured to obtain a target task from a task queue;

and a task processing module 520, configured to process the target task.

Optionally, in a specific implementation manner, the number of the task queues is multiple, and there are execution nodes with the same node label in the multiple execution nodes, where the node label is different from the execution node of any other execution node and corresponds to all the task queues, and in the execution nodes with the same node label, different execution nodes correspond to different task queues; the task obtaining module 510 includes:

and the task obtaining submodule is used for obtaining the target task from the task queue corresponding to the execution node.

Optionally, in a specific implementation manner, the task obtaining submodule is specifically configured to:

Optionally, in a specific implementation manner, the task obtaining module is specifically configured to:

and acquiring the target task from the task queue in a preset filtering mode.

Corresponding to the task scheduling system provided by the embodiment of the present invention, an embodiment of the present invention further provides an electronic device, as shown in fig. 6, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604, where the processor 601, the communication interface 602, and the memory 603 complete mutual communication through the communication bus 604,

a memory 603 for storing a computer program;

the processor 601 is configured to, when executing the program stored in the memory 603, implement any of the steps of the task scheduling method applied to the dispatch node in the task scheduling system according to the foregoing embodiments of the present invention, or any of the steps of the task scheduling method applied to each execution node in the task scheduling system according to the foregoing embodiments of the present invention.

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

In another embodiment of the present invention, a computer-readable storage medium is further provided, where a computer program is stored in the computer-readable storage medium, and when executed by a processor, the computer program implements any of the steps of the task scheduling method applied to the dispatch node in the task scheduling system, or any of the steps of the task scheduling method applied to each execution node in the task scheduling system, provided in the above embodiments of the present invention.

In another embodiment of the present invention, a computer program product is also provided, which includes instructions, when the computer program product runs on a computer, to make the computer execute any of the steps of the task scheduling method applied to the dispatch node in the task scheduling system or any of the steps of the task scheduling method applied to each execution node in the task scheduling system provided by the above embodiments of the present invention.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the method embodiment, the apparatus embodiment, the electronic device embodiment, the computer-readable storage medium embodiment, and the computer program product embodiment, since they are substantially similar to the system embodiment, the description is relatively simple, and reference may be made to some descriptions of the method embodiment for relevant points.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A task scheduling system, comprising a dispatch node and a plurality of execution nodes, each execution node having a node label;

2. The system according to claim 1, wherein the number of the task queues is multiple, and there are execution nodes with the same node label in the multiple execution nodes, where the node label is different from the execution node of any other execution node and corresponds to all the task queues, and in the execution nodes with the same node label, different execution nodes correspond to different task queues;

3. The system according to claim 2, wherein said obtaining the target task from the task queue corresponding to the execution node comprises:

4. The system of claim 1, wherein the task queue is a rocktmq message queue; the information type of the task to be processed is a Rockettmq message; adding the target node label to the task to be processed, including:

5. The system of claim 4, wherein said retrieving a target task from the task queue comprises:

and acquiring a target task from the task queue in a preset filtering mode.

6. A task scheduling method is applied to a dispatching node in a task scheduling system, and comprises the following steps:

acquiring a task to be processed and a target node label;

adding the target node label for the task to be processed;

7. A task scheduling method is applied to each execution node in a task scheduling system, and comprises the following steps:

and processing the target task.

8. A task scheduling apparatus, applied to a dispatch node in a task scheduling system, the apparatus comprising:

9. A task scheduling apparatus applied to each execution node in a task scheduling system, the apparatus comprising:

and the task processing module is used for processing the target task.

10. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;

a memory for storing a computer program;

a processor for implementing the method steps of claim 6 or 7 when executing a program stored in the memory.

11. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of claim 6 or 7.