WO2020232875A1 - 一种基于Actor模型的任务调度方法、装置及存储介质 - Google Patents

一种基于Actor模型的任务调度方法、装置及存储介质 Download PDF

Info

Publication number
WO2020232875A1
WO2020232875A1 PCT/CN2019/102741 CN2019102741W WO2020232875A1 WO 2020232875 A1 WO2020232875 A1 WO 2020232875A1 CN 2019102741 W CN2019102741 W CN 2019102741W WO 2020232875 A1 WO2020232875 A1 WO 2020232875A1
Authority
WO
WIPO (PCT)
Prior art keywords
task
node
scheduling
target
scheduling node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/102741
Other languages
English (en)
French (fr)
Inventor
袁康
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Publication of WO2020232875A1 publication Critical patent/WO2020232875A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication

Definitions

  • This application relates to the field of task scheduling, and in particular to a task scheduling method, device and storage medium based on an Actor model.
  • Task scheduling is often deadlocked due to concurrent access to a critical resource by multiple threads or processes.
  • the critical resource can be locked to establish a mutual exclusion synchronization mechanism between threads or processes to ensure the critical resource.
  • the embodiment of the present application provides a task scheduling method based on the Actor model, which can realize distributed task scheduling and processing without using locks.
  • an embodiment of the present application provides a task scheduling method based on an Actor model, the method including:
  • an Actor model is created for the target task, the Actor model includes a plurality of working nodes, and the plurality of working nodes are communicated through messages In communication, at least two working nodes among the plurality of working nodes are respectively distributed on different terminals, and the plurality of working nodes includes a scheduling node and at least one task node, and the scheduling node is used to call the task node Perform tasks
  • an embodiment of the present application provides a task scheduling device, the task scheduling device includes a unit for executing the task scheduling method of the first aspect, the task scheduling device includes:
  • the detection unit is used to detect whether there is a new target task in the task cluster
  • the creation unit is used to create an Actor model for the target task when it is detected that there is a newly added target task in the task cluster.
  • the Actor model includes a plurality of working nodes, and messages are passed between the plurality of working nodes. Communication, at least two of the multiple working nodes are distributed on different terminals, the multiple working nodes include a scheduling node and at least one task node, and the scheduling node is used to call all The task node executes the task;
  • An execution unit configured to use the scheduling node to invoke the at least one task node to execute the target task in a message transfer manner
  • the deleting unit is configured to delete the target task in the task cluster after detecting that the target task is completed.
  • the embodiments of the present application provide another task scheduling device, including a processor and a memory, the processor and the memory are connected to each other, wherein the memory is used to store information that supports the task scheduling device to perform the above task scheduling method.
  • a computer program, the computer program including program instructions, and the processor is configured to call the program instructions to execute:
  • an Actor model is created for the target task, the Actor model includes a plurality of working nodes, and the plurality of working nodes are communicated through messages In communication, at least two working nodes among the plurality of working nodes are respectively distributed on different terminals, and the plurality of working nodes includes a scheduling node and at least one task node, and the scheduling node is used to call the task node Perform tasks
  • an embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the program instructions are executed by a processor to The task scheduling method of any one of the foregoing first aspect to the first aspect is executed.
  • the present application can realize complex distributed task scheduling without resorting to locks, ensure that data competition and deadlock will not occur in the distributed task scheduling process, and greatly improve the concurrent performance of distributed task processing.
  • FIG. 1 is a schematic flowchart of a task scheduling method provided by an embodiment of the present application
  • FIG. 2 is a schematic flowchart of a task scheduling method provided by another embodiment of the present application.
  • FIG. 3 is a schematic block diagram of a task scheduling device provided by an embodiment of the present application.
  • Fig. 4 is a structural block diagram of a task scheduling device provided by an embodiment of the present application.
  • This application is mainly applied to task scheduling devices, which can be traditional task scheduling devices, large storage systems, desktop computers, notebook computers, tablet computers, handheld computers, smart phones, portable digital players, smart watches, and smart bracelets Wait, this application does not restrict this.
  • the terminals described in the embodiments of this application include, but are not limited to, devices with communication functions, smart phones, tablet computers, notebook computers, desktop computers, portable digital players, smart bracelets, smart watches, etc.
  • the terminal sends data to the task scheduling device, it records and transmits the characteristics of the data according to a preset format, where the characteristics of the data include time, location, type, etc.
  • the task scheduling method may include:
  • the task cluster refers to the collection of tasks to be processed, and different tasks in the task cluster may be proposed by different terminals, and the tasks in the task cluster may be distributed tasks processed by multiple terminals. It may be a stand-alone task processed by only one terminal.
  • the detection process refers to whether the task scheduling device on the local side regularly detects whether there is a new task in the task cluster, or if a task is added to the task cluster, the local device can receive it. To a message indicating a new task in the task cluster.
  • the above-mentioned target task is not an arbitrary task newly added in the task cluster, but a distributed task newly added in the task cluster. Specifically, whether there is a newly-added target task in the above-mentioned detection task cluster It refers to detecting a newly added task in the task cluster, and if it is determined that the newly added task is a distributed task, it is determined that the newly added target task is detected in the task cluster.
  • the process of detecting that the newly added task in the task cluster is a distributed task refers to obtaining the type of the newly added task in the task cluster, if the type of the newly added task is a distributed task , It means that the type of the newly added task is a distributed task, or the difficulty level of the newly added task is obtained. If it is determined that the difficulty level of the newly added task exceeds the preset level, it means that the newly added task requires Take up more computing resources, and determine that the newly added task is a distributed task, that is, the task whose difficulty level exceeds the preset level is regarded as the target task, and then the distributed task processing method is used to process it .
  • the embodiment of the present application only determines that there is a newly added target task in the task cluster when it is determined that a distributed task is added to the task cluster, and the newly added distributed task in the task cluster detected above is the target task.
  • an Actor model is created for the target task.
  • the Actor model includes multiple working nodes, and the multiple working nodes include a scheduling node and at least one task node.
  • the Actor model is a model used for task processing.
  • the Actor model is a conceptual model used for processing concurrent computing. It defines a series of general rules for how system components should act and interact.
  • the Actor model contains multiple Actors. Actors are called working nodes in the embodiments of this application. Each working node is the most basic computing unit for task processing in the Actor model. It can receive a message and perform calculations based on it. .
  • the above creation of an Actor model containing multiple work nodes based on the execution content of the target task refers to the decomposition of the target task, and then based on the task content of the target task after the decomposition, multiple work nodes are created to execute the corresponding task content , And the work node used to execute the target task is the task node.
  • the task When the task is executed, multiple task nodes may be executed in parallel. If the task node is not scheduled properly, it may cause problems between multiple task nodes. Resource competition, so in order to determine the order of scheduling and avoid resource competition, the above Actor model also includes a working node for performing task scheduling.
  • the working node for performing task scheduling is called a scheduling node in the embodiment of the application.
  • the scheduling node is used to arrange the execution order of each task node, and schedule each task node to execute tasks in an orderly manner.
  • the Actor model created for the aforementioned target task includes multiple work nodes, and the multiple work nodes include one scheduling node and at least one task node.
  • the scheduling node and task node used in the embodiment of this application are both actors, and the Actors communicate through message passing, no matter it is in task scheduling or task execution, they will not The problem of resource competition arises.
  • the way of message passing means that in the message passing system, the data exchange between processes is based on formatted messages (Message).
  • Message formatted messages
  • An important feature of Actors is that Actors are isolated from each other, and they are not each other. With shared memory, an actor can maintain a private state, and this state cannot be changed by another actor, and message passing also ensures asynchronous processing between actors, so there is no data competition between actors. If there is no directly accessible shared space between the communicating processes, the message transfer method provided by the operating system must be used to implement process communication.
  • Actors share resources through message passing.
  • an Actor needs another Actor's resources, it only needs to send the message to another Actor, but does not care when the other Actor will process it. Therefore, the Actor
  • the messages between are transmitted asynchronously, similar to mail communication.
  • each actor has an address, so they can send messages to each other.
  • an actor can only process messages sequentially. In other words, the other three actors have sent a message to an actor. This actor can only process one message at a time.
  • Actors separate the accessible resources of different Actors through message passing, and guarantee the asynchronous processing of messages between Actors, thereby avoiding the use of locks to deal with data race issues.
  • the biggest advantage of the message passing mechanism is that it will not produce data race.
  • the embodiment of the present application is applied to the scheduling and processing of distributed tasks. It can be seen that because the working nodes located on different terminals When a worker node is working, it will only occupy the computing resources of the terminal where it is located. Therefore, multiple terminals in a terminal cluster jointly process a distributed task. Compared with stand-alone processing, distributed processing can greatly improve the computing power and calculation. speed. It should be noted that since the Actor in the Actor model is used as the working node in the embodiment of this application, distributed task scheduling and processing can be realized without the need of a distributed lock. This is because the Actor only pays attention to the received messages.
  • the above-mentioned multiple work sections further include at least one backup scheduling node, and the backup scheduling node is used to replace a down scheduling node. If it is detected that the above-mentioned scheduling node is down, a backup scheduling node is selected from the above-mentioned at least one backup scheduling node as a new scheduling node.
  • the Actor model created for the target task also includes at least one backup scheduling node.
  • the backup scheduling node is used to replace the downtime when the scheduling node is down.
  • the new scheduling node can start the scheduling from the beginning, or continue to perform the scheduling following the scheduling progress of the down scheduling node before the downtime.
  • the backup scheduling node is created according to the initialization information of the scheduling node, so the backup scheduling node can replace the scheduling node.
  • the backup scheduling node can be activated when the scheduling node is down, so as to solve the problem that the scheduling node may be down.
  • only one backup scheduling node when the scheduling node is down, only one backup scheduling node included in the above Actor model will be directly used as a new scheduling node.
  • the above Actor model includes multiple standby scheduling nodes, when the scheduling node is down, select any one of the multiple standby scheduling nodes as the new scheduling node, or obtain each standby scheduling node separately And determine the earliest creation time among the backup scheduling nodes included in the Actor model, and then use the earliest creation time as the new scheduling node.
  • the standby scheduling node used to replace the down scheduling node is determined according to the time of the standby scheduling time, the advantage is that it is convenient for the local device to read the creation time of the currently used scheduling node to quickly determine that it is down. The number of calling nodes in order to know the downtime of the calling node.
  • creating the Actor model for the target task mentioned above refers to obtaining the label of the target task.
  • the label is used to indicate the importance of the target task, and the number of targets corresponding to the label of the target task is obtained.
  • To indicate the number of standby scheduling nodes in the Actor model and then create an Actor model including the target number of standby scheduling nodes.
  • the number of standby scheduling nodes in the Actor model is related to the importance of the target task.
  • the more important the target task the more standby scheduling nodes are created for the target task.
  • the label of the target task is obtained, and the target quantity corresponding to the label of the target task is obtained in the database, and then a backup scheduling node containing the target quantity is created.
  • the corresponding relationship between the label of the task and the number of standby scheduling nodes is stored in the database.
  • the above-mentioned multiple working nodes further include at least one monitoring node, and the monitoring node is used to initialize the scheduling node that is down. Specifically, if it is detected that the scheduling node is down, the monitoring node is used to The down scheduling node is initialized, so that the down scheduling node returns to normal.
  • the embodiment of the present application describes another method for solving the downtime of the scheduling node, that is, if the scheduling node is down, the monitoring device is used to initialize the down scheduling node. Specifically, after detecting that the scheduling node is down, the monitoring node is used to initialize the down scheduling node according to the initialization information of the scheduling node, so that the down scheduling node returns to normal. Alternatively, the monitoring node is used to restore the downtime scheduling node to the normal state at the moment before the downtime. Specifically, after detecting the downtime of the above scheduling node, obtain the downtime scheduling node recorded in the database. The state information at a time before the machine is used, and the monitoring node is used to restore the scheduling node according to the state information of the scheduling node at the time before the downtime, so that the scheduling node is restored to the normal state corresponding to the state information.
  • an Actor model for the target task it is determined whether to create a monitoring node in the above Actor model according to the importance of the target task. Specifically, after the target task is detected in the task cluster, a label of the target task is obtained, and the label is used to indicate the importance of the target task. Get the importance level of the target task corresponding to the tag of the target task. If the importance level of the target task exceeds the preset level, create the above monitoring node when creating the Actor model. If the importance level of the target task does not exceed the preset level, Then the above monitoring node is not created.
  • the aforementioned monitoring node is also used to initialize the down task node, so that the down task node returns to normal.
  • the above-mentioned Actor model created for the target task includes at least one standby task node for each task node included in the model, and the standby task node is used to replace a down task node. That is to say, a task node in the Actor model corresponds to a standby task node, and the task node and the standby task node have a one-to-one correspondence.
  • the solution also includes: detecting the down task node; if the down task node is detected, selecting the standby task node corresponding to the down task node from at least one standby task node in the Actor model, And use the selected standby task node as the new task node.
  • the scheduling node is used to call the at least one task node to execute the target task by means of message transmission. Specifically, the scheduling node sends a message for triggering the task node to perform the task to the task node, so that the task node executes the task. After the task node completes the task, it will feed back a message indicating the completion of the task to the scheduling node, so that the scheduling The node obtains the task node that completes the task, and continues to schedule and execute other task nodes in the order of execution, thereby finally completing the target task.
  • the target task is deleted from the task cluster, and the next target task in the task cluster is detected and executed.
  • the number of created worker nodes is detected, and the creation time of the created worker node is obtained; if the created job is If the number of nodes is greater than the preset number, the created working nodes are sequentially deleted according to the creation time from early to late until the number of remaining created working nodes is less than or equal to the The preset number.
  • the number of working nodes that have been created is detected. If the number of nodes exceeds the preset number, the working nodes are deleted until the number of remaining working nodes is less than or equal to the preset number.
  • the work nodes are deleted in the order of creation time from early to late. This is because the work nodes that are created earlier are less likely to be used again than those created later.
  • the detailed process of deleting the created working nodes is to detect the number of created working nodes and obtain the creation time of the created working nodes. If the number of created working nodes is greater than the preset number, Determine the time node with the earliest creation time among the created working nodes, and delete the working node with the earliest creation time among the created working nodes, and then if the created working node is deleted, the job with the latest creation time is deleted After the node, if the number of created working nodes is still greater than the preset number, the execution of the working node with the earliest creation time among the above determined working nodes will be triggered to delete the working node with the earliest creation time among the created working nodes Until the number of created work nodes is less than or equal to the preset number after deleting the work node with the latest creation time.
  • an Actor model including a scheduling node and a task node is established for the target task, and the scheduling node is used to schedule the task node to execute the target task.
  • Delete the target task it can be seen that in the task scheduling method provided by this application, since the scheduling node and the task node are both actors, the scheduling node and the task node communicate through information transmission, and the advantage of information transmission is that There will be a problem of data competition. When processing tasks, there is no need to synchronize and block waiting. Therefore, this application can realize complex distributed task scheduling without the help of locks, and greatly improve the concurrent performance of distributed task processing. It can be seen that the embodiment of the present application provides an efficient task scheduling method.
  • the task scheduling method may include:
  • the specific detection process refers to the task scheduling device at the local end periodically detecting whether there is a new task in the task cluster, or whether there is a new task in the task cluster.
  • Task the local device can receive a message indicating a new task in the task cluster.
  • the task cluster refers to the collection of tasks to be processed, and different tasks in the task cluster may be proposed by different terminals, and the tasks in the task cluster may be distributed tasks processed by multiple terminals. It may be a stand-alone task handled by only one terminal.
  • the newly added task is a target task. Specifically, it is determined whether the newly added task is a distributed task, and if so, it is determined that the newly added target task is detected in the task cluster.
  • the process of detecting that the newly added task in the task cluster is a distributed task refers to obtaining the type of the newly added task in the task cluster, if the type of the newly added task is a distributed task , It means that the type of the newly added task is a distributed task, or the difficulty level of the newly added task is obtained. If it is determined that the difficulty level of the newly added task exceeds the preset level, it means that the newly added task requires Take up more computing resources, and determine that the newly added task is a distributed task, that is, the task whose difficulty level exceeds the preset level is regarded as the target task, and then the distributed task processing method is used to process it .
  • the embodiment of the present application only determines that there is a newly added target task in the task cluster when it is determined that a distributed task is added to the task cluster, and the newly added distributed task in the task cluster detected above is the target task.
  • the actor model includes multiple work nodes, and the multiple work nodes include a scheduling node, at least one task node, and at least A spare scheduling node.
  • the Actor model is a model used for task processing.
  • the Actor model is a conceptual model used for processing concurrent computing. It defines a series of general rules for how system components should act and interact.
  • the Actor model contains multiple Actors. Actors are called working nodes in the embodiments of this application. Each working node is the most basic computing unit for task processing in the Actor model. It can receive a message and perform calculations based on it. .
  • the above creation of an Actor model containing multiple work nodes based on the execution content of the target task refers to the decomposition of the target task, and then based on the task content of the target task after the decomposition, multiple work nodes are created to execute the corresponding task content , And the work node used to execute the target task is the task node.
  • the task When the task is executed, multiple task nodes may be executed in parallel. If the task node is not scheduled properly, it may cause problems between multiple task nodes. Resource competition, so in order to determine the order of scheduling and avoid resource competition, the above Actor model also includes a working node for performing task scheduling.
  • the working node for performing task scheduling is called a scheduling node in the embodiment of the application.
  • the scheduling node is used to arrange the execution order of each task node, and schedule each task node to execute tasks in an orderly manner.
  • the Actor since there is only one scheduling node, in order to deal with the situation of the scheduling node, the Actor also includes at least one backup scheduling node, and the backup scheduling node is used to replace the down scheduling node.
  • the Actor model created for the above-mentioned target task includes multiple working nodes, and the multiple working nodes include one scheduling node, at least one task node, and at least one standby scheduling node.
  • the scheduling node and task node used in the embodiment of this application are both actors, and the Actors communicate through message passing, no matter it is in task scheduling or task execution, they will not The problem of resource competition arises.
  • the way of message passing means that in the message passing system, the data exchange between processes is based on formatted messages (Message).
  • Message formatted messages
  • An important feature of Actors is that Actors are isolated from each other, and they are not each other. With shared memory, an actor can maintain a private state, and this state cannot be changed by another actor, and message passing also ensures asynchronous processing between actors, so there is no data competition between actors. If there is no directly accessible shared space between the communicating processes, the message transfer method provided by the operating system must be used to implement process communication.
  • Actors share resources through message passing.
  • an Actor needs another Actor's resources, it only needs to send the message to another Actor, but does not care when the other Actor will process it. Therefore, the Actor
  • the messages between are transmitted asynchronously, similar to mail communication.
  • each actor has an address, so they can send messages to each other.
  • an actor can only process messages sequentially. In other words, the other three actors have sent a message to an actor. This actor can only process one message at a time.
  • Actors separate the accessible resources of different Actors through message passing, and guarantee the asynchronous processing of messages between Actors, thereby avoiding the use of locks to deal with data race issues.
  • the biggest advantage of the message passing mechanism is that it will not produce data race.
  • the embodiment of the present application is applied to the scheduling and processing of distributed tasks. It can be seen that because the working nodes located on different terminals When a worker node is working, it will only occupy the computing resources of the terminal where it is located. Therefore, multiple terminals in a terminal cluster jointly process a distributed task. Compared with stand-alone processing, distributed processing can greatly improve the computing power and calculation. speed. It should be noted that since the Actor in the Actor model is used as the working node in the embodiment of this application, distributed task scheduling and processing can be realized without the need of a distributed lock. This is because the Actor only pays attention to the received messages.
  • the scheduling node is used to call the at least one task node to execute the target task by means of message transmission. Specifically, the scheduling node sends a message for triggering the task node to perform the task to the task node, so that the task node executes the task. After the task node completes the task, it will feed back a message indicating the completion of the task to the scheduling node, so that the scheduling The node obtains the task node that completes the task, and continues to schedule and execute other task nodes in the order of execution, thereby finally completing the target task.
  • the standby scheduling node in the Actor model is used to replace the down scheduling node as a new scheduling node when the scheduling node is down, and then the new scheduling node can start scheduling from the beginning, or follow The scheduling progress of the downtime scheduling node before the downtime continues to execute the scheduling.
  • the backup scheduling node is created according to the initialization information of the scheduling node, so the backup scheduling node can replace the scheduling node.
  • Step 205 is performed during the execution of step 204, starting from step 204 and ending at any time point between step 204, as long as the scheduling node is detected In case of downtime, select a backup scheduling node from the above at least one backup scheduling node as the new scheduling node, and then continue to use the new scheduling node to perform the above step 204. After the execution of step 204 is completed, it means that the target task is completed, so step 205 is also over.
  • the backup scheduling node can be activated when the scheduling node is down, so as to solve the problem that the scheduling node may be down.
  • only one backup scheduling node when the scheduling node is down, only one backup scheduling node included in the above Actor model will be directly used as a new scheduling node.
  • the above Actor model includes multiple standby scheduling nodes, when the scheduling node is down, select any one of the multiple standby scheduling nodes as the new scheduling node, or obtain each standby scheduling node separately And determine the earliest creation time among the backup scheduling nodes included in the Actor model, and then use the earliest creation time as the new scheduling node.
  • the standby scheduling node used to replace the down scheduling node is determined according to the time of the standby scheduling time, the advantage is that it is convenient for the local device to read the creation time of the currently used scheduling node to quickly determine that it is down. The number of calling nodes in order to know the downtime of the calling node.
  • creating the Actor model for the target task mentioned above refers to obtaining the label of the target task.
  • the label is used to indicate the importance of the target task, and the number of targets corresponding to the label of the target task is obtained.
  • To indicate the number of standby scheduling nodes in the Actor model and then create an Actor model including the target number of standby scheduling nodes.
  • the number of standby scheduling nodes in the Actor model is related to the importance of the target task.
  • the more important the target task the more standby scheduling nodes are created for the target task.
  • the label of the target task is obtained, and the target quantity corresponding to the label of the target task is obtained in the database, and then a backup scheduling node containing the target quantity is created.
  • the corresponding relationship between the label of the task and the number of standby scheduling nodes is stored in the database.
  • the above-mentioned multiple working nodes further include at least one monitoring node, and the monitoring node is used to initialize the scheduling node that is down. Specifically, if it is detected that the scheduling node is down, the monitoring node is used to The down scheduling node is initialized, so that the down scheduling node returns to normal.
  • the embodiment of the present application describes another method for solving the downtime of the scheduling node, that is, if the scheduling node is down, the monitoring device is used to initialize the down scheduling node. Specifically, after detecting that the scheduling node is down, the monitoring node is used to initialize the down scheduling node according to the initialization information of the scheduling node, so that the down scheduling node returns to normal. Alternatively, the monitoring node is used to restore the downtime scheduling node to the normal state at the moment before the downtime. Specifically, after detecting the downtime of the above scheduling node, obtain the downtime scheduling node recorded in the database. The state information at a time before the machine is used, and the monitoring node is used to restore the scheduling node according to the state information of the scheduling node at the time before the downtime, so that the scheduling node is restored to the normal state corresponding to the state information.
  • an Actor model for the target task it is determined whether to create a monitoring node in the above Actor model according to the importance of the target task. Specifically, after the target task is detected in the task cluster, a label of the target task is obtained, and the label is used to indicate the importance of the target task. Get the importance level of the target task corresponding to the tag of the target task. If the importance level of the target task exceeds the preset level, create the above monitoring node when creating the Actor model. If the importance level of the target task does not exceed the preset level, Then the above monitoring node is not created.
  • the aforementioned monitoring node is also used to initialize the down task node, so that the down task node returns to normal.
  • the above-mentioned Actor model created for the target task includes at least one standby task node for each task node included in the model, and the standby task node is used to replace a down task node. That is to say, a task node in the Actor model corresponds to a standby task node, and the task node and the standby task node have a one-to-one correspondence.
  • the solution also includes: detecting the down task node; if the down task node is detected, selecting the standby task node corresponding to the down task node from at least one standby task node in the Actor model, And use the selected standby task node as the new task node.
  • the local device can detect whether the target task is completed, and if the target task is detected to be completed, the target is deleted from the task cluster Task, and detect and execute the next target task in the task cluster.
  • the above-mentioned created working nodes are detected, the number of working nodes and the creation time of each created working node are counted. Sort the created work nodes based on the order of creation time from morning to night.
  • the created working nodes are sequentially deleted according to the above-mentioned creation time from early to late until the remaining number of created working nodes Less than or equal to the preset number.
  • the number of working nodes that have been created is detected. If the number of nodes exceeds the preset number, the working nodes are deleted until the number of remaining working nodes is less than or equal to the preset number.
  • the work nodes are deleted in the order of creation time from early to late. This is because the work nodes that are created earlier are less likely to be used again than those created later.
  • the detailed process of deleting the created working nodes is to detect the number of created working nodes and obtain the creation time of the created working nodes. If the number of created working nodes is greater than the preset number, Determine the time node with the earliest creation time among the created working nodes, and delete the working node with the earliest creation time among the created working nodes, and then if the created working node is deleted, the job with the latest creation time is deleted After the node, if the number of created working nodes is still greater than the preset number, the execution of the working node with the earliest creation time among the above determined working nodes will be triggered to delete the working node with the earliest creation time among the created working nodes Until the number of created work nodes is less than or equal to the preset number after deleting the work node with the latest creation time.
  • This application embodiment is more detailed than the previous application embodiment, and describes in detail that in the process of using the scheduling node to call the task node to complete the target task, if it is detected that the scheduling node is down, the Actor model includes At least one of the backup scheduling nodes is selected to replace the down scheduling node as the new scheduling node.
  • the Actor model includes At least one of the backup scheduling nodes is selected to replace the down scheduling node as the new scheduling node.
  • the target task is completed, if the number of created working nodes exceeds the preset number, then The redundant work nodes are cleared, and the work nodes with an earlier creation time are cleared, so that the number of work nodes that have been created after deletion is less than or equal to the preset number.
  • the embodiment of the present application uses the standby scheduling node as a new scheduling node when the scheduling node is down, so that the task scheduling process is more reliable and the success rate of task scheduling is increased.
  • the embodiment of the present application By clearing the redundant working nodes in time after the target task is completed, the smooth operation of the system of the device is ensured, and the scheduling and processing of the next target task are guaranteed. In general, the efficiency of task scheduling has been further improved.
  • the embodiment of the present application also provides a task scheduling device, which is used to execute the unit of any one of the foregoing task scheduling methods.
  • a task scheduling device which is used to execute the unit of any one of the foregoing task scheduling methods.
  • FIG. 3 is a schematic block diagram of a task scheduling apparatus provided by an embodiment of the present application.
  • the task scheduling apparatus of this embodiment includes: a detection unit 310, a creation unit 320, an execution unit 330, and a deletion unit 340. specific:
  • the detection unit 310 is configured to detect whether there is a newly added target task in the task cluster
  • the creation unit 320 is configured to create an Actor model for the target task in the case of detecting that there is a newly added target task in the task cluster.
  • the Actor model includes multiple working nodes, and the multiple working nodes are communicated through messages. Communication, at least two working nodes among the multiple working nodes are respectively distributed on different terminals, the multiple working nodes include a scheduling node and at least one task node, and the scheduling node is used to call the task node to perform tasks;
  • the execution unit 330 is configured to use the scheduling node to call the at least one task node to execute the target task in the message transmission manner;
  • the deleting unit 340 is configured to delete the target task in the task cluster after detecting that the target task is completed.
  • the multiple work sections also include at least one backup scheduling node, the backup scheduling node is used to replace a down scheduling node, and the detection unit 310 is also used to detect whether the scheduling node is down.
  • the above task scheduling device also includes a replacement unit 350, which is used to select a backup scheduling node from the above at least one backup scheduling node as a new scheduling node.
  • the above-mentioned task scheduling method further includes an obtaining unit 370, configured to obtain the creation time of the at least one backup scheduling node; and further including a determining unit 360, configured to determine the at least one backup scheduling node The scheduling node with the earliest creation time; the replacement unit 350 is specifically configured to use the standby scheduling node with the earliest creation time as the new scheduling node.
  • the acquiring unit 370 is further configured to acquire a tag of the target task, the tag is used to indicate the importance of the target task; the acquiring unit 370 is also configured to acquire the tag of the target task The number of targets corresponding to the label. The number of targets is used to indicate the number of standby scheduling nodes in the Actor model; the creation unit 320 is also used to create an Actor model that includes the number of standby scheduling nodes described above.
  • the above-mentioned multiple working nodes further include at least one monitoring node, the above-mentioned monitoring node is used to initialize a down-time scheduling node, and the above-mentioned detection unit 310 is also used to detect that the above-mentioned scheduling node is down;
  • the task scheduling device further includes a repair unit 380, which is configured to use the monitoring node to initialize the down scheduling node, so that the down scheduling node returns to normal.
  • the detection unit 310 is configured to detect the number of working nodes that have been created; the acquiring unit 370 is configured to acquire the creation time of the working nodes that have been created; the deletion unit 340, It is also used for if the number of the above created work nodes is greater than the preset number, according to the order of the above creation time from early to late, the above created work nodes are sequentially deleted until the remaining created work nodes The number is less than or equal to the above preset number.
  • the detection unit 310 is specifically configured to detect newly added tasks in the task cluster; the detection unit 310 is also specifically configured to determine that the newly added tasks are distributed tasks, It is determined that a newly-added target task is detected in the task cluster.
  • the creation unit establishes an Actor model for the target task including a scheduling node and a task node, and then the execution unit uses the scheduling node to schedule the task node to execute the target task.
  • the delete unit deletes the target task.
  • the task scheduling apparatus in this embodiment may include: one or more processors 410 and a memory 420.
  • the aforementioned processor 410 and memory 420 are connected through a bus 430.
  • the memory 420 is configured to store a computer program.
  • the computer program includes program instructions.
  • the processor 410 is configured to execute the program instructions stored in the memory 420. specific:
  • the processor 410 is used to perform the function of the detection unit 310 and is used to detect whether there is a new target task in the task cluster; it is also used to perform the function of the creation unit 320 and is used to detect the presence of a new target in the task cluster
  • an Actor model is created for the above target task.
  • the above Actor model includes multiple working nodes, and the multiple working nodes communicate through message passing, and at least two working nodes among the multiple working nodes are respectively distributed On different terminals, the multiple working nodes include a scheduling node and at least one task node.
  • the scheduling node is used to call the task node to perform tasks; it is also used to perform the function of the execution unit 330 and is used to use the scheduling node to pass
  • the message transfer method calls the at least one task node to execute the target task; it is also used to perform the function of the deleting unit 340, which is used to delete the target task in the task cluster after detecting that the target task is completed.
  • the multiple work sections also include at least one backup scheduling node, the backup scheduling node is used to replace a down scheduling node, and the processor 410 is also used to detect whether the scheduling node is down.
  • the processor 410 is also used to perform the function of the replacement unit 350, and is used to select a backup scheduling node from the above at least one backup scheduling node as a new scheduling node.
  • the aforementioned processor 410 is further configured to execute the function of the obtaining unit 370, and is configured to obtain the creation time of the aforementioned at least one standby scheduling node; the aforementioned processor 410 is further configured to execute the function of the determining unit 360 The function is used to determine the scheduling node with the earliest creation time among the at least one backup scheduling node; the processor 410 is also specifically configured to use the backup scheduling node with the earliest creation time as the new scheduling node.
  • the processor 410 is further configured to obtain a label of the target task, the label is used to indicate the importance of the target task; and the processor 410 is also used to obtain the number of targets corresponding to the label of the target task.
  • the above-mentioned target quantity is used to indicate the number of standby scheduling nodes in the above Actor model; it is also used to create an Actor model including the above-mentioned target quantity of standby scheduling nodes.
  • the above-mentioned multiple working nodes further include at least one monitoring node, the above-mentioned monitoring node is used to initialize a down-time scheduling node, and the above-mentioned processor 410 is also used to detect that the above-mentioned scheduling node is down;
  • the device 410 is also used to perform the function of the repair unit 380, and is used to initialize the above-mentioned down scheduling node with the above-mentioned monitoring node, so that the above-mentioned down scheduling node is restored to normal.
  • the above-mentioned processor 410 is also used to detect the number of created working nodes; also used to obtain the creation time of the above-mentioned created working nodes; and also used to detect the above-mentioned created working nodes. If the number of nodes is greater than the preset number, the above-mentioned created working nodes are sequentially deleted according to the above-mentioned creation time from early to late, until the number of remaining created working nodes is less than or equal to the above-mentioned preset number.
  • the aforementioned processor 410 is specifically configured to detect the newly added task in the aforementioned task cluster; and is also specifically configured to determine that the aforementioned newly added task is a distributed task. There are new target tasks in the task cluster.
  • the processor 410 may be a central processing unit (CPU), and the processor may also be other general-purpose processors or digital signal processors (DSP). , Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.
  • the memory 420 may include a read-only memory and a random access memory, and provides instructions and data to the processor 410. A part of the memory 420 may also include a non-volatile random access memory. For example, the memory 420 may also store device type information.
  • the processor 410 described in the embodiment of this application can execute the implementations described in the first embodiment and the second embodiment of the task scheduling method provided in the embodiment of this application, and can also execute the embodiment of this application.
  • the implementation of the described task scheduling device will not be repeated here.
  • a computer-readable storage medium stores a computer program, the computer program includes program instructions, and the program instructions are executed by a processor.
  • the computer-readable storage medium may be an internal storage unit of the task scheduling device of any of the foregoing embodiments, such as a hard disk or memory of the task scheduling device.
  • the computer-readable storage medium may also be an external storage device of the task scheduling device, such as a plug-in hard disk equipped on the task scheduling device, a smart memory card (Smart Media Card, SMC), a Secure Digital (SD) card, and a flash memory Card (Flash Card) etc.
  • the computer-readable storage medium may also include both an internal storage unit of the task scheduling apparatus and an external storage device.
  • the computer-readable storage medium is used to store computer programs and other programs and data required by the task scheduling device.
  • the computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Multi Processors (AREA)

Abstract

本申请公开了一种基于Actor模型的任务调度方法、装置及存储介质，应用于数据处理领域，其中方法包括：检测任务集群中是否存在新增的目标任务；在检测到任务集群中存在新增的目标任务的情况下，为目标任务创建Actor模型，Actor模型包括多个工作节点，多个工作节点包括一个调度节点和至少一个任务节点；利用调度节点通过消息传递的方式调用至少一个任务节点执行目标任务；在检测到目标任务完成之后，在任务集群中删除目标任务。本申请提供了一种基于Actor模型的任务调度方法，由于调度节点和任务节点都是Actor节点，使得本申请不需要锁便能完成任务调度，大大的提高了任务调度的效率。

Description

一种基于Actor模型的任务调度方法、装置及存储介质
本申请要求于2019年05月20日提交中国专利局、申请号为201910421282.5、申请名称为“一种基于Actor模型的任务调度方法、装置及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及任务调度领域，尤其涉及一种基于Actor模型的任务调度方法、装置及存储介质。
背景技术
任务调度常常因为多线程或多进程并发访问某个临界资源而陷入死锁，这时可通过对该临界资源进行加锁，来建立线程或进程之间的互斥同步机制，以保障临界资源的完整性和一致性。单机的进程同步和线程同步有很多可用的互斥方案，比如互斥锁、信号量和条件变量等。但这些单机下的互斥方案在多机上是无法使用的，因为分布式系统的组件运行在不同的机器上，使得其不在统一的运行环境中。
分布式任务调度中，常常通过分布式锁解决对临界资源的互斥访问，比如Redis分布式锁和Zookeeper分布式锁的等。
但是分布式锁的实现比较复杂，且维护比较麻烦，需要较多维护成本，因此分布式锁不是一个解决分布式任务调度的死锁问题的理想方案，还是缺少了一种可以更高效的分布式任务调度方法。
发明内容
本申请实施例提供一种基于Actor模型的任务调度方法，可以不使用锁便实现分布式任务调度和处理。
第一方面，本申请实施例提供了一种基于Actor模型的任务调度方法，该方法包括：
检测任务集群中是否存在新增的目标任务；
在检测到所述任务集群中存在新增的目标任务的情况下，为所述目标任务创建Actor模型，所述Actor模型包括多个工作节点，所述多个工作节点之间通过消息传递的方式通信，所述多个工作节点中至少存在两个工作节点分别分布于不同的终端上，所述多个工作节点包括一个调度节点和至少一个任务节点，所述调度节点用于调用所述任务节点执行任务；
利用所述调度节点通过所述消息传递的方式调用所述至少一个任务节点执行所述目标任务；
在检测到所述目标任务完成之后，在所述任务集群中删除所述目标任务。
第二方面，本申请实施例提供了一种任务调度装置，该任务调度装置包括用于执行上述第一方面的任务调度方法的单元，该任务调度装置包括：
检测单元，用于检测任务集群中是否存在新增的目标任务；
创建单元，用于在检测到任务集群中存在新增的目标任务的情况下，为所述目标任务创建Actor模型，所述Actor模型包括多个工作节点，所述多个工作节点之间通过消息传递的方式通信，所述多个工作节点中至少存在两个工作节点分别分布于不同的终端上，所述多个工作节点包括一个调度节点和至少一个任务节点，所述调度节点用于调用所述任务节点执行任务；
执行单元，用于利用所述调度节点通过所述消息传递的方式调用所述至少一个任务节点执行所述目标任务；
删除单元，用于在检测到所述目标任务完成之后，在所述任务集群中删除所述目标任务。
第三方面，本申请实施例提供了另一种任务调度装置，包括处理器和存储器，所述处理器和存储器相互连接，其中，所述存储器用于存储支持任务调度装置执行上述任务调度方法的计算机程序，所述计算机程序包括程序指令，所述处理器被配置用于调用所述程序指令，用以执行：
检测任务集群中是否存在新增的目标任务；
在检测到所述任务集群中存在新增的目标任务的情况下，为所述目标任务创建Actor模型，所述Actor模型包括多个工作节点，所述多个工作节点之间通过消息传递的方式通信，所述多个工作节点中至少存在两个工作节点分别分布于不同的终端上，所述多个工作节点包括一个调度节点和至少一个任务节点，所述调度节点用于调用所述任务节点执行任务；
利用所述调度节点通过所述消息传递的方式调用所述至少一个任务节点执行所述目标任务；
在检测到所述目标任务完成之后，在所述任务集群中删除所述目标任务。
第四方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被处理器执行，用以执行上述第一方面至第一方面的任意一种实现方式的任务调度方法。
本申请不需要借助于锁便可以实现复杂的分布式任务调度，保证分布式任务调度过程中不会产生数据竞争和死锁，并且大大提高分布式任务处理的并发性能。
附图说明
为了更清楚地说明本申请实施例技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍。
图1是本申请实施例提供的一种任务调度方法的示意流程图；
图2是本申请另一实施例提供的一种任务调度方法的示意流程图；
图3是本申请实施例提供的一种任务调度装置的示意性框图；
图4是本申请实施例提供的一种任务调度装置的结构性框图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。
本申请主要应用于任务调度装置，该任务调度装置可以是传统任务调度装置、大型存储系统、台式电脑、笔记本电脑、平板电脑、掌上电脑、智能手机、便携式数字播放器、智能手表以及智能手环等等，本申请对此不做限制。
本申请实施例中描述的终端包括但不限于带通讯功能的设备、智能手机、平板电脑、笔记本电脑、台式电脑、便携式数字播放器、智能手环以及智能手表等。当终端向任务调度装置发送数据的时候，按照预设格式进行对数据的特性进行记录并传送，其中，数据的特性包括时间、地点、类型等。
参见图1，是本申请实施例提供一种任务调度方法的示意流程图，如图1所示任务调度方法可包括：
101：检测任务集群中是否存在新增的目标任务。
在本申请实施例中，检测任务集群中是否存在新增的任务，若存在新增的任务，则将该新增的任务作为目标任务。其中，任务集群指的是待处理的任务的集合，且该任务集群中的不同任务可能由不同的终端所提出，而任务集群中的任务可能是由多个终端共同处理的分布式任务，也可能是仅由一个终端处理的单机任务，检测的过程指的是，本端的任务调度装置定时检测任务集群中是否存在新增的任务，或者任务集群中新增一个任务，本端装置便能接收到用于指示任务集群中新增任务的消息。
在另一种可实施的方式中，上述目标任务不是任务集群中新增的任意任务，而是任务集群中新增的分布式任务，具体的，上述检测任务集群中是否存在新增的目标任务指的是，检测任务集群中的新增的任务，若确定该新增的任务为分布式任务，则确定检测到任务集群中存在新增的目标任务。
在本申请实施例中，上述检测任务集群中新增的任务为分布式任务的过程指的是，获取上述任务集群中新增的任务的类型，若该新增的任务的类型为分布式任务，则表示该新增的任务的类型为分布式任务，或者，获取上述新增的任务的难度等级，若确定该新增的任务的难度等级超过预设级别，则说明该新增的任务需要占用较多的计算资源，并确定该新增的任务为分布式任务，也即是将该难度级别超过预设级别的任务作为目标任务，并接下来采用分布式任务的处理方法对其进行处理。
可见，本申请实施例只有在确定任务集群新增了分布式任务的时候，才确定任务集群中有新增的目标任务，上述检测到的任务集群中新增的分布式任务即目标任务。
102：在检测到上述任务集群中存在新增的目标任务的情况下，为该目标任务创建Actor模型，Actor模型包括多个工作节点，多个工作节点包括一个调度节点和至少一个任务节点。
在本申请实施例中，Actor模型为一个用于任务处理的模型，具体来说Actor模型是一个概念模型，用于处理并发计算，它定义了一系列系统组件应该如何动作和交互的通用规则。Actor模型中包含了多个Actor，Actor在本申请实施例中被称为工作节点，每个工作节点是Actor模型中进行任务处理的最基本的计算单元，它能接收一个消息并且基于其执行计算。于是上述根据该目标任务的执行内容创建包含多个工作节点的Actor模型指的是，对目标任务进行分解，然后基于分解之后的目标任务的任务内容，创建多个工作节点来执行对应的任务内容，而该用于执行目标任务的工作节点即任务节点，而在执行任务的时候，多个任务节点可能是并行执行的，如果调度任务节点时调度不当，可能会造成多个任务节点之间的资源竞争，于是为了确定调度的顺序，并避免资源竞争，上述Actor模型中还包括用于执行任务调度的工作节点，该用于执行任务调度的工作节点在本申请实施例中被称为调度节点，调度节点用于安排各个任务节点的执行顺序，并调度各个任务节点有序执行任务。总的来说，为上述目标任务创建的Actor模型中包含多个工作节点，该多个工作节点中包含一个调度节点和至少一个任务节点。
需要说明的是，由于本申请实施例采用的调度节点和任务节点都是Actor，而Actor之间是通过消息传递的方式进行通信，因此不管是在任务调度，还是任务执行的时候，都不会产生资源竞争的问题。其中，消息传递的方式指的是在消息传递系统中，进程间的数据交换是以格式化的消息(Message)为单位的，Actors的一大重要特征在于Actors之间相互隔离，它们并不互相共享内存，一个Actor能维持一个私有的状态，并且这个状态不可能被另一个Actor所改变，而消息传递也保证了Actor之间的异步处理，因此Actor之间也不会存在数据竞争。若通信的进程之间不存在可直接访问的共享空间，则必须利用操作系统提供的消息传递方法实现进程通信，进程通过系统提供的发送消息和接收消息两个原语进行数据交换。简单来说，Actor之间通过消息传递的方式来进行资源共享，当Actor需要另一个Actor的资源的时候，只需要将消息发送给另一个Actor，但不关心另一个Actor什么时候处理，因此Actor之间的消息是异步传输的，类似邮件通信。具体的，在Actor模型里每个Actor都有地址，所以它们才能够相互发送消息，尽管许多Actor同时运行，但是一个Actor只能顺序地处理消息。也就是说其它三个Actor发送了给一个Actor分别发送了一条消息，这个Actor只能一次处理一条，如果需要并行处理三条消息，则需要三个Actor分别对这三条消息进行处理，并且如果Actor在处理消息的过程中又接收到一条新的消息，则该新的消息会被存储到Actor的邮箱Mailbox中，每个Actor对应有一个Mailbox。
可见，Actor通过消息传递的方式将不同Actor的可访问的资源进行了分隔，并保障了Actor之间的消息的异步处理，从而避免了使用锁来处理数据条件竞争(data race)的问题。消息传递机制最大的优点就是不会产生数据竞争状态(data race)。
还需要说明的是，上述多个工作节点中至少存在两个工作节点分别分布于不同的终端 上，于是本申请实施例应用于分布式任务的调度和处理，可见，由于位于不同的终端上的工作节点在工作的时候，只会占用其所在终端的计算资源，于是在一个终端集群中的多个终端共同处理一个分布式任务，相比单机处理，分布式处理可以大大提升了计算能力和计算速度。需要注意的是，由于本申请实施例采用了Actor模型中的Actor来作为工作节点，才可以不需要分布式锁便能实现分布式任务调度和处理，这是因为Actor只关注接收到的消息的内容，而不关注消息的发送方是本地的Actor还是终端上的Actor，于是只要保证多个Actor之间的消息能够顺利达到即可，也也允许了本申请实施例可以不及住锁便能在包含多个终端的终端集群中构建Actor模型，并实现分布式任务的调度和处理。
在另一种可实施的方式中，上述多个工作节还包括至少一个备用调度节点，备用调度节点用于替换宕机的调度节点。若检测到上述调度节点宕机，则从上述至少一个备用调度节点中选取一个备用调度节点作为新的调度节点。
在本申请实施例中，为目标任务创建的Actor模型中除了上述任务节点和调度节点以外，还包括至少一个备用调度节点，该备用调度节点用于在调度节点宕机的时候，替换该宕机的调度节点作为新的调度节点，然后新的调度节点可以从头开始执行调度，或者接着宕机的调度节点在宕机之前的调度进度，继续执行调度。其中，备用调度节点是按照调度节点的初始化信息创建的，因此备用调度节点可以替代调度节点。
可见，本申请实施例通过创建至少一个备用调度节点，可以在调度节点宕机时，启用该备用调度节点，以解决调度节点可能宕机的问题。
在一种可实施的方式中，若上述Actor模型中只包括一个备用调度节点，则将在调度节点宕机的时候，直接将上述Actor模型中仅包括的一个备用调度节点作为新的调度节点，若上述Actor模型中包括多个备用调度节点，则在调度节点宕机的时候，从该多个备用调度节点中选取任意一个备用调度节点作为新的调度节点，或者，分别获取每个备用调度节点的创建时间，并确定Actor模型包括的备用调度节点中创建时间最早的调度节点，然后将创建时间最早的备用调度节点作为新的调度节点。
可见，如果是根据备用调度时间的早晚来确定用于替换宕机的调度节点的备用调度节点，好处在于，便于本端装置通过读取当前使用的调度节点的创建时间，可以快速判定已宕机的调用节点的个数，从而获知调用节点的宕机情况。
在一种可实施的方式中，上述为目标任务创建Actor模型指的是，获取目标任务的标签，标签用于指示目标任务的重要程度，并获取目标任务的标签对应的目标数量，目标数量用于指示Actor模型中的备用调度节点的个数，然后创建包括目标数量的备用调度节点的Actor模型。
在本申请实施例中，在创建Actor模型的时候，Actor模型中的备用调度节点的个数多少与目标任务的重要程度有关，目标任务越重要，就为目标任务创建更多的备用调度节点。具体的，在为目标任务创建Actor模型的时候，获取目标任务的标签，并在数据库中获取该目标任务的标签对应的目标数量，然后创创建包含目标数量的备用调度节点。其中，数 据库中保存有任务的标签与备用调度节点的个数的对应关系。
在另一种可实施的方式中，上述多个工作节点还包括至少一个监控节点，监控节点用于初始化宕机的调度节点，具体的，若检测到调度节点宕机，则利用监控节点对该宕机的调度节点进行初始化，使得该宕机的调度节点恢复正常。
本申请实施例描述了另一种解决调度节点宕机问题的方法，即如果调度节点宕机，则使用该监控设备对宕机的调度节点进行初始化。具体的，在检测到调度节点宕机之后，使用监控节点根据调度节点的初始化信息对宕机的调度节点进行初始化，使得宕机的调度节点恢复正常。或者，监控节点用于将宕机的调度节点恢复到宕机前一个时刻的正常状态，具体的，在检测到上述调度节点宕机之后，获取记录在数据库中的该宕机的调度节点在宕机前一个时刻的状态信息，并使用监控节点根据上述调度节点在宕机前一个时刻的状态信息对调度节点进行恢复，使得调度节点恢复到上述状态信息对应的正常状态。
在另一种可实施的方式中，在为目标任务创建Actor模型的时候，根据目标任务的重要程度决定是否在上述Actor模型中创建监控节点。具体的，在任务集群中检测到目标任务之后，获取该目标任务的标签，该标签用于指示目标任务的重要程度。获取目标任务的标签对应的目标任务的重要等级，若目标任务的重要等级超过了预设等级，则在创建Actor模型的时候，创建上述监控节点，若目标任务的重要等级未超过预设等级，则不创建上述监控节点。
在另一种可实施的方式中，上述监控节点还用于对宕机的任务节点进行初始化，以使得该宕机的任务节点恢复正常。
在另一种可实施的方式中，上述为目标任务创建的Actor模型针对于该模型中包含的每个任务节点分别包括至少一个备用任务节点，该备用任务节点用于替换宕机的任务节点。也即是说Actor模型中的一个任务节点对应一个备用任务节点，任务节点与备用任务节点是一一对应的。具体的，本方案还包括：检测宕机的任务节点；若检测到宕机的任务节点，则从Actor模型中的至少一个备用任务节点中选取出与宕机的任务节点对应的备用任务节点，并将该选出的备用任务节点作为新的任务节点。
103：利用上述调度节点通过消息传递的方式调用上述至少一个任务节点执行上述目标任务。
在本申请实施例中，利用调度节点通过消息传递的方式，调用上述至少一个任务节点执行上述目标任务。具体的，调度节点将用于触发任务节点执行任务的消息发送给任务节点，使得任务节点执行任务，在任务节点执行完任务之后，会向调度节点反馈一个用于指示任务完成的信息，使得调度节点获取完成任务的任务节点，并继续按照执行顺序调度执行其他任务节点，从而最终完成目标任务。
104：在检测到上述目标任务完成之后，在上述任务集群中删除上述目标任务。
在本申请实施例中，在检测到上述目标任务完成之后，便在上述任务集群中删除上述目标任务，并检测和执行任务集群中的下一个目标任务。
在另一种可实施的方式中，上述在任务集群中删除目标任务之后，检测已创建的工作节点的个数，并获取所述已创建的工作节点的创建时间；若所述已创建的工作节点的个数大于预设个数，则按照所述创建时间从早到晚的排序，依次删除所述已创建的工作节点，直到剩下的已创建的工作节点的个数小于或等于所述预设个数。
在本申请实施例中，系统中的工作节点可能太多而导致本端装置存储空间不足或者系统运行缓慢，于是在完成目标任务之后，检测已创建的工作节点的个数，如果已创建的工作节点的个数超过预设个数，就对工作节点进行删减，直到剩下的工作节点的个数小于等于预设个数。其中，按照工作节点的创建时间由早到晚顺序进行删除，这是因为早创建的工作节点，相较于晚创建的工作节点，被再次使用的几率更小。
上述删除已创建的工作节点的详细过程为，检测已创建的工作节点的个数，并获取该已创建的工作节点的创建时间，若该已创建的工作节点的个数大于预设个数，则确定该已创建的工作节点中创建时间最早的时间节点，并删除所述已创建的工作节点中创建时间最早的工作节点，然后若在删除该已创建的工作节点中创建时间最晚的工作节点之后，已创建的工作节点的个数仍大于预设个数，则触发执行上述确定已创建的工作节点中创建时间最早的工作节点，至删除已创建的工作节点中创建时间最早的工作节点的步骤，直到已创建的工作节点的个数在删除了创建时间最晚的工作节点之后小于或等于预设个数。
在本申请实施例中，每当检测到目标任务，就为该目标任务建立包括调度节点和任务节点的Actor模型，并利用调度节点来调度任务节点执行目标任务，在检测到目标任务完成之后，删除该目标任务，可见，本申请所提供的任务调度方法中，由于调度节点和任务节点都是Actor，于是调度节点和任务节点是通过信息传递的方式进行通信的，而信息传递的好处就是不会产生数据竞争的问题，在处理任务的时候无需同步阻塞等待，于是本申请也便不需要借助于锁便可以实现复杂的分布式任务调度，并且大大提高分布式任务处理的并发性能。可见本申请实施例提供了一种高效的任务调度方法。
参见图2，是本申请实施例提供另一种任务调度方法的示意流程图，如图2所示任务调度方法可包括：
201：检测任务集群中的新增的任务。
在本申请实施例中，检测任务集群中是否存在新增的任务，具体的检测过程指的是，本端的任务调度装置定时检测任务集群中是否存在新增的任务，或者任务集群中新增一个任务，本端装置便能接收到用于指示任务集群中新增任务的消息。其中，任务集群指的是待处理的任务的集合，且该任务集群中的不同任务可能由不同的终端所提出，而任务集群中的任务可能是由多个终端共同处理的分布式任务，也可能是仅由一个终端处理的单机任务。
202：若确定上述新增的任务为分布式任务，则确定检测到上述任务集群中存在新增的目标任务。
在本申请实施例中，在检测到任务集群中的新增的任务之后，进一步确定该新增的任务是否为目标任务。具体的，确定该新增的任务是否为分布式任务，若是则确定检测到任务集群中存在新增的目标任务。
在本申请实施例中，上述检测任务集群中新增的任务为分布式任务的过程指的是，获取上述任务集群中新增的任务的类型，若该新增的任务的类型为分布式任务，则表示该新增的任务的类型为分布式任务，或者，获取上述新增的任务的难度等级，若确定该新增的任务的难度等级超过预设级别，则说明该新增的任务需要占用较多的计算资源，并确定该新增的任务为分布式任务，也即是将该难度级别超过预设级别的任务作为目标任务，并接下来采用分布式任务的处理方法对其进行处理。
可见，本申请实施例只有在确定任务集群新增了分布式任务的时候，才确定任务集群中有新增的目标任务，上述检测到的任务集群中新增的分布式任务即目标任务。
203：在检测到上述任务集群中存在新增的目标任务的情况下，为该目标任务创建actor模型，actor模型包括多个工作节点，多个工作节点包括一个调度节点、至少一个任务节点和至少一个备用调度节点。
在本申请实施例中，Actor模型为一个用于任务处理的模型，具体来说Actor模型是一个概念模型，用于处理并发计算，它定义了一系列系统组件应该如何动作和交互的通用规则。Actor模型中包含了多个Actor，Actor在本申请实施例中被称为工作节点，每个工作节点是Actor模型中进行任务处理的最基本的计算单元，它能接收一个消息并且基于其执行计算。于是上述根据该目标任务的执行内容创建包含多个工作节点的Actor模型指的是，对目标任务进行分解，然后基于分解之后的目标任务的任务内容，创建多个工作节点来执行对应的任务内容，而该用于执行目标任务的工作节点即任务节点，而在执行任务的时候，多个任务节点可能是并行执行的，如果调度任务节点时调度不当，可能会造成多个任务节点之间的资源竞争，于是为了确定调度的顺序，并避免资源竞争，上述Actor模型中还包括用于执行任务调度的工作节点，该用于执行任务调度的工作节点在本申请实施例中被称为调度节点，调度节点用于安排各个任务节点的执行顺序，并调度各个任务节点有序执行任务。此外，由于调度节点只有一个，于是为了应对调度节点的情况，Actor中还包括至少一个备用调度节点，备用调度节点用于替换宕机的调度节点。总的来说，为上述目标任务创建的Actor模型中包含多个工作节点，该多个工作节点中包含一个调度节点、至少一个任务节点和至少一个备用调度节点。
需要说明的是，由于本申请实施例采用的调度节点和任务节点都是Actor，而Actor之间是通过消息传递的方式进行通信，因此不管是在任务调度，还是任务执行的时候，都不会产生资源竞争的问题。其中，消息传递的方式指的是在消息传递系统中，进程间的数据交换是以格式化的消息(Message)为单位的，Actors的一大重要特征在于Actors之间相互隔离，它们并不互相共享内存，一个Actor能维持一个私有的状态，并且这个状态不可能被另一个Actor所改变，而消息传递也保证了Actor之间的异步处理，因此Actor之间也不会 存在数据竞争。若通信的进程之间不存在可直接访问的共享空间，则必须利用操作系统提供的消息传递方法实现进程通信，进程通过系统提供的发送消息和接收消息两个原语进行数据交换。简单来说，Actor之间通过消息传递的方式来进行资源共享，当Actor需要另一个Actor的资源的时候，只需要将消息发送给另一个Actor，但不关心另一个Actor什么时候处理，因此Actor之间的消息是异步传输的，类似邮件通信。具体的，在Actor模型里每个Actor都有地址，所以它们才能够相互发送消息，尽管许多Actor同时运行，但是一个Actor只能顺序地处理消息。也就是说其它三个Actor发送了给一个Actor分别发送了一条消息，这个Actor只能一次处理一条，如果需要并行处理三条消息，则需要三个Actor分别对这三条消息进行处理，并且如果Actor在处理消息的过程中又接收到一条新的消息，则该新的消息会被存储到Actor的邮箱Mailbox中，每个Actor对应有一个Mailbox。
可见，Actor通过消息传递的方式将不同Actor的可访问的资源进行了分隔，并保障了Actor之间的消息的异步处理，从而避免了使用锁来处理数据条件竞争(data race)的问题。消息传递机制最大的优点就是不会产生数据竞争状态(data race)。
还需要说明的是，上述多个工作节点中至少存在两个工作节点分别分布于不同的终端上，于是本申请实施例应用于分布式任务的调度和处理，可见，由于位于不同的终端上的工作节点在工作的时候，只会占用其所在终端的计算资源，于是在一个终端集群中的多个终端共同处理一个分布式任务，相比单机处理，分布式处理可以大大提升了计算能力和计算速度。需要注意的是，由于本申请实施例采用了Actor模型中的Actor来作为工作节点，才可以不需要分布式锁便能实现分布式任务调度和处理，这是因为Actor只关注接收到的消息的内容，而不关注消息的发送方是本地的Actor还是终端上的Actor，于是只要保证多个Actor之间的消息能够顺利达到即可，也也允许了本申请实施例可以不及住锁便能在包含多个终端的终端集群中构建Actor模型，并实现分布式任务的调度和处理。
204：利用上述调度节点通过消息传递的方式调用上述至少一个任务节点执行上述目标任务。
在本申请实施例中，利用调度节点通过消息传递的方式，调用上述至少一个任务节点执行上述目标任务。具体的，调度节点将用于触发任务节点执行任务的消息发送给任务节点，使得任务节点执行任务，在任务节点执行完任务之后，会向调度节点反馈一个用于指示任务完成的信息，使得调度节点获取完成任务的任务节点，并继续按照执行顺序调度执行其他任务节点，从而最终完成目标任务。
205：若检测到上述调度节点宕机，则从上述至少一个备用调度节点中选取一个备用调度节点作为新的调度节点。
在本申请实施例中，Actor模型中的备用调度节点用于在调度节点宕机的时候，替换该宕机的调度节点作为新的调度节点，然后新的调度节点可以从头开始执行调度，或者接着宕机的调度节点在宕机之前的调度进度，继续执行调度。其中，备用调度节点是按照调度节点的初始化信息创建的，因此备用调度节点可以替代调度节点。
需要说明的是，步骤205与步骤204之间不存在先后关系，步骤205是在步骤204的执行过程中进行，从步骤204开始，步骤204结束之间的任意一个时间点，只要检测到调度节点宕机，就从上述至少一个备用调度节点中选取一个备用调度节点作为新的调度节点，然后继续利用新的调度节点执行上述步骤204，在步骤204执行结束之后，即表示目标任务完成，于是步骤205也结束了。
可见，本申请实施例通过创建至少一个备用调度节点，可以在调度节点宕机时，启用该备用调度节点，以解决调度节点可能宕机的问题。
在一种可实施的方式中，若上述Actor模型中只包括一个备用调度节点，则将在调度节点宕机的时候，直接将上述Actor模型中仅包括的一个备用调度节点作为新的调度节点，若上述Actor模型中包括多个备用调度节点，则在调度节点宕机的时候，从该多个备用调度节点中选取任意一个备用调度节点作为新的调度节点，或者，分别获取每个备用调度节点的创建时间，并确定Actor模型包括的备用调度节点中创建时间最早的调度节点，然后将创建时间最早的备用调度节点作为新的调度节点。
可见，如果是根据备用调度时间的早晚来确定用于替换宕机的调度节点的备用调度节点，好处在于，便于本端装置通过读取当前使用的调度节点的创建时间，可以快速判定已宕机的调用节点的个数，从而获知调用节点的宕机情况。
在一种可实施的方式中，上述为目标任务创建Actor模型指的是，获取目标任务的标签，标签用于指示目标任务的重要程度，并获取目标任务的标签对应的目标数量，目标数量用于指示Actor模型中的备用调度节点的个数，然后创建包括目标数量的备用调度节点的Actor模型。
在本申请实施例中，在创建Actor模型的时候，Actor模型中的备用调度节点的个数多少与目标任务的重要程度有关，目标任务越重要，就为目标任务创建更多的备用调度节点。具体的，在为目标任务创建Actor模型的时候，获取目标任务的标签，并在数据库中获取该目标任务的标签对应的目标数量，然后创创建包含目标数量的备用调度节点。其中，数据库中保存有任务的标签与备用调度节点的个数的对应关系。
在另一种可实施的方式中，上述多个工作节点还包括至少一个监控节点，监控节点用于初始化宕机的调度节点，具体的，若检测到调度节点宕机，则利用监控节点对该宕机的调度节点进行初始化，使得该宕机的调度节点恢复正常。
本申请实施例描述了另一种解决调度节点宕机问题的方法，即如果调度节点宕机，则使用该监控设备对宕机的调度节点进行初始化。具体的，在检测到调度节点宕机之后，使用监控节点根据调度节点的初始化信息对宕机的调度节点进行初始化，使得宕机的调度节点恢复正常。或者，监控节点用于将宕机的调度节点恢复到宕机前一个时刻的正常状态，具体的，在检测到上述调度节点宕机之后，获取记录在数据库中的该宕机的调度节点在宕机前一个时刻的状态信息，并使用监控节点根据上述调度节点在宕机前一个时刻的状态信息对调度节点进行恢复，使得调度节点恢复到上述状态信息对应的正常状态。
在另一种可实施的方式中，在为目标任务创建Actor模型的时候，根据目标任务的重要程度决定是否在上述Actor模型中创建监控节点。具体的，在任务集群中检测到目标任务之后，获取该目标任务的标签，该标签用于指示目标任务的重要程度。获取目标任务的标签对应的目标任务的重要等级，若目标任务的重要等级超过了预设等级，则在创建Actor模型的时候，创建上述监控节点，若目标任务的重要等级未超过预设等级，则不创建上述监控节点。
在另一种可实施的方式中，上述监控节点还用于对宕机的任务节点进行初始化，以使得该宕机的任务节点恢复正常。
在另一种可实施的方式中，上述为目标任务创建的Actor模型针对于该模型中包含的每个任务节点分别包括至少一个备用任务节点，该备用任务节点用于替换宕机的任务节点。也即是说Actor模型中的一个任务节点对应一个备用任务节点，任务节点与备用任务节点是一一对应的。具体的，本方案还包括：检测宕机的任务节点；若检测到宕机的任务节点，则从Actor模型中的至少一个备用任务节点中选取出与宕机的任务节点对应的备用任务节点，并将该选出的备用任务节点作为新的任务节点。
206：在检测到上述目标任务完成之后，在上述任务集群中删除上述目标任务。
在本申请实施例中，由于调度节点可以感知到目标任务是否完成，于是本端装置可以检测到上述目标任务是否已完成，若在检测到上述目标任务完成，则在上述任务集群中删除上述目标任务，并检测和执行任务集群中的下一个目标任务。
207：检测已创建的工作节点的个数，并获取该已创建的工作节点的创建时间。
在本申请实施例中，在上述目标任务完成，检测上述已创建的工作节点，统计工作节点的个数，以及每个已创建的工作节点的创建时间。基于创建时间从早到晚的顺序，对已创建的工作节点进行排序。
208：若上述已创建的工作节点的个数大于预设个数，则按照上述创建时间从早到晚的排序，依次删除已创建的工作节点，直到剩下的已创建的工作节点的个数小于或等于预设个数。
在本申请实施例中，系统中的工作节点可能太多而导致本端装置存储空间不足或者系统运行缓慢，于是在完成目标任务之后，检测已创建的工作节点的个数，如果已创建的工作节点的个数超过预设个数，就对工作节点进行删减，直到剩下的工作节点的个数小于等于预设个数。其中，按照工作节点的创建时间由早到晚顺序进行删除，这是因为早创建的工作节点，相较于晚创建的工作节点，被再次使用的几率更小。
上述删除已创建的工作节点的详细过程为，检测已创建的工作节点的个数，并获取该已创建的工作节点的创建时间，若该已创建的工作节点的个数大于预设个数，则确定该已创建的工作节点中创建时间最早的时间节点，并删除所述已创建的工作节点中创建时间最早的工作节点，然后若在删除该已创建的工作节点中创建时间最晚的工作节点之后，已创建的工作节点的个数仍大于预设个数，则触发执行上述确定已创建的工作节点中创建时间 最早的工作节点，至删除已创建的工作节点中创建时间最早的工作节点的步骤，直到已创建的工作节点的个数在删除了创建时间最晚的工作节点之后小于或等于预设个数。
本申请实施例相对于上一申请实施例来说更加的详细，且详细描述了在利用调度节点调用任务节点完成目标任务的过程中，若检测到调度节点宕机，则从Actor模型中包括的至少一个备用调度节点中选取一个备用调度节点，替换宕机的调度节点，作为新的调度节点，此外，还详细描述了在目标任务完成之后，若已创建的工作节点超过预设个数，就对多余的工作节点进行清除，清除创建时间较早的工作节点，使得删除之后已创建的工作节点的个数小于等于预设个数。因此本申请实施例一方面在调度节点宕机时，利用备用调度节点来作为新的调度节点，使得任务调度的过程更加可靠，也增加了任务调度的成功率，另一方面，本申请实施例通过在目标任务完成之后及时得对多余的工作节点进行清除，来保障了本装置的系统的顺畅运行，并且保障了下一个目标任务的调度和处理。总的来说任务调度的效率进一步被提高了。
需要说明的是，上文对各个实施例的描述倾向于强调各个实施例之间的不同之处，其相同或相似之处可以互相参考，为了简洁，本文不再赘述。
本申请实施例还提供一种任务调度装置，该任务调度装置用于执行前述任一项的任务调度方法的单元。具体地，参见图3，是本申请实施例提供的一种任务调度装置的示意框图。本实施例的任务调度装置包括：检测单元310、创建单元320、执行单元330以及删除单元340。具体的：
检测单元310，用于检测任务集群中是否存在新增的目标任务；
创建单元320，用于在检测到任务集群中存在新增的目标任务的情况下，为上述目标任务创建Actor模型，上述Actor模型包括多个工作节点，上述多个工作节点之间通过消息传递的方式通信，上述多个工作节点中至少存在两个工作节点分别分布于不同的终端上，上述多个工作节点包括一个调度节点和至少一个任务节点，上述调度节点用于调用上述任务节点执行任务；
执行单元330，用于利用上述调度节点通过上述消息传递的方式调用上述至少一个任务节点执行上述目标任务；
删除单元340，用于在检测到上述目标任务完成之后，在上述任务集群中删除上述目标任务。
在另一种可实施的方式中，上述多个工作节还包括至少一个备用调度节点，上述备用调度节点用于替换宕机的调度节点，上述检测单元310，还用于检测上述调度节点是否宕机；上述任务调度装置还包括替换单元350，用于从上述至少一个备用调度节点中选取一个备用调度节点作为新的调度节点。
在另一种可实施的方式中，上述任务调度方式还包括获取单元370，用于分别获取上述至少一个备用调度节点的创建时间；还包括确定单元360，用于确定上述至少一个备用 调度节点中创建时间最早的调度节点；上述替换单元350，具体用于将上述创建时间最早的备用调度节点作为上述新的调度节点。
在另一种可实施的方式中，上述获取单元370，还用于获取上述目标任务的标签，上述标签用于指示上述目标任务的重要程度；上述获取单元370，还用于获取上述目标任务的标签对应的目标数量，上述目标数量用于指示上述Actor模型中的备用调度节点的个数；上述创建单元320，还用于创建包括上述目标数量的备用调度节点的Actor模型。
在另一种可实施的方式中，上述多个工作节点还包括至少一个监控节点，上述监控节点用于初始化宕机的调度节点，上述检测单元310，还用于检测上述调度节点宕机；上述任务调度装置还包括修复单元380，用于利用上述监控节点对上述宕机的调度节点进行初始化，使得上述宕机的调度节点恢复正常。
在另一种可实施的方式中，上述检测单元310，用于检测已创建的工作节点的个数；上述获取单元370，用于获取上述已创建的工作节点的创建时间；上述删除单元340，还用于若上述已创建的工作节点的个数大于预设个数，则按照上述创建时间从早到晚的排序，依次删除上述已创建的工作节点，直到剩下的已创建的工作节点的个数小于或等于上述预设个数。
在另一种可实施的方式中，上述检测单元310，具体用于检测上述任务集群中的新增的任务；上述检测单元310，还具体用于若确定上述新增的任务为分布式任务，则确定检测到上述任务集群中存在新增的目标任务。
在本申请实施例中，每当检测单元检测到目标任务，创建单元就为该目标任务建立包括调度节点和任务节点的Actor模型，然后执行单元利用调度节点来调度任务节点执行目标任务，在检测单元检测到目标任务完成之后，删除单元删除该目标任务，可见，本申请所提供的任务调度方法中，由于调度节点和任务节点都是Actor，于是调度节点和任务节点是通过信息传递的方式进行通信的，而信息传递的好处就是不会产生数据竞争的问题，在处理任务的时候无需同步阻塞等待，于是本申请也便不需要借助于锁便可以实现复杂的分布式任务调度，并且大大提高分布式任务处理的并发性能。可见本申请实施例提供了一种高效的任务调度方法。
参见图4，是本申请另一实施例提供的一种任务调度装置示意框图。如图所示的本实施例中的任务调度装置可以包括：一个或多个处理器410和存储器420。上述处理器410和存储器420通过总线430连接。存储器420用于存储计算机程序，计算机程序包括程序指令，处理器410用于执行存储器420存储的程序指令。具体的：
处理器410，用于执行检测单元310的功能，用于检测任务集群中是否存在新增的目标任务；还用于执行创建单元320的功能，用于在检测到任务集群中存在新增的目标任务的情况下，为上述目标任务创建Actor模型，上述Actor模型包括多个工作节点，上述多个工作节点之间通过消息传递的方式通信，上述多个工作节点中至少存在两个工作节点分别 分布于不同的终端上，上述多个工作节点包括一个调度节点和至少一个任务节点，上述调度节点用于调用上述任务节点执行任务；还用于执行执行单元330的功能，用于利用上述调度节点通过上述消息传递的方式调用上述至少一个任务节点执行上述目标任务；还用于执行删除单元340的功能，用于在检测到上述目标任务完成之后，在上述任务集群中删除上述目标任务。
在另一种可实施的方式中，上述多个工作节还包括至少一个备用调度节点，上述备用调度节点用于替换宕机的调度节点，上述处理器410，还用于检测上述调度节点是否宕机；处理器410还用于执行替换单元350的功能，用于从上述至少一个备用调度节点中选取一个备用调度节点作为新的调度节点。
在另一种可实施的方式中，上述处理器410还用于执行获取单元370的功能，用于分别获取上述至少一个备用调度节点的创建时间；上述处理器410还用于执行确定单元360的功能，用于确定上述至少一个备用调度节点中创建时间最早的调度节点；上述处理器410还具体用于将上述创建时间最早的备用调度节点作为上述新的调度节点。
在另一种可实施的方式中，上述处理器410还用于获取上述目标任务的标签，上述标签用于指示上述目标任务的重要程度；还用于获取上述目标任务的标签对应的目标数量，上述目标数量用于指示上述Actor模型中的备用调度节点的个数；还用于创建包括上述目标数量的备用调度节点的Actor模型。
在另一种可实施的方式中，上述多个工作节点还包括至少一个监控节点，上述监控节点用于初始化宕机的调度节点，上述处理器410还用于检测上述调度节点宕机；上述处理器410还用于执行修复单元380的功能，用于利用上述监控节点对上述宕机的调度节点进行初始化，使得上述宕机的调度节点恢复正常。
在另一种可实施的方式中，上述处理器410还用于检测已创建的工作节点的个数；还用于获取上述已创建的工作节点的创建时间；还用于若上述已创建的工作节点的个数大于预设个数，则按照上述创建时间从早到晚的排序，依次删除上述已创建的工作节点，直到剩下的已创建的工作节点的个数小于或等于上述预设个数。
在另一种可实施的方式中，上述处理器410，具体用于检测上述任务集群中的新增的任务；还具体用于若确定上述新增的任务为分布式任务，则确定检测到上述任务集群中存在新增的目标任务。
应当理解，在本申请实施例中，所称处理器410可以是中央处理单元(Central Processing Unit，CPU)，该处理器还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
该存储器420可以包括只读存储器和随机存取存储器，并向处理器410提供指令和数 据。存储器420的一部分还可以包括非易失性随机存取存储器。例如，存储器420还可以存储设备类型的信息。
具体实现中，本申请实施例中所描述的处理器410可执行本申请实施例提供的任务调度方法的第一实施例和第二实施例中所描述的实现方式，也可执行本申请实施例所描述的任务调度装置的实现方式，在此不再赘述。
在本申请的另一实施例中提供一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，计算机程序包括程序指令，程序指令被处理器执行。
计算机可读存储介质可以是前述任一实施例的任务调度装置的内部存储单元，例如任务调度装置的硬盘或内存。计算机可读存储介质也可以是任务调度装置的外部存储设备，例如任务调度装置上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。进一步地，计算机可读存储介质还可以既包括任务调度装置的内部存储单元也包括外部存储设备。计算机可读存储介质用于存储计算机程序以及任务调度装置所需的其他程序和数据。计算机可读存储介质还可以用于暂时地存储已经输出或者将要输出的数据。

Claims (20)

  1. 一种基于Actor模型的任务调度方法，其特征在于，包括：
    检测任务集群中是否存在新增的目标任务；
    在检测到所述任务集群中存在新增的目标任务的情况下，为所述目标任务创建Actor模型，所述Actor模型包括多个工作节点，所述多个工作节点之间通过消息传递的方式通信，所述多个工作节点中至少存在两个工作节点分别分布于不同的终端上，所述多个工作节点包括一个调度节点和至少一个任务节点，所述调度节点用于调用所述任务节点执行任务；
    利用所述调度节点通过所述消息传递的方式调用所述至少一个任务节点执行所述目标任务；
    在检测到所述目标任务完成之后，在所述任务集群中删除所述目标任务。
  2. 根据权利要求1所述的方法，其特征在于，所述多个工作节还包括至少一个备用调度节点，所述备用调度节点用于替换宕机的调度节点，所述方法还包括：
    若检测到所述调度节点宕机，则从所述至少一个备用调度节点中选取一个备用调度节点作为新的调度节点。
  3. 根据权利要求2所述的方法，其特征在于，所述从所述至少一个备用调度节点中选取一个备用调度节点作为新的调度节点，包括：
    分别获取所述至少一个备用调度节点的创建时间，并确定所述至少一个备用调度节点中创建时间最早的调度节点；
    将所述创建时间最早的备用调度节点作为所述新的调度节点。
  4. 根据权利要求2所述的方法，其特征在于，所述为所述目标任务创建Actor模型，包括：
    获取所述目标任务的标签，所述标签用于指示所述目标任务的重要程度；
    获取所述目标任务的标签对应的目标数量，所述目标数量用于指示所述Actor模型中的备用调度节点的个数；
    创建包括所述目标数量的备用调度节点的Actor模型。
  5. 根据权利要求1所述的方法，其特征在于，所述多个工作节点还包括至少一个监控节点，所述监控节点用于初始化宕机的调度节点，所述方法还包括：
    若检测到所述调度节点宕机，则利用所述监控节点对所述宕机的调度节点进行初始化，使得所述宕机的调度节点恢复正常。
  6. 根据权利要求1所述的方法，其特征在于，所述在所述任务集群中删除所述目标任务之后，还包括：
    检测已创建的工作节点的个数，并获取所述已创建的工作节点的创建时间；
    若所述已创建的工作节点的个数大于预设个数，则按照所述创建时间从早到晚的排序， 依次删除所述已创建的工作节点，直到剩下的已创建的工作节点的个数小于或等于所述预设个数。
  7. 根据权利要求1所述的方法，其特征在于，所述检测任务集群中是否存在新增的目标任务，包括：
    检测所述任务集群中的新增的任务；
    若确定所述新增的任务为分布式任务，则确定检测到所述任务集群中存在新增的目标任务。
  8. 根据权利要求1所述的方法，其特征在于，所述在检测到所述任务集群中存在新增的目标任务的情况下，所述方法还包括：
    获取所述目标任务的标签，所述标签用于指示所述目标任务的重要程度；
    获取所述目标任务的标签对应所述目标任务的重要等级；
    若所述目标任务的重要等级超过了预设等级，则创建包括有至少一个监控节点的Actor模型，所述监控节点用于初始化宕机的调度节点。
  9. 根据权利要求1所述的方法，其特征在于，所述多个工作节还包括与所述至少一个任务节点分别对应的至少一个备用任务节点，所述备用任务节点用于替换宕机的任务节点，所述方法还包括：
    检测宕机的任务节点；
    若检测到宕机的任务节点，则从所述至少一个备用任务节点中选取出与宕机的任务节点对应的备用任务节点，并将所述与宕机的任务节点对应的备用任务节点作为新的任务节点。
  10. 一种基于Actor模型的任务调度装置，其特征在于，包括：
    检测单元，用于检测任务集群中是否存在新增的目标任务；
    创建单元，用于在检测到任务集群中存在新增的目标任务的情况下，为所述目标任务创建Actor模型，所述Actor模型包括多个工作节点，所述多个工作节点之间通过消息传递的方式通信，所述多个工作节点中至少存在两个工作节点分别分布于不同的终端上，所述多个工作节点包括一个调度节点和至少一个任务节点，所述调度节点用于调用所述任务节点执行任务；
    执行单元，用于利用所述调度节点通过所述消息传递的方式调用所述至少一个任务节点执行所述目标任务；
    删除单元，用于在检测到所述目标任务完成之后，在所述任务集群中删除所述目标任务。
  11. 一种任务调度装置，其特征在于，包括处理器和存储器，所述处理器和存储器相互连接；所述存储器用于存储计算机程序，所述计算机程序包括程序指令；所述处理器被配置用于调用所述程序指令，用以执行：
    检测任务集群中是否存在新增的目标任务；
    在检测到所述任务集群中存在新增的目标任务的情况下，为所述目标任务创建Actor模型，所述Actor模型包括多个工作节点，所述多个工作节点之间通过消息传递的方式通信，所述多个工作节点中至少存在两个工作节点分别分布于不同的终端上，所述多个工作节点包括一个调度节点和至少一个任务节点，所述调度节点用于调用所述任务节点执行任务；
    利用所述调度节点通过所述消息传递的方式调用所述至少一个任务节点执行所述目标任务；
    在检测到所述目标任务完成之后，在所述任务集群中删除所述目标任务。
  12. 根据权利要求11所述的装置，其特征在于，所述多个工作节还包括至少一个备用调度节点，所述备用调度节点用于替换宕机的调度节点，所述处理器还用于：
    若检测到所述调度节点宕机，则从所述至少一个备用调度节点中选取一个备用调度节点作为新的调度节点。
  13. 根据权利要求12所述的装置，其特征在于，所述处理器具体用于：
    分别获取所述至少一个备用调度节点的创建时间，并确定所述至少一个备用调度节点中创建时间最早的调度节点；
    将所述创建时间最早的备用调度节点作为所述新的调度节点。
  14. 根据权利要求12所述的装置，其特征在于，所述处理器具体用于：
    获取所述目标任务的标签，所述标签用于指示所述目标任务的重要程度；
    获取所述目标任务的标签对应的目标数量，所述目标数量用于指示所述Actor模型中的备用调度节点的个数；
    创建包括所述目标数量的备用调度节点的Actor模型。
  15. 根据权利要求11所述的装置，其特征在于，所述多个工作节点还包括至少一个监控节点，所述监控节点用于初始化宕机的调度节点，所述处理器还用于：
    若检测到所述调度节点宕机，则利用所述监控节点对所述宕机的调度节点进行初始化，使得所述宕机的调度节点恢复正常。
  16. 根据权利要求1所述的装置，其特征在于，所述处理器还用于：
    检测已创建的工作节点的个数，并获取所述已创建的工作节点的创建时间；
    若所述已创建的工作节点的个数大于预设个数，则按照所述创建时间从早到晚的排序，依次删除所述已创建的工作节点，直到剩下的已创建的工作节点的个数小于或等于所述预设个数。
  17. 根据权利要求1所述的装置，其特征在于，所述处理器具体用于：
    检测所述任务集群中的新增的任务；
    若确定所述新增的任务为分布式任务，则确定检测到所述任务集群中存在新增的目标任务。
  18. 根据权利要求11所述的装置，其特征在于，所述处理器还用于：
    获取所述目标任务的标签，所述标签用于指示所述目标任务的重要程度；
    获取所述目标任务的标签对应所述目标任务的重要等级；
    若所述目标任务的重要等级超过了预设等级，则创建包括有至少一个监控节点的Actor模型，所述监控节点用于初始化宕机的调度节点。
  19. 根据权利要求11所述的装置，其特征在于，所述多个工作节还包括与所述至少一个任务节点分别对应的至少一个备用任务节点，所述备用任务节点用于替换宕机的任务节点，所述处理器还用于：
    检测宕机的任务节点；
    若检测到宕机的任务节点，则从所述至少一个备用任务节点中选取出与宕机的任务节点对应的备用任务节点，并将所述与宕机的任务节点对应的备用任务节点作为新的任务节点。
  20. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序包括程序指令，所述程序指令当被处理器执行时使所述处理器执行如权利要求1-9任一项所述的方法。
PCT/CN2019/102741 2019-05-20 2019-08-27 一种基于Actor模型的任务调度方法、装置及存储介质 Ceased WO2020232875A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910421282.5A CN110231995B (zh) 2019-05-20 2019-05-20 一种基于Actor模型的任务调度方法、装置及存储介质
CN201910421282.5 2019-05-20

Publications (1)

Publication Number Publication Date
WO2020232875A1 true WO2020232875A1 (zh) 2020-11-26

Family

ID=67861430

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/102741 Ceased WO2020232875A1 (zh) 2019-05-20 2019-08-27 一种基于Actor模型的任务调度方法、装置及存储介质

Country Status (2)

Country Link
CN (1) CN110231995B (zh)
WO (1) WO2020232875A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112416562A (zh) * 2020-12-11 2021-02-26 深圳市思迪信息技术股份有限公司 一种分布式任务调度引擎的方法及装置
CN113722060A (zh) * 2021-07-30 2021-11-30 浪潮电子信息产业股份有限公司 一种告警任务管理方法、装置及相关组件
CN114610466A (zh) * 2022-03-23 2022-06-10 杭州隆埠科技有限公司 分布式任务调度系统及方法
CN118502935A (zh) * 2024-04-30 2024-08-16 福建天泉教育科技有限公司 一种处理任务数据的优化方法及终端

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110967621A (zh) * 2019-11-11 2020-04-07 北京中电华大电子设计有限责任公司 一种基于Actor模型的SE芯片测试设计方法
CN111026526B (zh) * 2019-11-12 2024-01-30 珠海格力电器股份有限公司 程序的定时器配置方法、装置、存储介质及终端设备
CN111723323B (zh) * 2020-06-11 2021-02-19 湖北美和易思教育科技有限公司 一种在线考试系统试卷答案数据传输方法及装置
CN112291321B (zh) * 2020-10-22 2023-08-08 北京梆梆安全科技有限公司 业务处理方法、装置及系统
CN112612604B (zh) * 2020-12-14 2022-03-22 上海哔哩哔哩科技有限公司 基于Actor模型的任务调度方法、装置
CN112685184A (zh) * 2020-12-31 2021-04-20 广州极飞科技有限公司 分布式任务调度的方法、任务调度平台及任务执行器
CN113703871A (zh) * 2021-08-31 2021-11-26 平安养老保险股份有限公司 基于自动化处理脚本的任务处理方法及其对应装置
CN115996215A (zh) * 2021-10-09 2023-04-21 腾讯科技（深圳）有限公司 实时数据推送的方法和装置
CN113961341A (zh) * 2021-10-15 2022-01-21 杭州安恒信息技术股份有限公司 基于Actor模型的并发数据处理方法、系统、设备和存储介质
CN113918640A (zh) * 2021-10-29 2022-01-11 中电四川数据服务有限公司 一种基于异步模式的etl流程执行方法
CN114253690B (zh) * 2021-12-20 2025-07-22 重庆市通信建设有限公司 任务调度方法、装置、电子设备及计算机可读存储介质
CN115695213A (zh) * 2022-10-26 2023-02-03 北京国泰网信科技有限公司 一种基于Actor模型的数据处理方法及系统
CN115904734A (zh) * 2022-12-29 2023-04-04 中国联合网络通信集团有限公司 任务请求调度集群

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106406983A (zh) * 2015-07-27 2017-02-15 阿里巴巴集团控股有限公司 一种集群中的任务调度方法及装置
US20180077520A1 (en) * 2016-09-15 2018-03-15 WiSilica Inc. Wireless control of multiple actor devices by multiple controller devices with communication security

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6345287B1 (en) * 1997-11-26 2002-02-05 International Business Machines Corporation Gang scheduling for resource allocation in a cluster computing environment
CN102831012A (zh) * 2011-06-16 2012-12-19 日立(中国)研究开发有限公司 多节点分布式系统中的任务调度装置和任务调度方法
CN103559082A (zh) * 2013-11-04 2014-02-05 北京华胜天成科技股份有限公司 一种基于队列的分布式任务调度方法、装置及系统
CN103593236B (zh) * 2013-11-14 2017-02-22 华为技术有限公司 一种业务流程调度的计算机设备、方法和装置
CN104461752B (zh) * 2014-11-21 2018-09-18 浙江宇视科技有限公司 一种两级故障容错的多媒体分布式任务处理方法
CN105630589A (zh) * 2014-11-24 2016-06-01 航天恒星科技有限公司 分布式流程调度系统及流程调度、执行方法
CN105912402B (zh) * 2016-04-11 2019-04-02 深圳益邦阳光有限公司 一种基于Actor模型的调度方法及装置
CN107623731B (zh) * 2017-09-15 2020-09-01 浪潮软件股份有限公司 一种任务调度方法、客户端、服务集群及系统
CN108304255A (zh) * 2017-12-29 2018-07-20 北京城市网邻信息技术有限公司 分布式任务调度方法及装置、电子设备及可读存储介质

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106406983A (zh) * 2015-07-27 2017-02-15 阿里巴巴集团控股有限公司 一种集群中的任务调度方法及装置
US20180077520A1 (en) * 2016-09-15 2018-03-15 WiSilica Inc. Wireless control of multiple actor devices by multiple controller devices with communication security

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
陈昊 等 (CHEN, HAO ET AL.): "基于Actor模型的高性能分布式XMPP服务器 (High-performance distributed XMPP server based on actor model)", 计算机系统应用 (COMPUTER SYSTEMS & APPLICATIONS), vol. 24, no. 10, 15 October 2015 (2015-10-15), ISSN: 1003-3254, DOI: 20200207124235X *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112416562A (zh) * 2020-12-11 2021-02-26 深圳市思迪信息技术股份有限公司 一种分布式任务调度引擎的方法及装置
CN112416562B (zh) * 2020-12-11 2024-06-04 深圳市思迪信息技术股份有限公司 一种分布式任务调度引擎的方法及装置
CN113722060A (zh) * 2021-07-30 2021-11-30 浪潮电子信息产业股份有限公司 一种告警任务管理方法、装置及相关组件
CN114610466A (zh) * 2022-03-23 2022-06-10 杭州隆埠科技有限公司 分布式任务调度系统及方法
CN118502935A (zh) * 2024-04-30 2024-08-16 福建天泉教育科技有限公司 一种处理任务数据的优化方法及终端

Also Published As

Publication number Publication date
CN110231995A (zh) 2019-09-13
CN110231995B (zh) 2023-08-08

Similar Documents

Publication Publication Date Title
WO2020232875A1 (zh) 一种基于Actor模型的任务调度方法、装置及存储介质
CN110113420B (zh) 基于nvm的分布式消息队列管理系统
US11709620B2 (en) Methods and systems for memory management in a publish and subscribe system
CN110908788B (zh) 基于Spark Streaming的数据处理方法、装置、计算机设备及存储介质
EP2746948A1 (en) Device and method for optimization of data processing in a MapReduce framework
US8381230B2 (en) Message passing with queues and channels
CN106933672B (zh) 一种分布式环境协调消费队列方法和装置
CN101464811A (zh) 多任务监控管理系统
CN114401239B (zh) 元数据传输方法、装置、计算机设备和存储介质
WO2019223599A1 (zh) 数据采集系统、方法、节点设备及存储介质
US8543722B2 (en) Message passing with queues and channels
US8631086B2 (en) Preventing messaging queue deadlocks in a DMA environment
CN115221116A (zh) 一种数据写入方法、装置、设备及可读存储介质
CN110955461B (zh) 计算任务的处理方法、装置、系统、服务器和存储介质
US20220155975A1 (en) Methods and systems for providing a lockless access to a shared memory region in a publish and subscribe system
WO2020232870A1 (zh) 一种基于Actor模型的数据采集方法、装置及存储介质
US11363099B2 (en) Methods and systems for enabling publish-subscribe message transmission in a distributed environment
JPH02118756A (ja) マルチプロセッサ・システムにおけるメッセージ通信制御システム
CN119783340A (zh) 基于负载均衡的并行仿真方法、介质及设备
CN119743366A (zh) 一种Paxos消息发送方法、装置、设备、介质及产品
Lifflander et al. Adoption protocols for fanout-optimal fault-tolerant termination detection
US20230376339A1 (en) Methods and apparatus for managing task timeouts within distributed computing networks
CN118689591A (zh) 用于处理计算任务的方法、装置、介质和程序产品
CN112153114A (zh) 一种分时访问共享器件的方法、系统、设备及介质
WO2022237570A1 (zh) 死锁检测方法、装置以及相关设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19929809

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19929809

Country of ref document: EP

Kind code of ref document: A1