WO2021212965A1 - 一种资源调度方法及相关装置 - Google Patents

一种资源调度方法及相关装置 Download PDF

Info

Publication number
WO2021212965A1
WO2021212965A1 PCT/CN2021/075526 CN2021075526W WO2021212965A1 WO 2021212965 A1 WO2021212965 A1 WO 2021212965A1 CN 2021075526 W CN2021075526 W CN 2021075526W WO 2021212965 A1 WO2021212965 A1 WO 2021212965A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
task
node
computing
execution
Prior art date
Application number
PCT/CN2021/075526
Other languages
English (en)
French (fr)
Inventor
黄小建
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to EP21792088.3A priority Critical patent/EP4123449A4/en
Publication of WO2021212965A1 publication Critical patent/WO2021212965A1/zh
Priority to US17/970,232 priority patent/US20230037783A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/485—Task life-cycle, e.g. stopping, restarting, resuming execution
    • G06F9/4856—Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration
    • G06F9/4862—Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration the task being a mobile agent, i.e. specifically designed to migrate
    • G06F9/4875—Task life-cycle, e.g. stopping, restarting, resuming execution resumption being on a different machine, e.g. task migration, virtual machine migration the task being a mobile agent, i.e. specifically designed to migrate with migration policy, e.g. auction, contract negotiation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G06F9/4887—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues involving deadlines, e.g. rate based, periodic
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/501—Performance criteria
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/50—Indexing scheme relating to G06F9/50
    • G06F2209/506—Constraint

Definitions

  • This application relates to the field of big data technology, and in particular to a resource scheduling method and related devices.
  • FIG. 1 is a schematic diagram of the architecture of a resource scheduling system.
  • the scheduling node 102 receives the task submitted by the client 101, and allocates the task according to the computing resources required by the task. To the computing module in the computing node 103 for execution.
  • the open source big data processing platform Hadoop launched by Apache is responsible for the system's resource allocation and task scheduling through the resource scheduling system Yarn.
  • Yarn can allow multiple computing frameworks (programming framework MapReduce, memory computing framework Spark, streaming computing framework Strom, graph computing framework, etc.) to run in a system, providing unified resource allocation services for different parallel computing, making the system have The advantages of easy operation and maintenance, resource elasticity and scalability, and data sharing.
  • the resource scheduling system is one of the core components in the big data platform, and the scheduling strategy of the resource scheduling system will directly affect the task allocation of distributed resources, thereby affecting the overall performance of the distributed resource system.
  • the current resource scheduling system is based on the task's demand for processing resources to decide which computing module of which computing node to schedule the task for processing. For example: Yarn schedules the task according to the computing power or video memory required by the task submitted by the client .
  • a scheduling strategy does not take into account the differences in tasks submitted by users, so that there are a large number of resource fragments in the computing nodes, and the processing resources cannot be fully utilized.
  • Figure 1 is a schematic diagram of the operating scenario of a resource scheduling system. Some tasks submitted by the client 201 require a short execution time, such as task 1, task 3, task 4, etc., which occupy computing resources. The time is very small, and some tasks require a long execution time, such as task 2 and so on.
  • the time to occupy computing resources is usually longer. If the scheduling system 202 assigns tasks 1, task 2, and task 3 to the same computing When processing on a computing module (that is, computing module 1) on a node (that is, computing node 1), when the short execution time (such as task 1, task 2) is finished and released, the remaining fragment resources in the computing node cannot be used by others The use of tasks is a serious waste of resources, which will greatly reduce the utilization of resources, which directly affects the efficiency of task execution and ultimately leads to a decline in overall system performance.
  • the embodiments of the present application disclose a resource scheduling method and related devices, which can reduce resource fragments in computing nodes and improve resource utilization.
  • an embodiment of the present application discloses a resource scheduling method, including:
  • the scheduling node obtains the target execution time level to which the task belongs, where the target execution time level is used to represent the time length, and the target execution time level indicates that the target computing module of the target computing node among the multiple computing nodes is used to execute the task ;
  • the scheduling node preferentially sends the task to the target computing node corresponding to the target execution time level.
  • the scheduling node establishes the corresponding relationship between the execution duration level and the computing node (or the computing module in the computing node).
  • a task is received, it is scheduled to the computing node corresponding to the execution duration level to which the task belongs.
  • Tasks with the same execution duration level can be scheduled to be executed on the same computing node first, that is, a computing node
  • the execution time levels of tasks processed in the same computing module are usually the same. In this way, when a task with a short execution time is completed, it is easier to obtain a whole free computing module, thereby reducing resource fragmentation in the computing node and improving resource utilization.
  • the method before the scheduling node obtains the target execution duration level to which the task belongs, the method further includes:
  • the scheduling node receives a scheduling request for the task, and the scheduling request includes a target execution duration level;
  • the scheduling node obtains the target execution time level to which the task belongs, including:
  • the scheduling node parses the foregoing scheduling request to obtain the target execution duration level.
  • the scheduling request can carry indication information for indicating the execution duration level, so that the scheduling node can schedule the task according to the target execution duration indicated by the indication information.
  • the obtaining of the target execution duration level to which the task belongs by the scheduling node includes:
  • the scheduling node determines an execution duration level corresponding to the execution duration of the task, and the determined execution duration level is the target execution duration level.
  • the scheduling node when it receives a task, it can determine the execution time required by the task according to the information of the task, and then can determine the target execution time level corresponding to the execution time. For example, the scheduling node stores the execution time of the previously scheduled task. When the scheduling node receives the task, it can estimate the execution time of the task according to the execution time of the similar task previously scheduled, and then obtain the target execution time level of the task. .
  • the target execution time level is one of the three execution time levels; the above three execution time levels include long execution time, medium execution time, and short execution time. Execution time
  • the long execution time is greater than or equal to the first threshold
  • the execution time is less than the first threshold and greater than the second threshold
  • the short execution duration is less than or equal to the second threshold.
  • the scheduling node preferentially sends the task to the target computing node corresponding to the target execution duration level, including:
  • the task is sent to the target computing node.
  • the target execution duration level may indicate that the target computing module of the target computing node among the multiple computing nodes is used to execute the task. Therefore, the scheduling node sends the task to the target computing node, so that the target in the target computing node
  • the execution time levels of tasks executed by the computing modules are the same. In this way, when tasks with a shorter execution time are completed, larger idle computing resources can be obtained, which reduces resource fragmentation in the computing modules.
  • the target calculation module is a calculation module marked with a first label, and the first label characterizes that the calculation module executes other tasks with the target execution duration level.
  • the calculation module may be marked with different tags, and different tags indicate that the calculation module currently processes tasks with different execution duration levels.
  • the scheduling node performs scheduling, it can prioritize the scheduling of tasks to the computing modules that are processing tasks of the same execution duration level.
  • the method of computing modules with different tags corresponding to different duration levels can be used. Improve the flexibility of system resource allocation, thereby improving resource utilization.
  • the scheduling node preferentially sends the task to the target computing node corresponding to the target execution duration level, including:
  • the task is sent to other computing nodes among the above-mentioned multiple computing nodes.
  • sending the task to other computing nodes among the multiple computing nodes includes:
  • the task is sent to the first computing node, where there is a computing module of an unprocessed task in the first computing node.
  • the method further includes:
  • the scheduling node sends the task to the second computing node, where the second computing node has a computing module marked with a second label, wherein, the second tag indicates that other tasks of the first execution duration level are being processed in the calculation module, and the execution time required for the task of the first execution duration level is shorter than the execution time required for the task.
  • the method further includes:
  • the scheduling node sends the task to a third computing node, where the second computing node has a computing module marked with a third label, wherein, the third tag represents that other tasks of the second execution duration level are being processed in the calculation module, and the execution time required for the second execution duration level task is longer than the execution time required for the task.
  • the method further includes:
  • the scheduling node marks a first label for the label of the target computing module of the target computing node, and the first label represents the execution of the task with the target execution duration level in the computing module.
  • the scheduling node can modify the label of the computing module so that the label of the computing module corresponds to the execution time level of the task it is processing, which is convenient for subsequent scheduling of other tasks.
  • the method further includes:
  • the scheduling node deletes the tag of the target computing module.
  • an embodiment of the present application discloses a resource scheduling device, including:
  • the processing unit is configured to obtain the target execution time level to which the task belongs, where the target execution time level is used to represent the time length, and the target execution time level indicates that the target calculation module of the target calculation node among the multiple calculation nodes is used for execution
  • the target execution time level is used to represent the time length
  • the target execution time level indicates that the target calculation module of the target calculation node among the multiple calculation nodes is used for execution
  • the sending unit is configured to preferentially send the task to the target computing node corresponding to the target execution time level.
  • the resource scheduling device establishes a corresponding relationship between the execution duration level and the computing node (or computing module in the computing node).
  • a task is received, it is scheduled to the computing node corresponding to the execution duration level to which the task belongs.
  • Tasks with the same execution duration level can be scheduled to be executed on the same computing node first, that is, a computing node
  • the execution time levels of tasks processed in the same computing module are usually the same. In this way, when a task with a short execution time is completed, it is easier to obtain a whole free computing module, thereby reducing resource fragmentation in the computing node and improving resource utilization.
  • the receiving unit is further configured to receive a scheduling request for the task, and the scheduling request includes a target execution duration level;
  • the processing unit is specifically used to parse the scheduling request to obtain the target execution time level.
  • the scheduling request can carry indication information for indicating the execution duration level, so that the scheduling node can schedule the task according to the target execution duration indicated by the indication information.
  • the processing unit is specifically configured to determine an execution duration level corresponding to the required execution duration of the task, and the determined execution duration level is the target execution duration level.
  • the above-mentioned device when the above-mentioned device receives a task, it can determine the execution time required by the task according to the information of the task, and then can determine the target execution time level corresponding to the execution time. For example, the foregoing device stores the execution duration of a previously scheduled task. When the foregoing device receives a task, it can estimate the execution duration of the task based on the execution duration of a similar task previously scheduled, thereby obtaining the target execution duration level of the task.
  • the target execution duration level is one of the three execution duration levels; the foregoing three execution duration levels include long execution duration, medium execution duration, and short execution duration. Execution time
  • the long execution time is greater than or equal to the first threshold
  • the execution time is less than the first threshold and greater than the second threshold
  • the short execution duration is less than or equal to the second threshold.
  • the sending unit is specifically configured to send the task to the target computing node when there is a target computing node with a target computing module.
  • the target execution time level may indicate that the target computing module of the target computing node among the multiple computing nodes is used to execute the task. Therefore, the above-mentioned device sends the task to the target computing node, so that the target in the target computing node
  • the execution time levels of tasks executed by the computing modules are the same. In this way, when tasks with a shorter execution time are completed, larger idle computing resources can be obtained, which reduces resource fragmentation in the computing modules.
  • the target calculation module is a calculation module marked with a first label, and the first label characterizes that the calculation module executes other tasks with the target execution duration level.
  • the calculation module may be marked with different tags, and different tags indicate that the calculation module currently processes tasks with different execution duration levels.
  • the above-mentioned device performs scheduling, tasks can be prioritized to be scheduled to calculation modules that are processing tasks of the same execution duration level.
  • the calculation modules of different duration levels corresponding to different tags can be used. Improve the flexibility of system resource allocation, thereby improving resource utilization.
  • the sending unit is specifically configured to send the task to other computing nodes when there is no target computing node with the target computing module.
  • the sending unit is specifically configured to send the task to the first computing node when there is no target computing node with the target computing module, where the first node There are calculation modules for unprocessed tasks.
  • the sending unit is further configured to send the target computing node to the second computing node if there is no target computing node with the target computing module, and the first computing node does not exist.
  • Task where the second computing node has a computing module marked with a second label, where the second label indicates that other tasks of the second execution duration level are being processed in the computing module, and the tasks of the second execution duration level are The execution time required is shorter than the execution time required for the task.
  • the sending unit is further configured to send the target computing node to the third computing node if there is no target computing node with the target computing module, and the first computing node does not exist.
  • Task where the second computing node has a computing module marked with a third label, where the third label represents that other tasks of the third execution duration level are being processed in the computing module, and the tasks of the third execution duration level are The execution time required is longer than the execution time required for the task.
  • the processing unit is further configured to mark a first label for the label of the target calculation module of the target calculation node, and the first label represents the execution time level of the target calculation module in the calculation module. Task.
  • the above-mentioned device can modify the label of the calculation module so that the label of the calculation module corresponds to the execution time level of the task it is processing, which is convenient for subsequent scheduling of other tasks.
  • the processing unit is further configured to delete the first tag of the target calculation module if the task execution ends and the target calculation module has no other tasks being processed.
  • the embodiments of the present application disclose a scheduling node, including a processor and a memory; the processor is used to execute computer instructions stored in the memory, so that the scheduling node implements the first aspect or any possible implementation of the first aspect The method described in the method.
  • the scheduling node further includes a communication interface, and the processor is specifically configured to:
  • the target execution time level is used to represent the time length, and the target execution time level indicates that the target computing module of the target computing node among the multiple computing nodes is used to execute the task;
  • the task is preferentially sent to the target computing node corresponding to the target execution time level through the above-mentioned communication interface.
  • the scheduling node establishes the corresponding relationship between the execution duration level and the computing node (or the computing module in the computing node). After receiving a task, it is scheduled to the calculation corresponding to the execution duration level to which the task belongs.
  • tasks with the same execution duration level can be preferentially scheduled for execution on the same computing node, that is, the execution duration levels of tasks processed in the same computing module in a computing node are usually the same. In this way, when a task with a short execution time is completed, it is easier to obtain a whole free computing module, thereby reducing resource fragmentation in the computing node and improving resource utilization.
  • the processor is specifically configured to: receive a scheduling request for the task through the communication interface, where the scheduling request includes a target execution duration level;
  • the scheduling request can carry indication information for indicating the execution duration level, so that the scheduling node can schedule the task according to the target execution duration indicated by the indication information.
  • the processor is specifically configured to determine an execution duration level corresponding to the execution duration of the task, and the determined execution duration level is the target execution duration level.
  • the scheduling node when it receives a task, it can determine the execution time required by the task according to the information of the task, and then can determine the target execution time level corresponding to the execution time. For example, the scheduling node stores the execution time of the previously scheduled task. When the scheduling node receives the task, it can estimate the execution time of the task according to the execution time of the similar task previously scheduled, and then obtain the target execution time level of the task.
  • the target execution time level is one of the three execution time levels; the above three execution time levels include long execution time, medium execution time, and short execution time. Execution time
  • the long execution time is greater than or equal to the first threshold
  • the execution time is less than the first threshold and greater than the second threshold
  • the short execution duration is less than or equal to the second threshold.
  • the processor is specifically configured to: when there is a target computing node with a target computing module, send the task to the target computing node through the aforementioned communication interface.
  • the target execution duration level may indicate that the target computing module of the target computing node among the multiple computing nodes is used to execute the task. Therefore, the scheduling node sends the task to the target computing node, so that the target in the target computing node
  • the execution time levels of tasks executed by the computing modules are the same. In this way, when tasks with a shorter execution time are completed, larger idle computing resources can be obtained, which reduces resource fragmentation in the computing modules.
  • the target calculation module is a calculation module marked with a first label, and the first label characterizes that the calculation module executes other tasks with the target execution duration level.
  • the calculation module may be marked with different tags, and different tags indicate that the calculation module currently processes tasks with different execution duration levels.
  • the scheduling node performs scheduling, it can prioritize the scheduling of tasks to the computing modules that are processing tasks of the same execution duration level.
  • the method of computing modules with different tags corresponding to different duration levels can be used. Improve the flexibility of system resource allocation, thereby improving resource utilization.
  • the processor is specifically configured to: when there is no target computing node with a target computing module, send to other computing nodes of the multiple computing nodes through the communication interface. The task.
  • the processor is specifically configured to send the task to the first computing node through the communication interface when there is no target computing node with the target computing module, wherein the There are calculation modules for unprocessed tasks in the calculation points of the first section.
  • the processor is further configured to: if there is no target computing node with the target computing module, and there is no first computing node, then send to the second computing node through the above-mentioned communication interface.
  • the node sends the task, where the second computing node has a computing module marked with a second label, where the second label indicates that other tasks of the first execution duration level are being processed in the computing module, and the first execution duration level
  • the execution time required for the task of is shorter than the execution time required for the task.
  • the processor is further configured to: if there is no target computing node with the target computing module, and there is no first computing node, then send to the third computing node through the above communication interface The node sends the task, where the second computing node has a computing module marked with a third label, where the third label represents that other tasks of the second execution duration level are being processed in the computing module, and the second execution duration level The execution time required for the task of is longer than the execution time required for the task.
  • the processor is further configured to: mark a first label for the label of the target computing module of the target computing node, and the first label represents the execution time level of the target computing module in the computing module. Task.
  • the scheduling node can modify the label of the computing module so that the label of the computing module corresponds to the execution time level of the task it is processing, which is convenient for subsequent scheduling of other tasks.
  • the processor is further configured to delete the first tag of the target calculation module if the execution of the task ends and the target calculation module has no other tasks being processed.
  • an embodiment of the present application discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the first aspect or any of the first aspects.
  • the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the first aspect or any of the first aspects.
  • an embodiment of the present application discloses a chip system that includes at least one processor, a memory, and an interface circuit.
  • the interface circuit is used to provide information input/output for the at least one processor, and the memory stores
  • an embodiment of the present application discloses a resource scheduling system.
  • the resource scheduling system includes a resource scheduling device, at least one client, and at least one computing node. Wherein, the resource scheduling device is used to receive a task from the at least one client, and send the task to at least one computing node for execution.
  • the resource scheduling device is the resource scheduling device described in the second aspect or any one of the possible implementation manners of the second aspect.
  • FIG. 1 is a schematic diagram of the architecture of a resource scheduling system provided by an embodiment of the present application
  • FIG. 2 is a schematic diagram of an operation scenario of resource scheduling provided by an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of another resource scheduling system provided by an embodiment of the present application.
  • FIG. 4 is a schematic flowchart of a resource scheduling method provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a scheduling request provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of another operation scenario of resource scheduling provided by an embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of another resource scheduling system provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a device provided by an embodiment of the present application.
  • Fig. 9 is a schematic structural diagram of a scheduling node provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of the architecture of a resource scheduling system provided by an embodiment of the present application, which includes a client 301, a scheduling node 302, and at least one compute node 303. in:
  • the client 301 is an electronic device with the ability to send and receive data, and can be a physical device such as a host, a server, etc., or a virtual device such as a virtual machine, a container, and the like.
  • the client 301 is used to submit a task, and the task needs to be executed by a computing module in the computing node 302.
  • the task submitted by the client can carry the program package to execute the task (or the mirror file corresponding to the algorithm, etc.), and specify the computing power required for the task (for example, the required central processing unit (CPU)) One or more of the number of resources, the number of graphics processor (Graphics Processing Unit, GPU) resources, the size of memory, etc.), the computing node dispatches tasks to the computing node for calculation, and returns the result to the client.
  • the client can submit a face recognition task, carry a program package for face recognition or a mirror file of a face recognition algorithm and corresponding database files, etc., and specify the computing power required for the face recognition task (for example, It needs 8-core CPU, 30% GPU resources, 8G memory). After the computing node calculates, it returns the result of face recognition to the client.
  • the computing node 302 is a device used to provide computing services, and may be a physical device such as a host, a server, etc., or a virtual device such as a virtual machine, a container, and the like.
  • the computing node 301 includes one or more computing modules for providing computing resources.
  • the computing modules may include a graphics processor (Graphics Processing Unit, GPU), a microprocessor (Micro Processing Unit, MPU), a sound card (Sound Card), One or more of accelerator cards used for artificial intelligence calculations, etc. Further, the computing module may also include computing resources such as CPU and memory.
  • the scheduling node 303 is one or more nodes on which a scheduling system is deployed.
  • the node may be a physical device such as a host, a server, etc., or a virtual device such as a virtual machine and a container.
  • the scheduling node 302 can receive the task from the client 301, and schedule the task to the computing module in the computing node 302 for execution according to the scheduling policy.
  • the client 301, the computing node 302, and the scheduling node 303 may be deployed in different physical devices, or may be deployed in the same physical device, which is not limited in this application.
  • FIG. 4 is a resource scheduling method provided by an embodiment of the present application.
  • the method can be implemented based on the resource scheduling system shown in FIG. 3.
  • the method includes but is not limited to the following steps:
  • Step S401 The scheduling node receives the task.
  • the task is a task submitted by the client.
  • the task can be a face recognition task, a video decoding task, a deep learning model training task, and so on.
  • the task can carry the corresponding program package (or the mirror file corresponding to the algorithm, etc.).
  • the computing power required for the task for example, one or more of the number of CPU resources required, the number of GPU resources, the memory size, etc. can also be carried.
  • Step S402 The scheduling node obtains the target execution duration level to which the task belongs.
  • the task submitted by the client has multiple execution duration levels
  • the target execution duration level can be used to indicate the length of time, and/or can instruct to use the target computing module of the target computing node to execute the task.
  • this application provides three levels of execution duration: long execution duration, medium execution duration, and short execution duration.
  • the long execution duration is greater than or equal to the first threshold
  • the medium execution duration is less than the first threshold and greater than the second threshold
  • the short The execution time is less than or equal to the second threshold.
  • Method 1 The scheduling node parses the scheduling request for the task, and obtains the execution time level of the task. Specifically, the scheduling node receives a scheduling request for the task, and the scheduling request includes the execution duration level of the task. For example, when the client submits a task, the submitted task carries indication information for indicating the execution duration level, so that the scheduling node can schedule the task according to the target execution duration indicated by the indication information.
  • FIG. 5 is a schematic diagram of a possible scheduling request provided by an embodiment of the present application, where the area 501 represents a possible scheduling request received by the scheduling node, including the name and job number of the scheduling request. (jobidentification, jobID), memory (memory), priority (priority), execution time level (taskcycle), etc.
  • the scheduling node analyzes the scheduling request shown in the area 501 to obtain the execution time level shown in the area 502, that is, the target execution time level.
  • the scheduling request may also carry one or more of the number of CPU resources and the number of GPU resources (not shown in FIG. 5) of the task.
  • Method 2 The scheduling node determines the target execution time level corresponding to the execution time required by the task. Specifically, when the scheduling node receives a task, it determines the execution time required by the task according to the information of the task, and then can determine the target execution time level corresponding to the execution time.
  • the scheduling node stores the execution time information of the previously scheduled task.
  • the scheduling node can estimate the execution time of the task according to the execution time of the similar task previously scheduled, and then obtain the target execution time level of the task.
  • the three execution duration levels are long execution duration, medium execution duration, and short execution duration
  • the scheduling node estimates that the execution duration of the task is greater than or equal to the first threshold, then the execution duration to which the task belongs The level is long execution time. For example, taking the first threshold of 24 hours as an example, if the scheduling node estimates that the execution time required for the task is greater than or equal to 24 hours, the execution time level of the task can be determined to be the long execution time.
  • the execution time required by the estimated task is less than the first threshold and greater than the second threshold, it is a medium execution time; if the execution time required by the estimated task is less than or equal to the first and second thresholds, it is a short execution time.
  • the scheduling node can estimate the execution time of the task according to the image ID of the algorithm, and then obtain the target execution time level of the task. For example, when the three execution time levels are long execution time, medium execution time, and short execution time, taking the second threshold of 10 minutes as an example, if the algorithm mirror number carried by the task indicates a portrait search algorithm, because The portrait search algorithm usually takes less than 10 minutes to process, so the execution time level of the task can be determined as short execution time.
  • Step S403 The scheduling node preferentially sends tasks to the target computing node corresponding to the target execution time level.
  • the embodiment of the present application establishes the corresponding relationship between the execution level duration and the computing node (or the computing module in the computing node), and when a task is received, it is scheduled to the computing node corresponding to the execution duration level to which the task belongs
  • the corresponding tasks with the same execution duration level can be preferentially scheduled to be executed on the same computing node. That is to say, the execution duration level of the tasks processed in the same computing module in a computing node is usually Are the same. In this way, when a task with a short execution time ends, it is easier to obtain a whole free computing module, thereby reducing resource fragmentation in the computing node and improving resource utilization.
  • FIG. 6 is a schematic diagram of a possible resource scheduling operation scenario provided by an embodiment of the present application.
  • the execution duration levels of the two tasks submitted by the client 301 are long execution duration and short execution duration, respectively.
  • the short execution time corresponds to computing node 1 (further, it can specifically correspond to computing module 1 and computing module 2 in computing node 1)
  • the long execution time corresponds to computing node 2 (further, it can also specifically correspond to computing node 2 the calculation module 3).
  • the short execution time indicates that the calculation module 1 or the calculation module 2 in the computing node 1 is used to execute the task, so the scheduling node preferentially sends tasks with the short execution time to the computing node 1 for execution.
  • a whole block of computing modules may exist in the computing node 1 for performing other tasks, which reduces resource fragmentation and improves resource utilization.
  • a corresponding mechanism is used to ensure that each execution duration level corresponds to a computing node.
  • the scheduling node directly sends tasks to the target computing node corresponding to the target execution duration level.
  • the target row duration level has a corresponding target computing node, but with the release of the task on the target computing node or the execution of other operations, at a later time or time Segment, the target row duration level does not have a corresponding target computing node.
  • the scheduling node to preferentially send tasks to the target computing node corresponding to the target execution time level can be specifically as follows:
  • the task When there is a target computing node with a target computing module, the task is sent to the target computing node. For example, when the execution time level of the task is short execution time, it is instructed to use the computing module of computing node 1 among multiple computing nodes to execute the task. If the computing node 1 has the computing module, the task is scheduled to computing node 1. Executed on the calculation module.
  • the calculation module can be marked with different labels, and the different labels represent the current processing tasks with different execution duration levels.
  • the target calculation module is the calculation module marked with the first label.
  • the label characterizes the execution of tasks with a target execution time level in the calculation module.
  • FIG. 7 is a schematic diagram of another possible resource scheduling operation scenario provided by an embodiment of the present application.
  • the execution duration level of the task submitted by the client 301 may be medium execution duration, short execution duration or long execution duration. Medium execution time, short execution time and long execution time respectively correspond to the calculation modules labeled "label A", "label B", and "label C”.
  • the computing node 701 includes a computing module 702 and a computing module 703. Task 1 is being executed in the computing module 702, and the execution duration level to which task 1 belongs is "medium execution duration", so the label of the computing module is "label A" ; There is no task being processed in the calculation module 703, and the label is empty.
  • the computing node 704 includes a computing module 705 and a computing module 706, where task 2 is being executed in the computing module 705, and the execution duration level to which task 2 belongs is "short execution duration", so the label of the computing module is "label B"; Task 3 is being executed in the calculation module 706, and the execution duration level to which task 3 belongs is "long execution duration", so the label of the calculation module is "label C”.
  • the execution time level of the task submitted by the client is medium execution time, if there is a calculation module labeled "label A", the task is sent to the calculation module of the "label A" calculation module (ie calculation module 702) It executes on the node (ie, computing node 701).
  • the task is sent to other computing nodes. For example, when the execution time level of the task is short execution time, it is instructed to use the computing module of the computing node 1 among the multiple computing nodes to execute the task. If the computing node 1 does not have the computing module or the computing module cannot be executed anymore The task is scheduled to be executed in other computing nodes. Further, there can be the following optional cases:
  • Case 1 When there is no target computing node with the target computing module or when the target computing module cannot perform the task, the task is sent to the computing node with the unprocessed computing module.
  • the computing module of the unprocessed task can be regarded as the idle computing module.
  • the computing node with the idle computing module is referred to as the first computing node. It can be seen that when there is no target computing node with a target computing module or the target computing module cannot execute the task, the task can be scheduled to an idle computing module for processing.
  • the calculation module can be marked with different labels, and the target calculation module is a calculation module marked with a first label, and the first label represents the execution of a task with a target execution time level in the calculation module.
  • the target execution time level indicates that the computing node to which the computing module with the first label belongs is used to execute the task. If there is no computing module with the first label, the task is sent to the computing node to which the computing module without the label belongs. For example, referring to Figure 7, when the execution duration level of the task submitted by the client is medium execution duration, if there is no calculation module with the label "label A", the task is sent to the calculation module without the label (ie, calculation module 703). ) Is executed on the computing node (ie, computing node 701) to which it belongs.
  • Case 3 If there is no target computing node with the target computing module, and there is no first computing node, then the task is sent to the computing node to which the computing module with the second label belongs, where the second label represents the computing module The task of the first execution duration level is being processed, and the execution time required for the task of the first execution duration level is shorter than the execution time required by the task.
  • the computing node to which the computing module marked with the second label belongs is referred to as the second computing node.
  • Case 4 If there is no target computing node with a target computing module, and there is no first computing node, then the task is sent to the computing node to which the computing module with the third label belongs, where the third label represents the computing module
  • the task of the second execution duration level is being processed, and the execution time required for the task of the second execution duration level is longer than the execution time required by the task.
  • the computing node to which the computing module marked with the third label belongs is referred to as the third computing node.
  • the scheduling node may modify the label of the computing module so that the label of the computing module corresponds to the execution time level of the task it is processing, which is convenient for subsequent task scheduling.
  • the execution time level of the task submitted by the client is short execution time.
  • the scheduling node sends the task to the computing node 701
  • the computing module 703 executes the task, so the scheduling node can modify the label of the computing module 703 As "Label A". In this way, when a task with an execution duration level of medium execution duration is subsequently received, it can be scheduled to the calculation module 703 for execution in priority.
  • the label of the calculation module corresponds to the execution time level of the task it is processing. It can be specifically that the label of the calculation module corresponds to the highest execution time level in the task being processed, that is, the longest execution time level. Corresponding tasks. For example, in Case 3, if the execution duration level of the task submitted by the client belongs to short execution duration, after the scheduling node sends the task to the computing node 704, the computing module 705 executes the task, because the original label of the computing module 705 is “label B”. "Corresponds to the short execution duration of the previous processing, and the execution duration level to which the task submitted by the client belongs is longer than the short execution duration, so the scheduling node can modify the label of the calculation module 705 to “label A”.
  • the computing node may delete the label of the computing module.
  • the execution duration level to which the task belongs is short execution duration, and after the scheduling node sends the task to the computing node 701, the label of the computing module 703 is modified to "label A".
  • the scheduling node can delete the label of the calculation module 703, indicating that the calculation module 703 is an idle calculation module.
  • the scheduling node establishes the corresponding relationship between the execution time level and the computing node (or the computing module in the computing node).
  • a task is received, it is scheduled to the computing node corresponding to the execution duration level to which the task belongs.
  • Tasks with the same execution duration level can be scheduled to be executed on the same computing node first, that is, a computing node
  • the execution time levels of tasks processed in the same computing module are usually the same. In this way, when a task with a short execution time is completed, it is easier to obtain a whole free computing module, thereby reducing resource fragmentation in the computing node and improving resource utilization.
  • FIG. 8 is a schematic structural diagram of an apparatus 80 provided by an embodiment of the present application.
  • the apparatus 80 may include a receiving unit 801, a processing unit 802, and a sending unit 803.
  • the description of each unit is as follows:
  • the receiving unit 801 is used to receive tasks
  • the processing unit 802 is configured to obtain the target execution time level to which the task belongs, where the target execution time level is used to represent the time length, and the target execution time level indicates that the target calculation module of the target calculation node among the multiple calculation nodes is used for execution. Task;
  • the sending unit 803 is configured to preferentially send tasks to the target computing node corresponding to the target execution time level.
  • the device 80 establishes the corresponding relationship between the execution duration level and the computing node (or the computing module in the computing node), and when a task is received, it is scheduled to the execution duration level to which the task belongs.
  • the computing node of the computing node tasks with the same execution time level can be scheduled to be executed on the same computing node first, that is, the execution time level of the tasks processed in the same computing module in a computing node is usually the same . In this way, when a task with a short execution time is completed, it is easier to obtain a whole free computing module, thereby reducing resource fragmentation in the computing node and improving resource utilization.
  • the receiving unit 801 is further configured to receive a scheduling request for a task, and the scheduling request includes a target execution duration level;
  • the processing unit is specifically used to parse the scheduling request to obtain the target execution time level.
  • the scheduling request when submitting a task, can carry indication information for indicating the execution duration level, so that the scheduling node can schedule the task according to the target execution duration indicated by the indication information.
  • the processing unit 802 is specifically configured to determine the execution duration level corresponding to the execution duration required by the task, and the determined execution duration level is the target execution duration level.
  • the device 80 when the above-mentioned device 80 receives a task, it can determine the execution time required for the task according to the information of the task, and then can determine the target execution time level corresponding to the execution time. For example, the device 80 stores the execution time of a previously scheduled task. When the device 80 receives a task, it can estimate the execution time of the task according to the execution time of a similar task previously scheduled, and then obtain the target execution time level of the task. .
  • the target execution duration level is one of the three execution duration levels; the foregoing three execution duration levels include long execution duration, medium execution duration, and short execution duration;
  • the long execution time is greater than or equal to the first threshold
  • the execution time is less than the first threshold and greater than the second threshold
  • the short execution duration is less than or equal to the second threshold.
  • the sending unit 803 is specifically configured to send a task to the target computing node when there is a target computing node with a target computing module.
  • the target execution duration level may indicate that the target computing module of the target computing node among the multiple computing nodes is used to execute the task. Therefore, the above-mentioned apparatus 80 sends the task to the target computing node, so that the target in the target computing node
  • the execution time levels of tasks executed by the computing modules are the same. In this way, when tasks with a shorter execution time are completed, larger idle computing resources can be obtained, which reduces resource fragmentation in the computing modules.
  • the target calculation module is a calculation module marked with a first label, and the first label represents that other tasks of the target execution duration level are executed in the calculation module.
  • the calculation module may be marked with different tags, and different tags indicate that the calculation module currently processes tasks with different execution duration levels.
  • the above-mentioned device 80 performs scheduling, it can prioritize the scheduling of tasks to the computing modules that are processing tasks of the same execution duration level.
  • the calculation modules of different duration levels correspond to different tags. Can improve the flexibility of system resource allocation, thereby improving resource utilization.
  • the sending unit 803 is specifically configured to send tasks to other computing nodes when there is no target computing node with a target computing module.
  • the sending unit 803 is specifically configured to send a task to the first computing node when there is no target computing node with the target computing module, where there is an unprocessed task in the first node Calculation module.
  • the sending unit 803 is further configured to send a task to a second computing node if there is no target computing node with a target computing module, and there is no first computing node, where the The second computing node has a computing module marked with a second label, where the second label represents that other tasks of the second execution duration level are being processed in the computing module, and the execution time required for the tasks of the second execution duration level is greater than The execution time required for this task is short and long.
  • the sending unit 803 is further configured to send a task to a third computing node if there is no target computing node with a target computing module, and there is no first computing node, where the The second computing node has a computing module marked with a third label, where the third label represents that other tasks of the third execution duration level are being processed in the computing module, and the execution time required for the tasks of the third execution duration level is greater than The execution time required for the task is long.
  • the processing unit 802 is further configured to mark a first label for the label of the target computing module of the target computing node, and the first label represents the execution of a task with a target execution duration level in the computing module.
  • the above-mentioned device can modify the label of the calculation module so that the label of the calculation module corresponds to the execution time level of the task it is processing, which is convenient for subsequent scheduling of other tasks.
  • processing unit 802 is further configured to delete the tag of the target calculation module if the task execution ends and the target calculation module has no other tasks being processed.
  • the above-mentioned division of multiple units is only a logical division based on functions, and is not intended to limit the specific structure of the device 80.
  • some of the functional modules may be subdivided into more small functional modules, and some functional modules may also be combined into one functional module.
  • the device 80 is performing resource scheduling The general process performed in the process is the same.
  • the above-mentioned multiple units can also be transformed into a communication unit and a processing unit, and the communication unit is used to implement the functions of the receiving unit 801 and the sending unit 803.
  • the aforementioned multiple units can be changed into a resource management unit (Resource Manager) and a node management unit (NodeManager).
  • the aforementioned resource management unit is used to implement the functions of the receiving unit 801 and part of the processing unit 802.
  • the aforementioned node management unit is used for To realize part of the functions of the processing unit 802 and the functions of the sending unit 803.
  • each unit corresponds to its own program code (or program instruction), and when the program code corresponding to each of these units runs on the processor, the unit executes the corresponding process to realize the corresponding function.
  • each unit in the foregoing apparatus 80 may also correspond to the corresponding description of the method embodiment shown in FIG. 4.
  • the apparatus 80 may be the scheduling node in the embodiment shown in FIG. 4
  • FIG. 9 is a schematic structural diagram of a scheduling node 90 provided by an embodiment of the present application.
  • the scheduling node 90 may be a node with data processing capability, or may be a device in a node with data processing capability. For example, chips or integrated circuits.
  • the scheduling node 90 may include a processor 901 and a memory 902. Further optionally, it may also include a communication interface 903 and a bus 904, where the processor 901, the memory 902, and the communication interface 903 are connected through the bus 1604.
  • the processor 901 is a module that performs arithmetic operations and logical operations, and can be a central processing unit (CPU), a graphics processing unit (GPU), or a microprocessor (MPU) processing unit.
  • CPU central processing unit
  • GPU graphics processing unit
  • MPU microprocessor
  • the memory 902 is used to provide storage space, and the storage space can store data such as an operating system and computer instructions.
  • the memory 902 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or Portable read-only memory (compact disc read-only memory, CD-ROM).
  • the communication interface 903 is used to receive data sent from the outside and/or send data to the outside, and may be a wired link interface such as an Ethernet cable or a wireless link (Wi-Fi, Bluetooth, etc.) interface.
  • the communication interface 903 may also include a transmitter (such as a radio frequency transmitter, an antenna, etc.) or a receiver coupled with the interface.
  • the processor 901 in the scheduling node 90 executes computer instructions stored in the memory 902, and the scheduling node 90 executes the aforementioned resource scheduling method.
  • the processor 901 in the scheduling node 90 executes computer instructions stored in the memory 902, so that the scheduling node 90 performs the following operations:
  • the target execution time level is used to represent the time length, and the target execution time level indicates that the target computing module of the target computing node among the multiple computing nodes is used to execute the task;
  • the task is preferentially sent to the target computing node corresponding to the target execution duration level.
  • the scheduling node 90 establishes the corresponding relationship between the execution duration level and the computing node (or the computing module in the computing node). After receiving a task, it is scheduled to the execution duration level to which the task belongs.
  • tasks with the same execution duration level can be preferentially scheduled to be executed on the same computing node. That is to say, the execution duration levels of tasks processed in the same computing module in a computing node are usually the same. In this way, when a task with a short execution time is completed, it is easier to obtain a whole free computing module, thereby reducing resource fragmentation in the computing node and improving resource utilization.
  • the processor 901 is specifically configured to: receive a scheduling request for a task through the communication interface 903, where the scheduling request includes a target execution duration level;
  • the scheduling request can carry indication information for indicating the execution duration level, so that the scheduling node 90 can schedule the task according to the target execution duration indicated by the indication information.
  • the processor 901 is specifically configured to determine an execution duration level corresponding to the execution duration of the task, and the determined execution duration level is the target execution duration level.
  • the scheduling node 90 when the scheduling node 90 receives a task, it can determine the execution time required by the task according to the information of the task, and then can determine the target execution time level corresponding to the execution time. For example, the scheduling node 90 stores the execution time of the previously scheduled task. When the scheduling node 90 receives the task, it can estimate the execution time of the task according to the execution time of the similar task previously scheduled, and then obtain the target execution time level of the task. .
  • the target execution duration level is one of the three execution duration levels; the foregoing three execution duration levels include long execution duration, medium execution duration, and short execution duration;
  • the long execution time is greater than or equal to the first threshold
  • the execution time is less than the first threshold and greater than the second threshold
  • the short execution duration is less than or equal to the second threshold.
  • the processor 901 is specifically configured to: when there is a target computing node with a target computing module, send a task to the target computing node through the aforementioned communication interface.
  • the target execution duration level may indicate that the target computing module of the target computing node among multiple computing nodes is used to execute the task. Therefore, the scheduling node 90 sends the task to the target computing node, so that the target in the target computing node
  • the execution time levels of tasks executed by the computing modules are the same. In this way, when tasks with a shorter execution time are completed, larger idle computing resources can be obtained, which reduces resource fragmentation in the computing modules.
  • the target calculation module is a calculation module marked with a first label, and the first label represents that the calculation module executes other tasks with a target execution duration level.
  • the calculation module may be marked with different tags, and different tags indicate that the calculation module currently processes tasks with different execution duration levels.
  • the scheduling node 90 performs scheduling, it can prioritize the scheduling of tasks to the computing modules that are processing tasks of the same execution duration level. Compared with the method of directly instructing the target module by the target execution duration, the method of calculating modules with different tags corresponding to different duration levels is used. Can improve the flexibility of system resource allocation, thereby improving resource utilization.
  • the processor 901 is specifically configured to: when there is no target computing node with a target computing module, send tasks to other computing nodes of the multiple computing nodes through the communication interface 903.
  • the processor 901 is specifically configured to: when there is no target computing node with a target computing module, send a task to the first computing node through the communication interface 903, where the first section There are calculation modules for unprocessed tasks in the calculation points.
  • the processor 901 is further configured to: if there is no target computing node with a target computing module, and there is no first computing node, send to the second computing node through the communication interface 903.
  • Task where the second computing node has a computing module marked with a second label, where the second label represents that other tasks of the first execution duration level are being processed in the computing module, and the tasks of the first execution duration level are The execution time required is shorter than the execution time required for the task.
  • the processor 901 is further configured to: if there is no target computing node with a target computing module, and there is no first computing node, send to the third computing node through the communication interface 903.
  • Task where the second computing node has a computing module marked with a third label, where the third label represents the task of the second execution duration level being processed in the computing module, and the task of the second execution duration level requires The execution time of is longer than the execution time required for the task.
  • the processor 901 is further configured to mark a first label for the label of the target computing module of the target computing node, and the first label represents the execution of a task with a target execution duration level in the computing module.
  • the scheduling node 90 can modify the label of the computing module so that the label of the computing module corresponds to the execution duration level of the task it is processing, which facilitates subsequent scheduling of other tasks.
  • the processor 901 is further configured to delete the first tag of the target calculation module if the task execution ends and the target calculation module has no other tasks being processed.
  • each unit may also correspond to the corresponding description of the method embodiment shown in FIG. 4, and the device 90 may be the scheduling node in the embodiment shown in FIG. 4.
  • An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the above-mentioned resource scheduling method, such as the one shown in FIG. 4 The resource scheduling method of the embodiment.
  • the embodiment of the present application also provides a chip system, the chip system includes at least one processor, a memory, and an interface circuit.
  • the interface circuit is used to provide information input/output for at least one processor, and the at least one memory stores Computer instructions, when the computer instructions run on one or more processors, the chip system executes the above-mentioned resource scheduling method, such as the resource scheduling method of the embodiment shown in FIG. 4.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions can be stored in a computer-readable storage medium or transmitted through a computer-readable storage medium.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).
  • the modules in the device embodiments of the present application can be combined, divided, and deleted according to actual needs.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mobile Radio Communication Systems (AREA)
  • Debugging And Monitoring (AREA)

Abstract

本申请实施例提供一种资源调度方法及相关装置，该方法包括：调度节点接收任务；调度节点获取任务所属的目标执行时长级别，其中，该目标执行时长级别用于表征时间长度，该目标执行时长级别指示使用多个计算节点中的目标计算节点的目标计算模块来执行所述任务；调度节点优先向所述目标执行时长级别对应的所述目标计算节点发送所述任务。通过本申请实施例，可以使得执行时长级别相同的任务可以优先调度到同一个计算节点上执行，当执行时长较短的任务在运行结束后，更易于得到整块空闲的计算模块，从而减少计算节点中的资源碎片，提高了资源的利用率。

Description

一种资源调度方法及相关装置 技术领域
本申请涉及大数据技术领域，尤其涉及一种资源调度方法及相关装置。
背景技术
随着大数据与互联网时代来临，大数据技术目前已成为学术界和产业界的研究热点，例如，人工智能(artificial intelligence，AI)、云计算等技术都需要依赖大数据技术来发展和进化。而大数据技术中使用的处理资源大部分是分布式资源，使得用于管理分布式资源的资源调度系统不论是在科研领域还是企业中都得到了广泛的应用。参见图1，图1是一种资源调度系统的架构示意图，计算节点中103中存在有至少一个计算模块，调度节点102接收客户端101提交的任务，根据任务所需的计算资源，将任务分配到计算节点103中的计算模块上进行执行。例如，由Apache推出的开源大数据处理平台Hadoop，通过资源调度系统Yarn来统一负责系统的资源分配以及任务调度。Yarn能够让多种计算框架(编程框架MapReduce、内存计算框架Spark、流式计算框架Strom和图计算框架等)运行在一个系统中，为不同的并行化计算提供统一的资源分配服务，使得系统具有易于运维、资源弹性可伸缩和数据可共享等优点。可以看出，资源调度系统是大数据平台中最核心的组件之一，资源调度系统的调度策略会直接影响分布式资源的任务分配，从而影响分布式资源系统的整体性能。
目前的资源调度系统是根据任务对处理资源的需求量来决策将任务调度到哪个计算节点的哪个计算模块上进行处理，例如：Yarn根据客户端提交的任务所需求的算力或显存来调度任务。但是这样的调度策略没有考虑到用户提交的任务的区别，使得计算节点中存在大量资源碎片，导致处理资源无法被充分利用。例如，参见图2，图1是一种资源调度系统的运行场景示意图，客户端201提交的一些任务所需的执行时长较短，例如任务1、任务3、任务4等等，占用计算资源的时间非常少，而一些任务所需的执行时长较长，例如任务2等等，占用计算资源的时间通常比较长，如果此时调度系统202为将任务1、任务2、任务3分配到同一计算节点(即计算节点1)上的一个计算模块(即计算模块1)上进行处理时，当短执行时长(如任务1、任务2)运行结束释放后，计算节点中剩余的碎片资源无法让其它任务使用，是一种严重的资源浪费，会极大程度上地降低资源的利用率，从而直接影响任务的执行效率，最终导致系统整体性能下降。
因此，如何减少计算节点中的资源碎片是本领域技术人员正在研究的问题。
发明内容
本申请实施例公开了一种资源调度方法及相关装置，能够减少计算节点中的资源碎片，提高资源利用率。
第一方面，本申请实施例公开了一种资源调度方法，包括：
调度节点接收任务；
该调度节点获取该任务所属的目标执行时长级别，其中，该目标执行时长级别用于 表征时间长度，该目标执行时长级别指示使用多个计算节点中的目标计算节点的目标计算模块来执行该任务；
该调度节点优先向目标执行时长级别对应的目标计算节点发送该任务。
本申请实施例中，调度节点建立了执行时长级别与计算节点(或者计算节点中的计算模块)的对应关系。当接收到一个任务之后，将其调度到该任务所属的执行时长级别对应的计算节点上，执行时长级别相同的任务就可以优先调度到同一个计算节点上执行，也即是说，一个计算节点中的同一个计算模块中处理的任务所属的执行时长级别通常是相同的。这样一来，当执行时长较短的任务在运行结束后，更易于得到整块空闲的计算模块，从而减少计算节点中的资源碎片，提高了资源的利用率。
在第一方面的一种可能的实施方式中，该调度节点获取该任务所属的目标执行时长级别之前，还包括：
该调度节点接收针对该任务的调度请求，该调度请求中包括目标执行时长级别；
该调度节点获取该任务所属的目标执行时长级别，包括：
该调度节点解析上述调度请求，得到该目标执行时长级别。
可以看出，在提交任务时，可以在调度请求中携带用于指示执行时长级别的指示信息，便于调度节点根据该指示信息指示的目标执行时长来调度任务。
在第一方面的又一种可能的实施方式中，该调度节点获取该任务所属的目标执行时长级别，包括：
调度节点确定与所述任务的执行时长对应的执行时长级别，确定的执行时长级别为所述目标执行时长级别。
可以看出，调度节点接收任务时，可以根据任务的信息确定任务所需的执行时长，进而可以确定该执行时长对应的目标执行时长级别。例如，调度节点中存储有之前调度的任务的执行时长，当调度节点接收该任务时，可以根据之前调度的相似的任务的执行时长，估计该任务的执行时长，进而得到任务的目标执行时长级别。
在第一方面的又一种可能的实施方式中，所述目标执行时长级别为三种执行时长级别中的一种执行时长级别；上述三种执行时长级别包括长执行时长、中执行时长和短执行时长；
其中，该长执行时长大于或者等于第一阈值；
该中执行时长小于第一阈值且大于第二阈值；
该短执行时长小于或者等于第二阈值。
在第一方面的又一种可能的实施方式中，该调度节点优先向目标执行时长级别对应的目标计算节点发送该任务，包括：
在存在具有目标计算模块的目标计算节点时，向目标计算节点发送该任务。
本申请实施例中，目标执行时长级别可以指示使用多个计算节点中的目标计算节点的目标计算模块来执行任务，因此调度节点向该目标计算节点发送该任务，从而使得目标计算节点中的目标计算模块执行的任务的执行时长级别是相同的，这样一来，当执行时间较短的任务运行结束后，可以得到较大的空闲计算资源，减少了计算模块中的资源碎片。
在第一方面的又一种可能的实施方式中，该目标计算模块为标记有第一标签的计算 模块，该第一标签表征计算模块中执行有该目标执行时长级别的其它任务。
本申请实施例中，计算模块中可以标记有不同的标签，不同的标签表征计算模块当前处理有不同执行时长级别的任务。调度节点进行调度时，可以优先将任务调度到正在处理同一执行时长级别的任务的计算模块中，相比直接通过目标执行时长指示目标模块，通过不同时长级别对应不同标签的计算模块的方法，可以提高系统资源分配的灵活性，从而提高了资源利用率。
在第一方面的又一种可能的实施方式中，该调度节点优先向目标执行时长级别对应的目标计算节点发送该任务，包括：
在不存在具有目标计算模块的目标计算节点时，向上述多个计算节点中的其它计算节点发送该任务。
在第一方面的又一种可能的实施方式中，在不存在具有目标计算模块的目标计算节点时，向上述多个计算节点中的其它计算节点发送该任务，包括：
在不存在具有目标计算模块的目标计算节点时，向第一计算节点发送该任务，其中，该第一节计算点中存在未处理任务的计算模块。
在第一方面的又一种可能的实施方式中，该方法还包括：
若不存在具有目标计算模块的目标计算节点，且不存在第一计算节点，则调度节点向第二计算节点发送该任务，其中，该第二计算节点存在有标记了第二标签的计算模块，其中，该第二标签表征计算模块中正在处理第一执行时长级别的其它任务，该第一执行时长级别的任务所需的执行时长比该任务所需的执行时长时间短。
在第一方面的又一种可能的实施方式中，该方法还包括：
若不存在具有目标计算模块的目标计算节点，且不存在第一计算节点，则调度节点向第三计算节点发送该任务，其中，该第二计算节点存在有标记了第三标签的计算模块，其中，该第三标签表征计算模块中正在处理第二执行时长级别的其它任务，该第二执行时长级别的任务所需的执行时长比该任务所需的执行时长时间长。
在第一方面的又一种可能的实施方式中，该方法还包括：
调度节点为目标计算节点的目标计算模块的标签标记第一标签，该第一标签表征计算模块中执行有目标执行时长级别的任务。
可以看出，调度节点可以修改计算模块的标签，使得计算模块的标签与其正在处理的任务的执行时长级别相对应，便于后续进行其它任务的调度。
在第一方面的又一种可能的实施方式中，该方法还包括：
若任务执行结束，且目标计算模块没有正在处理的其它任务，则调度节点删除目标计算模块的标签。
第二方面，本申请实施例公开了一种资源调度装置，包括：
接收单元，用于接收任务；
处理单元，用于获取该任务所属的目标执行时长级别，其中，该目标执行时长级别用于表征时间长度，该目标执行时长级别指示使用多个计算节点中的目标计算节点的目标计算模块来执行该任务；
发送单元，用于优先向目标执行时长级别对应的目标计算节点发送该任务。
本申请实施例中，该资源调度装置建立了执行时长级别与计算节点(或者计算节点中的计算模块)的对应关系。当接收到一个任务之后，将其调度到该任务所属的执行时长级别对应的计算节点上，执行时长级别相同的任务就可以优先调度到同一个计算节点上执行，也即是说，一个计算节点中的同一个计算模块中处理的任务所属的执行时长级别通常是相同的。这样一来，当执行时长较短的任务在运行结束后，更易于得到整块空闲的计算模块，从而减少计算节点中的资源碎片，提高了资源的利用率。
在第二方面的一种可能的实施方式中，该接收单元，还用于接收针对该任务的调度请求，该调度请求中包括目标执行时长级别；
该处理单元，具体用于解析调度请求，得到目标执行时长级别。
可以看出，在提交任务时，可以在调度请求中携带用于指示执行时长级别的指示信息，便于调度节点根据该指示信息指示的目标执行时长来调度任务。
在第二方面的又一种可能的实施方式中，该处理单元，具体用于确定与该任务所需的执行时长对应的执行时长级别，确定的执行时长级别为该目标执行时长级别。
可以看出，上述装置接收任务时，可以根据任务的信息确定任务所需的执行时长，进而可以确定该执行时长对应的目标执行时长级别。例如，上述装置中存储有之前调度的任务的执行时长，当上述装置接收任务时，可以根据之前调度的相似的任务的执行时长，估计任务的执行时长，进而得到任务的目标执行时长级别。
在第二方面的又一种可能的实施方式中，所述目标执行时长级别为三种执行时长级别中的一种执行时长级别；上述三种执行时长级别包括长执行时长、中执行时长和短执行时长；
其中，该长执行时长大于或者等于第一阈值；
该中执行时长小于第一阈值且大于第二阈值；
该短执行时长小于或者等于第二阈值。
在第二方面的又一种可能的实施方式中，该发送单元，具体用于在存在具有目标计算模块的目标计算节点时，向目标计算节点发送该任务。
本申请实施例中，目标执行时长级别可以指示使用多个计算节点中的目标计算节点的目标计算模块来执行任务，因此上述装置向该目标计算节点发送该任务，从而使得目标计算节点中的目标计算模块执行的任务的执行时长级别是相同的，这样一来，当执行时间较短的任务运行结束后，可以得到较大的空闲计算资源，减少了计算模块中的资源碎片。
在第二方面的又一种可能的实施方式中，该目标计算模块为标记有第一标签的计算模块，该第一标签表征计算模块中执行有该目标执行时长级别的其它任务。
本申请实施例中，计算模块中可以标记有不同的标签，不同的标签表征计算模块当前处理有不同执行时长级别的任务。上述装置进行调度时，可以优先将任务调度到正在处理同一执行时长级别的任务的计算模块中，相比直接通过目标执行时长指示目标模块，通过不同时长级别对应不同标签的计算模块的方法，可以提高系统资源分配的灵活性，从而提高了资源利用率。
在第二方面的又一种可能的实施方式中，该发送单元，具体用于在不存在具有目标计算模块的目标计算节点时，向其它计算节点发送该任务。
在第二方面的又一种可能的实施方式中，该发送单元，具体用于在不存在具有目标计算模块的目标计算节点时，向第一计算节点发送该任务，其中，该第一节点中存在未处理任务的计算模块。
在第二方面的又一种可能的实施方式中，该发送单元，还用于若不存在具有目标计算模块的目标计算节点，且不存在上述第一计算节点，则向第二计算节点发送该任务，其中，该第二计算节点存在有标记了第二标签的计算模块，其中，该第二标签表征计算模块中正在处理第二执行时长级别的其它任务，该第二执行时长级别的任务所需的执行时长比该任务所需的执行时长时间短。
在第二方面的又一种可能的实施方式中，该发送单元，还用于若不存在具有目标计算模块的目标计算节点，且不存在上述第一计算节点，则向第三计算节点发送该任务，其中，该第二计算节点存在有标记了第三标签的计算模块，其中，该第三标签表征计算模块中正在处理第三执行时长级别的其它任务，该第三执行时长级别的任务所需的执行时长比该任务所需的执行时长时间长。
在第二方面的又一种可能的实施方式中，该处理单元，还用于为目标计算节点的目标计算模块的标签标记第一标签，该第一标签表征计算模块中执行有目标执行时长级别的任务。
可以看出，上述装置可以修改计算模块的标签，使得计算模块的标签与其正在处理的任务的执行时长级别相对应，便于后续进行其它任务的调度。
在第二方面的又一种可能的实施方式中，该处理单元，还用于若任务执行结束，且目标计算模块没有正在处理的其它任务，则删除目标计算模块的第一标签。
第三方面，本申请实施例公开了一种调度节点，包括处理器和存储器；该处理器用于执行存储器存储的计算机指令，使得调度节点实现第一方面或者第一方面的任意一种可能的实施方式所描述的方法。
在第三方面的一种可能的实施方式中，该调度节点还包括通信接口，该处理器具体用于：
通过上述通信接口接收任务；
获取该任务所属的目标执行时长级别，其中，该目标执行时长级别用于表征时间长度，该目标执行时长级别指示使用多个计算节点中的目标计算节点的目标计算模块来执行该任务；
通过上述通信接口优先向目标执行时长级别对应的目标计算节点发送该任务。
本申请实施例中，调度节点建立了执行时长级别与计算节点(或者计算节点中的计算模块)的对应关系，当接收到一个任务之后，将其调度到该任务所属的执行时长级别对应的计算节点上，执行时长级别相同的任务就可以优先调度到同一个计算节点上执行，也即是说，一个计算节点中的同一个计算模块中处理的任务所属的执行时长级别通常是相同的。这样一来，当执行时长较短的任务在运行结束后，更易于得到整块空闲的计算模块，从而减少计算节点中的资源碎片，提高了资源的利用率。
在第三方面的又一种可能的实施方式中，该处理器具体用于：通过上述通信接口接收针对该任务的调度请求，该调度请求中包括目标执行时长级别；
解析上述调度请求，得到该目标执行时长级别。
可以看出，在提交任务时，可以在调度请求中携带用于指示执行时长级别的指示信息，便于调度节点根据该指示信息指示的目标执行时长来调度任务。
在第三方面的又一种可能的实施方式中，该处理器具体用于确定与该任务的执行时长对应的执行时长级别，确定的执行时长级别为所述目标执行时长级别。
可以看出，调度节点接收任务时，可以根据任务的信息确定任务所需的执行时长，进而可以确定该执行时长对应的目标执行时长级别。例如，调度节点中存储有之前调度的任务的执行时长，当调度节点接收任务时，可以根据之前调度的相似的任务的执行时长，估计任务的执行时长，进而得到任务的目标执行时长级别。
在第三方面的又一种可能的实施方式中，所述目标执行时长级别为三种执行时长级别中的一种执行时长级别；上述三种执行时长级别包括长执行时长、中执行时长和短执行时长；
其中，该长执行时长大于或者等于第一阈值；
该中执行时长小于第一阈值且大于第二阈值；
该短执行时长小于或者等于第二阈值。
在第三方面的又一种可能的实施方式中，该处理器具体用于：在存在具有目标计算模块的目标计算节点时，通过上述通信接口向目标计算节点发送该任务。
本申请实施例中，目标执行时长级别可以指示使用多个计算节点中的目标计算节点的目标计算模块来执行任务，因此调度节点向该目标计算节点发送该任务，从而使得目标计算节点中的目标计算模块执行的任务的执行时长级别是相同的，这样一来，当执行时间较短的任务运行结束后，可以得到较大的空闲计算资源，减少了计算模块中的资源碎片。
在第三方面的又一种可能的实施方式中，该目标计算模块为标记有第一标签的计算模块，该第一标签表征计算模块中执行有该目标执行时长级别的其它任务。
本申请实施例中，计算模块中可以标记有不同的标签，不同的标签表征计算模块当前处理有不同执行时长级别的任务。调度节点进行调度时，可以优先将任务调度到正在处理同一执行时长级别的任务的计算模块中，相比直接通过目标执行时长指示目标模块，通过不同时长级别对应不同标签的计算模块的方法，可以提高系统资源分配的灵活性，从而提高了资源利用率。
在第三方面的又一种可能的实施方式中，该处理器具体用于：在不存在具有目标计算模块的目标计算节点时，通过上述通信接口向上述多个计算节点中的其它计算节点发送该任务。
在第三方面的又一种可能的实施方式中，该处理器具体用于：在不存在具有目标计算模块的目标计算节点时，通过上述通信接口向第一计算节点发送该任务，其中，该第一节计算点中存在未处理任务的计算模块。
在第三方面的又一种可能的实施方式中，该处理器还用于：若不存在具有目标计算模块的目标计算节点，且不存在第一计算节点，则通过上述通信接口向第二计算节点发送该任务，其中，该第二计算节点存在有标记了第二标签的计算模块，其中，该第二标签表征计算模块中正在处理第一执行时长级别的其它任务，该第一执行时长级别的任务 所需的执行时长比该任务所需的执行时长时间短。
在第三方面的又一种可能的实施方式中，该处理器还用于：若不存在具有目标计算模块的目标计算节点，且不存在第一计算节点，则通过上述通信接口向第三计算节点发送该任务，其中，该第二计算节点存在有标记了第三标签的计算模块，其中，该第三标签表征计算模块中正在处理第二执行时长级别的其它任务，该第二执行时长级别的任务所需的执行时长比该任务所需的执行时长时间长。
在第三方面的又一种可能的实施方式中，该处理器还用于：为目标计算节点的目标计算模块的标签标记第一标签，该第一标签表征计算模块中执行有目标执行时长级别的任务。
可以看出，调度节点可以修改计算模块的标签，使得计算模块的标签与其正在处理的任务的执行时长级别相对应，便于后续进行其它任务的调度。
在第三方面的又一种可能的实施方式中，该处理器还用于：若该任务执行结束，且目标计算模块没有正在处理的其它任务，则删除目标计算模块的第一标签。
第四方面，本申请实施例公开了一种计算机可读存储介质，其特征在于，该计算机可读存储介质中存储有计算机指令，所述计算机指令用于实现第一方面或者第一方面的任意一种可能的实施方式所描述的方法。
第五方面，本申请实施例公开了一种芯片系统，该芯片系统包括至少一个处理器，存储器和接口电路，该接口电路用于为上述至少一个处理器提供信息输入/输出，该存储器中存储有计算机指令，当计算机指令在一个或多个处理器上运行时，执行第一方面或者第一方面的任意一种可能的实施方式所描述的方法。
第六方面，本申请实施例公开了一种资源调度系统，该资源调度系统包括资源调度装置，至少一个客户端和至少一个计算节点。其中，该资源调度装置用于接收来自该至少一个客户端的任务，并将该任务发送到至少一个计算节点上进行执行。该资源调度装置为第二方面或者第二方面任意一种可能的实施方式所描述的资源调度装置。
附图说明
以下对本申请实施例用到的附图进行介绍。
图1是本申请实施例提供的一种资源调度系统的架构示意图；
图2是本申请实施例提供的一种资源调度的运行场景示意图；
图3是本申请实施例提供的又一种资源调度系统的架构示意图；
图4是本申请实施例提供的一种资源调度方法的流程示意图；
图5是本申请实施例提供的一种调度请求的示意图；
图6是本申请实施例提供的又一种资源调度的运行场景示意图；
图7是本申请实施例提供的又一种资源调度系统的架构示意图；
图8是本申请实施例提供的一种装置的结构示意图；
图9是本申请实施例提供的一种调度节点的结构示意图。
具体实施方式
下面结合本申请实施例中的附图对本申请实施例进行描述。需要说明的是，本申请 中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。
本申请描述的系统架构及业务场景是为了更加清楚的说明本申请的技术方案，并不构成对于本申请提供的技术方案的限定，本领域普通技术人员可知，随着系统架构的演变和新业务场景的出现，本申请提供的技术方案对于类似的技术问题，同样适用。
参见图3，是本申请实施例提供的一种资源调度系统的架构示意图，包括客户端301、调度节点302和至少一个计算节点(compute node)303。其中：
客户端301是带有数据收发能力的电子设备，可以是实体设备如主机、服务器等，也可以是虚拟设备如虚拟机、容器等。客户端301用于提交任务，该任务需要通过计算节点302中的计算模块来执行。通常来说，客户端提交的任务中可以携带有执行该任务的程序包(或者算法对应的镜像文件等)，并说明该任务所需要的算力(例如需要的中央处理器(centralprocessingunit，CPU)资源数、图形处理器(Graphics Processing Unit，GPU)资源数、内存大小等等中的一个或者多个)，计算节点将任务调度到计算节点中进行计算，并将结果返回给客户端。例如，客户端可以提交人脸识别任务，携带用于进行人脸识别的程序包或者人脸识别算法的镜像文件和对应的数据库文件等，并说明该人脸识别任务所需要的算力(例如需要8核CPU、30％的GPU资源、8G内存)，计算节点计算后，将人脸识别的结果返回给客户端。
计算节点302是用于提供计算服务的设备，可以是实体设备如主机、服务器等，也可以是虚拟设备如虚拟机、容器等。计算节点301中包括一个或者多个计算模块，用于提供计算资源，该计算模块可以包括图形处理器(Graphics Processing Unit，GPU)、微处理器(MicroProcessing Unit，MPU)、声卡(Sound Card)、用于人工智能计算的加速卡等等中的一个或者多个。进一步的，该计算模块还可以包括CPU、内存等等计算资源。
调度节点303是部署有调度系统的一个或者多个节点，该节点可以是物理设备如主机、服务器等，也可以是虚拟机、容器等虚拟设备。调度节点302可以接收来自客户端301的任务，并按照调度策略将任务调度到计算节点302中的计算模块中进行执行。
需要说明的是，该客户端301、计算节点302和调度节点303可以部署在不同的物理设备中，也可以部署在同一物理设备中，本申请对此不做限定。
请参见图4，图4是本申请实施例提供的一种资源调度方法，该方法可以基于图3所示的资源调度系统来实现，该方法包括但不限于如下步骤：
步骤S401：调度节点接收任务。
具体的，该任务为客户端提交的任务。该任务可以为人脸识别任务、视频解码任务、深度学习的模型训练任务等等。通常来说，该任务可以携带有对应的程序包(或者算法对应的镜像文件等)。进一步的，还可以携带该任务所需要的算力(例如需要的CPU资源数、GPU资源数、内存大小等等中的一个或者多个)。
步骤S402：调度节点获取任务所属的目标执行时长级别。
具体地，客户端提交的任务有多种执行时长级别，而目标执行时长级别可以用于表 征时间长度，和/或可以指示使用目标计算节点的目标计算模块来执行任务。例如，本申请举例提供三种执行时长级别：长执行时长、中执行时长和短执行时长，其中，长执行时长大于或者等于第一阈值，中执行时长小于第一阈值且大于第二阈值，短执行时长小于或者等于第二阈值。
下面例举两种获取任务的执行时长级别的方法：
方法1：调度节点解析针对任务的调度请求，得到任务的执行时长级别。具体的，调度节点接收针对任务的调度请求，该调度请求中包括任务的执行时长级别。例如，客户端提交任务时，在提交的任务中携带用于指示执行时长级别的指示信息，便于调度节点根据该指示信息指示的目标执行时长调度任务。
参见图5，图5是本申请实施例提供的一种可能的调度请求的示意图，其中，区域501表示调度节点接收到的一种可能的调度请求，包括调度请求的名称(name)、工作编号(jobidentification，jobID)、内存(memory)、优先级(priority)、执行时长级别(taskcycle)等等。调度节点解析区域501中所示的调度请求，可以得到如区域502中所示的执行时长级别，即目标执行时长级别。可选的，调度请求中还可以携带有该任务的CPU资源数、GPU资源数(图5中未示出)等等中的一个或者多个。
方法2：调度节点确定任务所需的执行时长对应的目标执行时长级别。具体的，调度节点接收任务时，根据任务的信息确定任务所需的执行时长，进而可以确定该执行时长对应的目标执行时长级别。
例如，调度节点中存储有之前调度的任务的执行时长信息，当调度节点接收任务时，可以根据之前调度的相似的任务的执行时长，估计任务的执行时长，进而得到任务的目标执行时长级别。进一步的，在三种执行时长级别分别为长执行时长、中执行时长和短执行时长的情况下，若调度节点估计任务所需的执行时长大于或者等于第一阈值，则该任务所属的执行时长级别为长执行时长，例如，以第一阈值为24小时为例，若调度节点估计任务所需的执行时长大于或者等于24小时，则可以确定该任务的执行时长级别为长执行时长。同理，若估计任务所需的执行时长小于第一阈值且大于第二阈值，则为中执行时长；若估计任务所需的执行时长小于或者等于第一二阈值，则为短执行时长。
再如，客户端提交任务时可以携带该任务所需的算法镜像编号(imageID)，相应的，调度节点根据算法的镜像ID，可以估计任务的执行时长，进而得到任务的目标执行时长级别。例如，三种执行时长级别分别为长执行时长、中执行时长和短执行时长的情况下，以第二阈值为10分钟为例，若该任务携带的算法镜像编号指示为进行人像搜索算法，由于人像搜索算法通常处理时间通常小于10分钟，因此可以将该任务的执行时长级别确定为短执行时长。
步骤S403：调度节点优先向目标执行时长级别对应的目标计算节点发送任务。
具体地，本申请实施例建立了执行级别时长与计算节点(或者计算节点中的计算模块)的对应关系，当接收到一个任务之后，将其调度到该任务所属的执行时长级别对应的计算节点上，这样一来，对应的执行时长级别相同的任务就可以优先调度到同一个计算节点上执行，也即是说，一个计算节点中的同一个计算模块中处理的任务所属的执行时长级别通常是相同的。这样一来，当执行时间较短的任务在运行结束后，更易于得到整块空闲的计算模块，从而减少计算节点中的资源碎片，提高了资源的利用率。
例如，参见图6，图6是本申请实施例提供的一种可能的资源调度的运行场景示意图，客户端301提交的两个任务所属的执行时长级别分别为长执行时长和短执行时长。其中，短执行时长对应计算节点1(进一步的，还可以具体对应为计算节点1中的计算模块1和计算模块2)，长执行时长对应计算节点2(进一步的，还可以具体对应为计算节点2中的计算模块3)。也即是说，短执行时长指示使用计算节点1中的计算模块1或者计算模块2来执行该任务，因此调度节点优先将短执行时长的任务发送到计算节点1中执行。这样一来，当短执行时长的任务运行结束后，计算节点1中可以存在整块的计算模块用于执行其它任务，较少了资源碎片现象，提高了资源利用率。
可选的，通过相应的机制保证每个执行时长级别各自对应有计算节点，这种情况下，调度节点直接向目标执行时长级别对应的目标计算节点发送任务。
可选的，在某个特定的时刻或者时间段，目标行时长级别有对应的目标计算节点，但是随着目标计算节点上的任务的释放或者其它操作的执行，在后续的某个时刻或者时间段，目标行时长级别又没有对应的目标计算节点。这种情况下，调度节点优先向目标执行时长级别对应的目标计算节点发送任务可以具体为：
在存在具有目标计算模块的目标计算节点时，向目标计算节点发送任务。例如，任务的执行时长级别为短执行时长时，指示使用多个计算节点中的计算节点1的计算模块来执行任务，若存在该计算节点1具有该计算模块，则将任务调度到计算节点1的计算模块上进行执行。
在一种可能的方案中，计算模块中可以标记有不同的标签，不同的标签表征当前处理有不同执行时长级别的任务，其中，目标计算模块为标记有第一标签的计算模块，该第一标签表征计算模块中执行有目标执行时长级别的任务。调度节点进行调度时，可以优先将任务调度到正在处理同一执行时长级别的任务的计算模块中，也即是说，任务所属的目标执行时长级别可以指示使用标记有第一标签的计算模块的计算节点来执行任务。
参见图7，图7是本申请实施例提供的又一种可能的资源调度的运行场景示意图，客户端301提交的任务所属的执行时长级别可以是中执行时长，短执行时长或者长执行时长，中执行时长，短执行时长和长执行时长分别对应为标签为“标签A”“标签B”“标签C”的计算模块。其中，计算节点701中包括计算模块702和计算模块703，其中，计算模块702中正在执行任务1，任务1所属的执行时长级别为“中执行时长”，因此计算模块的标签为“标签A”；计算模块703中没有正在处理的任务，标签为空。而计算节点704中包括计算模块705和计算模块706，其中，计算模块705中正在执行任务2，任务2所属的执行时长级别为“短执行时长”，因此计算模块的标签为“标签B”；计算模块706中正在执行任务3，任务3所属的执行时长级别为“长执行时长”，因此计算模块的标签为“标签C”。当客户端提交的任务的执行时长级别为中执行时长时，若存在标签为“标签A”的计算模块，则将任务发送到该“标签A”的计算模块(即计算模块702)所属的计算节点(即计算节点701)上进行执行。
在不存在具有目标计算模块的目标计算节点时或者目标计算模块无法执行任务时，向其它计算节点发送任务。例如，当任务的执行时长级别为短执行时长时，指示使用多个计算节点中的计算节点1的计算模块来执行任务，若不存在该计算节点1具有该计算 模块或者该计算模块已经无法执行该任务，则将任务调度到其它的计算节点中执行。进一步的，可以有如下可选案例：
案例1：在不存在具有目标计算模块的目标计算节点时或者目标计算模块无法执行任务时，向存在未处理任务的计算模块的计算节点发送任务。其中，未处理任务的计算模块可以看作空闲的计算模块，为了方便描述，本申请各实施例中，将存在有空闲的计算模块的计算节点称为第一计算节点。可以看出，当不存在具有目标计算模块的目标计算节点时或者目标计算模块无法执行任务时，可以将任务调度到空闲的计算模块上进行处理。
案例2：计算模块中可以标记有不同的标签，目标计算模块为标记有第一标签的计算模块，该第一标签表征计算模块中执行有目标执行时长级别的任务。而目标执行时长级别指示使用标记有第一标签的计算模块所属的计算节点来执行任务，若不存在标记第一标签的计算模块，则向没有标签的计算模块所属的计算节点发送该任务。例如，参见图7，当客户端提交的任务所属的执行时长级别为中执行时长时，若不存在标签为“标签A”的计算模块，则将任务发送到没有标签计算模块(即计算模块703)所属的计算节点(即计算节点701)上进行执行。
案例3：若不存在具有目标计算模块的目标计算节点，且不存在第一计算节点，则向标记有第二标签的计算模块所属的计算节点发送该任务，其中，第二标签表征计算模块中正在处理第一执行时长级别的任务，该第一执行时长级别的任务所需的执行时长比任务所需的执行时长时间短。为了方便描述，本申请各实施例中，将标记有第二标签的计算模块所属的计算节点称为第二计算节点。例如，参见图7，当客户端提交的任务所属的执行时长级别为中执行时长时，若不存在标签为“标签A”的计算模块，也不存在没有标签计算模块，由于短执行时长的执行时长比中执行时长的执行时长更短，而“标签B”表示计算模块正在处理短执行时长的任务，因此可以将客户端提交的任务发送到标记有“标签B”的计算模块(即计算模块705)所属的计算节点(即计算节点704)上进行执行。
案例4：若不存在具有目标计算模块的目标计算节点，且不存在第一计算节点，则向标记有第三标签的计算模块所属的计算节点发送该任务，其中，第三标签表征计算模块中正在处理第二执行时长级别的任务，该第二执行时长级别的任务所需的执行时长比任务所需的执行时长时间长。为了方便描述，本申请各实施例中，将标记有第三标签的计算模块所属的计算节点称为第三计算节点。例如，参见图7，当客户端提交的任务所属的执行时长级别为中执行时长时，若不存在标签为“标签A”的计算模块，也不存在没有标签计算模块，由于长执行时长的执行时长比中执行时长的执行时长更长，而“标签C”表示计算模块正在处理短执行时长的任务，因此可以将客户端提交的任务发送到标记有“标签C”的计算模块(即计算模块706)所属的计算节点(即计算节点704)上进行执行。
可选的，调度节点可以修改计算模块的标签，使得计算模块的标签与其正在处理的任务的执行时长级别相对应，便于后续进行任务调度。例如，案例2中，客户端提交的任务所属的执行时长级别为短执行时长，调度节点向计算节点701发送任务后，由计算模块703执行该任务，因此调度节点可以将计算模块703的标签修改为“标签A”。这 样一来，当后续接收到执行时长级别为中执行时长的任务时，可以优先调度给计算模块703进行执行。
可选的，计算模块的标签与其正在处理的任务的执行时长级别相对应，可以具体为计算模块的标签与正在处理的任务中的级别最高的执行时长级别进行对应，即与执行时长最长的任务进行对应。例如，案例3中，若客户端提交的任务所属的执行时长级别为短执行时长，调度节点向计算节点704发送任务后，由计算模块705执行该任务，由于计算模块705原本的标签“标签B”对应于之前处理的短执行时长，而客户端提交的任务所属的执行时长级别比短执行时长更长，因此调度节点可以将计算模块705的标签修改为“标签A”。
可选的，当任务执行结束时，计算节点可以删除计算模块的标签。例如，案例2中，任务所属的执行时长级别为短执行时长，调度节点向计算节点701发送任务后，将计算模块703的标签修改为“标签A”。当任务执行结束后，该计算模块703中没有其它正在执行的任务，因此调度节点可以将计算模块703的标签删除，表明计算模块703为空闲的计算计算模块。
可以看出，在图4所示的方法中，调度节点建立了执行时长级别与计算节点(或者计算节点中的计算模块)的对应关系。当接收到一个任务之后，将其调度到该任务所属的执行时长级别对应的计算节点上，执行时长级别相同的任务就可以优先调度到同一个计算节点上执行，也即是说，一个计算节点中的同一个计算模块中处理的任务所属的执行时长级别通常是相同的。这样一来，当执行时长较短的任务在运行结束后，更易于得到整块空闲的计算模块，从而减少计算节点中的资源碎片，提高了资源的利用率。
上述详细阐述了本申请实施例的方法，下面提供了本申请实施例的装置。
请参见图8，图8是本申请实施例提供的一种装置80的结构示意图，该装置80可以包括接收单元801、处理单元802和发送单元803，其中，各个单元的描述如下：
接收单元801，用于接收任务；
处理单元802，用于获取任务所属的目标执行时长级别，其中，该目标执行时长级别用于表征时间长度，该目标执行时长级别指示使用多个计算节点中的目标计算节点的目标计算模块来执行任务；
发送单元803，用于优先向目标执行时长级别对应的目标计算节点发送任务。
本申请实施例中，该装置80建立了执行时长级别与计算节点(或者计算节点中的计算模块)的对应关系，当接收到一个任务)之后，将其调度到该任务所属的执行时长级别对应的计算节点上，执行时长级别相同的任务就可以优先调度到同一个计算节点上执行，也即是说，一个计算节点中的同一个计算模块中处理的任务所属的执行时长级别通常是相同的。这样一来，当执行时长较短的任务在运行结束后，更易于得到整块空闲的计算模块，从而减少计算节点中的资源碎片，提高了资源的利用率。
在一种可能的实施方式中，该接收单元801，还用于接收针对任务的调度请求，该调度请求中包括目标执行时长级别；
处理单元，具体用于解析调度请求，得到目标执行时长级别。
可以看出，在提交任务时，可以在调度请求中携带用于指示执行时长级别的指示信 息，便于调度节点根据该指示信息指示的目标执行时长来调度任务。
在又一种可能的实施方式中，该处理单元802，具体用于确定与任务所需的执行时长对应的执行时长级别，确定的执行时长级别为该目标执行时长级别。
可以看出，上述装置80接收任务时，可以根据任务的信息确定任务所需的执行时长，进而可以确定该执行时长对应的目标执行时长级别。例如，上述装置80中存储有之前调度的任务的执行时长，当上述装置80接收任务时，可以根据之前调度的相似的任务的执行时长，估计任务的执行时长，进而得到任务的目标执行时长级别。
在又一种可能的实施方式中，所述目标执行时长级别为三种执行时长级别中的一种执行时长级别；上述三种执行时长级别包括长执行时长、中执行时长和短执行时长；
其中，该长执行时长大于或者等于第一阈值；
该中执行时长小于第一阈值且大于第二阈值；
该短执行时长小于或者等于第二阈值。
在又一种可能的实施方式中，该发送单元803，具体用于在存在具有目标计算模块的目标计算节点时，向目标计算节点发送任务。
本申请实施例中，目标执行时长级别可以指示使用多个计算节点中的目标计算节点的目标计算模块来执行任务，因此上述装置80向该目标计算节点发送任务，从而使得目标计算节点中的目标计算模块执行的任务的执行时长级别是相同的，这样一来，当执行时间较短的任务运行结束后，可以得到较大的空闲计算资源，减少了计算模块中的资源碎片。
在又一种可能的实施方式中，该目标计算模块为标记有第一标签的计算模块，该第一标签表征计算模块中执行有该目标执行时长级别的其它任务。
本申请实施例中，计算模块中可以标记有不同的标签，不同的标签表征计算模块当前处理有不同执行时长级别的任务。上述装置80进行调度时，可以优先将任务调度到正在处理同一执行时长级别的任务的计算模块中，相比直接通过目标执行时长指示目标模块，通过不同时长级别对应不同标签的计算模块的方法，可以提高系统资源分配的灵活性，从而提高了资源利用率。
在又一种可能的实施方式中，该发送单元803，具体用于在不存在具有目标计算模块的目标计算节点时，向其它计算节点发送任务。
在又一种可能的实施方式中，该发送单元803，具体用于在不存在具有目标计算模块的目标计算节点时，向第一计算节点发送任务，其中，该第一节点中存在未处理任务的计算模块。
在又一种可能的实施方式中，该发送单元803，还用于若不存在具有目标计算模块的目标计算节点，且不存在第一计算节点，则向第二计算节点发送任务，其中，该第二计算节点存在有标记了第二标签的计算模块，其中，该第二标签表征计算模块中正在处理第二执行时长级别的其它任务，该第二执行时长级别的任务所需的执行时长比该任务所需的执行时长时间短。
在又一种可能的实施方式中，该发送单元803，还用于若不存在具有目标计算模块的目标计算节点，且不存在第一计算节点，则向第三计算节点发送任务，其中，该第二计算节点存在有标记了第三标签的计算模块，其中，该第三标签表征计算模块中正在处理 第三执行时长级别的其它任务，该第三执行时长级别的任务所需的执行时长比任务所需的执行时长时间长。
在又一种可能的实施方式中，该处理单元802，还用于为目标计算节点的目标计算模块的标签标记第一标签，该第一标签表征计算模块中执行有目标执行时长级别的任务。
可以看出，上述装置可以修改计算模块的标签，使得计算模块的标签与其正在处理的任务的执行时长级别相对应，便于后续进行其它任务的调度。
在又一种可能的实施方式中，该处理单元802，还用于若任务执行结束，且目标计算模块没有正在处理的其它任务，则删除目标计算模块的标签。
这里需要说明的是，上述多个单元的划分仅是一种根据功能进行的逻辑划分，不作为对装置80具体的结构的限定。在具体实现中，其中部分功能模块可能被细分为更多细小的功能模块，部分功能模块也可能组合成一个功能模块，但无论这些功能模块是进行了细分还是组合，装置80在资源调度的过程中所执行的大致流程是相同的。
例如，上述多个单元也可以变换为通信单元以及处理单元，该通信单元用于实现接收单元801和发送单元803的功能。再如，上述多个单元可以变化为资源管理单元(Resource Manager)和节点管理单元(NodeManager)，上述资源管理单元用于实现接收单元801的功能和部分处理单元802的功能，上述节点管理单元用于实现部分处理单元802的功能和发送单元803的功能。通常，每个单元都对应有各自的程序代码(或者说程序指令)，这些单元各自对应的程序代码在处理器上运行时，使得该单元执行相应的流程从而实现相应功能。
上述装置80中各个单元的实现还可以对应参照图4所示的方法实施例的相应描述，该装置80可以为图4所示的实施例中的调度节点
请参见图9，图9是本申请实施例提供的一种调度节点90的结构示意图，该调度节点90可以为具有数据处理能力的节点，也可以为具有数据处理能力的节点中的一个器件，例如芯片或者集成电路等。该调度节点90可以包括处理器901和存储器902。进一步可选的，还可以包含通信接口903和总线904，其中，处理器901、存储器902和通信接口903通过总线1604相连。
其中，处理器901是进行算术运算和逻辑运算的模块，可以是中央处理器(central processing unit，CPU)、显卡处理器(graphics processing unit，GPU)或微处理器(microprocessor unit，MPU)等处理模块中的一种或者多种的组合。
存储器902用于提供存储空间，存储空间中可以存储操作系统和计算机指令等数据。存储器902包括但不限于是随机存储记忆体(random access memory，RAM)、只读存储器(read-only memory，ROM)、可擦除可编程只读存储器(erasable programmable read only memory，EPROM)、或便携式只读存储器(compact disc read-only memory，CD-ROM)。
通信接口903用于接收外部发送的数据和/或向外部发送数据，可以为包括诸如以太网电缆等的有线链路接口，也可以是无线链路(Wi-Fi、蓝牙等)接口。可选的，通信接口903还可以包括与接口耦合的发射器(如射频发射器、天线等)，或者接收器等。
该调度节点90中的处理器901执行存储器902中存储的计算机指令，该调度节点90执行上述的资源调度方法。
举例说明，该调度节点90中的处理器901执行存储器902中存储的计算机指令，使得该调度节点90执行以下操作：
通过上述通信接口903接收任务；
获取任务所属的目标执行时长级别，其中，该目标执行时长级别用于表征时间长度，该目标执行时长级别指示使用多个计算节点中的目标计算节点的目标计算模块来执行任务；
通过上述通信接口903优先向目标执行时长级别对应的目标计算节点发送任务。
本申请实施例中，调度节点90建立了执行时长级别与计算节点(或者计算节点中的计算模块)的对应关系，当接收到一个任务之后，将其调度到该任务所属的执行时长级别对应的计算节点上，执行时长级别相同的任务就可以优先调度到同一个计算节点上执行，也即是说，一个计算节点中的同一个计算模块中处理的任务所属的执行时长级别通常是相同的。这样一来，当执行时长较短的任务在运行结束后，更易于得到整块空闲的计算模块，从而减少计算节点中的资源碎片，提高了资源的利用率。
在一种可能的实施方式中，该处理器901具体用于：通过上述通信接口903接收针对任务的调度请求，该调度请求中包括目标执行时长级别；
解析上述调度请求，得到该目标执行时长级别。
可以看出，在提交任务时，可以在调度请求中携带用于指示执行时长级别的指示信息，便于调度节点90根据该指示信息指示的目标执行时长来调度任务。
在又一种可能的实施方式中，该处理器901具体用于确定与该任务的执行时长对应的执行时长级别，确定的执行时长级别为所述目标执行时长级别。
可以看出，调度节点90接收任务时，可以根据任务的信息确定任务所需的执行时长，进而可以确定该执行时长对应的目标执行时长级别。例如，调度节点90中存储有之前调度的任务的执行时长，当调度节点90接收任务时，可以根据之前调度的相似的任务的执行时长，估计任务的执行时长，进而得到任务的目标执行时长级别。
在又一种可能的实施方式中，所述目标执行时长级别为三种执行时长级别中的一种执行时长级别；上述三种执行时长级别包括长执行时长、中执行时长和短执行时长；
其中，该长执行时长大于或者等于第一阈值；
该中执行时长小于第一阈值且大于第二阈值；
该短执行时长小于或者等于第二阈值。
在又一种可能的实施方式中，该处理器901具体用于：在存在具有目标计算模块的目标计算节点时，通过上述通信接口向目标计算节点发送任务。
本申请实施例中，目标执行时长级别可以指示使用多个计算节点中的目标计算节点的目标计算模块来执行任务，因此调度节点90向该目标计算节点发送任务，从而使得目标计算节点中的目标计算模块执行的任务的执行时长级别是相同的，这样一来，当执行时间较短的任务运行结束后，可以得到较大的空闲计算资源，减少了计算模块中的资源碎片。
在又一种可能的实施方式中，该目标计算模块为标记有第一标签的计算模块，该第一标签表征计算模块中执行有目标执行时长级别的其它任务。
本申请实施例中，计算模块中可以标记有不同的标签，不同的标签表征计算模块当 前处理有不同执行时长级别的任务。调度节点90进行调度时，可以优先将任务调度到正在处理同一执行时长级别的任务的计算模块中，相比直接通过目标执行时长指示目标模块，通过不同时长级别对应不同标签的计算模块的方法，可以提高系统资源分配的灵活性，从而提高了资源利用率。
在又一种可能的实施方式中，该处理器901具体用于：在不存在具有目标计算模块的目标计算节点时，通过上述通信接口903向上述多个计算节点中的其它计算节点发送任务。
在又一种可能的实施方式中，该处理器901具体用于：在不存在具有目标计算模块的目标计算节点时，通过上述通信接口903向第一计算节点发送任务，其中，该第一节计算点中存在未处理任务的计算模块。
在又一种可能的实施方式中，该处理器901还用于：若不存在具有目标计算模块的目标计算节点，且不存在第一计算节点，则通过上述通信接口903向第二计算节点发送任务，其中，该第二计算节点存在有标记了第二标签的计算模块，其中，该第二标签表征计算模块中正在处理第一执行时长级别的其它任务，该第一执行时长级别的任务所需的执行时长比该任务所需的执行时长时间短。
在又一种可能的实施方式中，该处理器901还用于：若不存在具有目标计算模块的目标计算节点，且不存在第一计算节点，则通过上述通信接口903向第三计算节点发送任务，其中，该第二计算节点存在有标记了第三标签的计算模块，其中，该第三标签表征计算模块中正在处理第二执行时长级别的任务，该第二执行时长级别的任务所需的执行时长比该任务所需的执行时长时间长。
在又一种可能的实施方式中，该处理器901还用于：为目标计算节点的目标计算模块的标签标记第一标签，该第一标签表征计算模块中执行有目标执行时长级别的任务。
可以看出，调度节点90可以修改计算模块的标签，使得计算模块的标签与其正在处理的任务的执行时长级别相对应，便于后续进行其它任务的调度。
在又一种可能的实施方式中，该处理器901还用于：若任务执行结束，且目标计算模块没有正在处理的其它任务，则删除目标计算模块的第一标签。
各个单元的实现还可以对应参照图4所示的方法实施例的相应描述，该装置90可以为图4所示的实施例中的调度节点。
本申请实施例还提供了一种计算机可读存储介质，其特征在于，该计算机可读存储介质中存储有计算机指令，所述计算机指令用于实现上述的资源调度方法，例如图4所示的实施例的资源调度方法。
本申请实施例还提供了一种芯片系统，该芯片系统包括至少一个处理器，存储器和接口电路，所示接口电路用于为至少一个处理器提供信息输入/输出，该至少一个存储器中存储有计算机指令，当计算机指令在一个或多个处理器上运行时，所述芯片系统执行上述的资源调度方法，例如图4所示的实施例的资源调度方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机指令产品的形式实现。在计算机上加载和执行该计算机指令指令时，可以全部或部分地实现本申请实施例所描述的流程或功能。该计算机可以是通用计算机、专用计算机、计算机网络、或者其它可编程装置。该计算 机指令可以存储在计算机可读存储介质中，或者通过计算机可读存储介质进行传输。该计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如，固态硬盘(solid state disk，SSD))等。
本申请方法实施例中的步骤可以根据实际需要进行顺序调整、合并和删减。
本申请装置实施例中的模块可以根据实际需要进行合并、划分和删减。
以上，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到的变化或替换，都应涵盖在本申请的保护范围之内。

Claims (26)

  1. 一种资源调度方法，其特征在于，所述方法包括：
    调度节点接收任务；
    所述调度节点获取所述任务所属的目标执行时长级别，其中，所述目标执行时长级别用于表征时间长度，所述目标执行时长级别指示使用多个计算节点中的目标计算节点的目标计算模块来执行所述任务；
    所述调度节点优先向所述目标执行时长级别对应的所述目标计算节点发送所述任务。
  2. 根据权利要求1中所述的方法，其特征在于，所述调度节点获取所述任务所属的目标执行时长级别之前，还包括：
    所述调度节点接收针对所述任务的调度请求，所述调度请求中包括所述目标执行时长级别；
    所述调度节点获取所述任务所属的目标执行时长级别，包括：
    所述调度节点解析所述调度请求，得到所述目标执行时长级别。
  3. 根据权利要求1中所述的方法，其特征在于，所述调度节点获取所述任务所属的目标执行时长级别，包括：
    所述调度节点确定与所述任务的执行时长对应的执行时长级别，确定的执行时长级别为所述目标执行时长级别。
  4. 根据权利要求1-3中任一项所述的方法，其特征在于，所述目标执行时长级别为三种执行时长级别中的一种执行时长级别；所述三种执行时长级别包括长执行时长、中执行时长和短执行时长；
    其中，所述长执行时长大于或者等于第一阈值；
    所述中执行时长小于第一阈值且大于第二阈值；
    所述短执行时长小于或者等于第二阈值。
  5. 根据权利要求1-4中任一项所述的方法，其特征在于，所述调度节点优先向所述目标执行时长级别对应的所述目标计算节点发送所述任务，包括：
    在存在具有所述目标计算模块的所述目标计算节点时，向所述目标计算节点发送所述任务。
  6. 根据权利要求5中任一项所述的方法，其特征在于，所述目标计算模块为标记有第一标签的计算模块，所述第一标签表征所述计算模块中执行有所述目标执行时长级别的其它任务。
  7. 根据权利要求1至6中任一项所述的方法，其特征在于，所述调度节点优先向所述目标执行时长级别对应的所述目标计算节点发送所述任务，包括：
    在不存在具有所述目标计算模块的所述目标计算节点时，向所述多个计算节点中的 其它计算节点发送所述任务。
  8. 根据权利要求7中所述的方法，其特征在于，在不存在具有所述目标计算模块的所述目标计算节点时，向所述多个计算节点中的其它计算节点发送所述任务务，包括：
    在不存在具有所述目标计算模块的所述目标计算节点时，向第一计算节点发送所述任务，其中，所述第一计算节点中存在未处理任务的计算模块。
  9. 根据权利要求7或8中所述的方法，其特征在于，所述方法还包括：
    若不存在具有所述目标计算模块的所述目标计算节点，且不存在第一计算节点，则所述调度节点向第二计算节点发送所述任务，其中，所述第二计算节点存在有标记了第二标签的计算模块，其中，所述第二标签表征所述计算模块中正在处理第一执行时长级别的其它任务，所述第一执行时长级别的任务所需的执行时长比所述任务所需的执行时长时间短。
  10. 根据权利要求7或8中所述的方法，其特征在于，所述方法还包括：
    若不存在具有所述目标计算模块的所述目标计算节点，且不存在第一计算节点，则所述调度节点向第三计算节点发送所述任务，其中，所述第三计算节点存在有标记了第三标签的计算模块，其中，所述第三标签表征所述计算模块中正在处理第二执行时长级别的其它任务，所述第二执行时长级别的任务所需的执行时长比所述任务所需的执行时长时间长。
  11. 根据权利要求1-9任一项所述的方法，其特征在于，所述方法还包括：
    所述调度节点为所述目标计算节点的所述目标计算模块标记第一标签，所述第一标签表征所述计算模块中执行有所述目标执行时长级别的任务。
  12. 根据权利要求1-11任一项中所述的方法，其特征在于，所述方法还包括：
    若所述任务执行结束，且所述目标计算模块没有正在处理的其它任务，则所述调度节点删除所述目标计算模块的第一标签。
  13. 一种资源调度装置，其特征在于，包括：
    接收单元，用于接收任务；
    处理单元，用于获取所述任务所属的目标执行时长级别，其中，所述目标执行时长级别用于表征时间长度，所述目标执行时长级别指示使用所述多个计算节点中的目标计算节点的目标计算模块来执行所述任务；
    发送单元，用于优先向所述目标执行时长级别对应的所述目标计算节点发送所述任务。
  14. 根据权利要求13中所述的装置，其特征在于，所述接收单元，还用于接收针对所述任务的调度请求，所述调度请求中包括所述目标执行时长级别；
    所述处理单元，具体用于解析所述调度请求，得到所述目标执行时长级别。
  15. 根据权利要求13中所述的装置，其特征在于，所述处理单元，具体用于确定与所述任务的执行时长对应的执行时长级别，确定的执行时长级别为所述目标执行时长级别。
  16. 根据权利要求13-15中任一项所述的装置，其特征在于，所述目标执行时长级别为三种执行时长级别中的一种执行时长级别；所述三种执行时长级别包括长执行时长、中执行时长和短执行时长；
    其中，所述长执行时长大于或者等于第一阈值；
    所述中执行时长小于第一阈值且大于第二阈值；
    所述短执行时长小于或者等于第二阈值。
  17. 根据权利要求13-16中任一项所述的装置，其特征在于所述发送单元，具体用于在存在具有所述目标计算模块的所述目标计算节点时，向所述目标计算节点发送所述任务。
  18. 根据权利要求17中任一项所述的装置，其特征在于，所述目标计算模块为标记有第一标签的计算模块，所述第一标签表征所述计算模块中执行有所述目标执行时长级别的其它任务。
  19. 根据权利要求13-18中任一项所述的装置，其特征在于，所述发送单元，具体用于在不存在具有所述目标计算模块的所述目标计算节点时，向所述多个计算节点中的其它计算节点发送所述任务。
  20. 根据权利要求19中所述的装置，其特征在于，所述发送单元，具体用于在不存在具有所述目标计算模块的所述目标计算节点时，向第一计算节点发送所述任务，其中，所述第一计算节点中存在未处理任务的计算模块。
  21. 根据权利要求19或20中所述的装置，其特征在于，所述发送单元，还用于若不存在具有所述目标计算模块的所述目标计算节点，且不存在第一计算节点，则所述调度节点向第二计算节点发送所述任务，其中，所述第二计算节点存在有标记了第二标签的计算模块，其中，所述第二标签表征所述计算模块中正在处理第一执行时长级别的其它任务，所述第一执行时长级别的任务所需的执行时长比所述任务所需的执行时长时间短。
  22. 根据权利要求19或20中所述的装置，其特征在于，所述发送单元，还用于若不存在具有所述目标计算模块的所述目标计算节点，且不存在第一计算节点，则所述调度节点向第三计算节点发送所述任务，其中，所述第三计算节点存在有标记了第三标签的计算模块，其中，所述第三标签表征所述计算模块中正在处理第二执行时长级别的其它任务，所述第二执行时长级别的任务所需的执行时长比所述任务所需的执行时长时间 长。
  23. 根据权利要求13-21中任一项所述的装置，其特征在于，所述处理单元，还用于为所述目标计算节点的所述目标计算模块标记第一标签，所述第一标签表征所述计算模块中执行有所述目标执行时长级别的任务。
  24. 根据权利要求13-22中任一项所述的装置，其特征在于，所述处理单元，还用于若所述任务执行结束，且所述目标计算模块没有正在处理的其它任务，则所述调度节点删除所述目标计算模块的第一标签。
  25. 一种调度节点，其特征在于，所述调度节点包括处理器和存储器；所述处理器用于执行所述存储器存储的计算机指令，使得所述调度节点实现权利要求1-12中任一项所述的方法。
  26. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有计算机指令，所述计算机指令用于实现权利要求1-12中任一项所述的方法。
PCT/CN2021/075526 2020-04-21 2021-02-05 一种资源调度方法及相关装置 WO2021212965A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21792088.3A EP4123449A4 (en) 2020-04-21 2021-02-05 RESOURCE PLANNING PROCESS AND ASSOCIATED MECHANISM
US17/970,232 US20230037783A1 (en) 2020-04-21 2022-10-20 Resource scheduling method and related apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010320718.4A CN113535348A (zh) 2020-04-21 2020-04-21 一种资源调度方法及相关装置
CN202010320718.4 2020-04-21

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/970,232 Continuation US20230037783A1 (en) 2020-04-21 2022-10-20 Resource scheduling method and related apparatus

Publications (1)

Publication Number Publication Date
WO2021212965A1 true WO2021212965A1 (zh) 2021-10-28

Family

ID=78123914

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/075526 WO2021212965A1 (zh) 2020-04-21 2021-02-05 一种资源调度方法及相关装置

Country Status (4)

Country Link
US (1) US20230037783A1 (zh)
EP (1) EP4123449A4 (zh)
CN (1) CN113535348A (zh)
WO (1) WO2021212965A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115081233A (zh) * 2022-06-30 2022-09-20 上海弘玑信息技术有限公司 一种流程仿真方法及电子设备

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114003378B (zh) * 2021-10-26 2022-12-13 深圳证券信息有限公司 容器集群负载均衡方法、装置、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100088706A1 (en) * 2008-10-07 2010-04-08 Futurewei Technologies, Inc. User Tolerance Based Scheduling Method for Aperiodic Real-Time Tasks
CN108920261A (zh) * 2018-05-23 2018-11-30 中国航天系统科学与工程研究院 一种适于大规模并行数据处理任务的两级自适应调度方法
CN110109752A (zh) * 2019-04-12 2019-08-09 平安普惠企业管理有限公司 一种任务分配方法、装置、电子设备及存储介质
CN110166282A (zh) * 2019-04-16 2019-08-23 苏宁易购集团股份有限公司 资源分配方法、装置、计算机设备和存储介质
CN110362392A (zh) * 2019-07-15 2019-10-22 深圳乐信软件技术有限公司 一种etl任务调度方法、系统、设备及存储介质
CN110689265A (zh) * 2019-09-27 2020-01-14 苏州达家迎信息技术有限公司 任务分配方法、装置、设备及存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109375992A (zh) * 2018-08-17 2019-02-22 华为技术有限公司 一种资源调度方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100088706A1 (en) * 2008-10-07 2010-04-08 Futurewei Technologies, Inc. User Tolerance Based Scheduling Method for Aperiodic Real-Time Tasks
CN108920261A (zh) * 2018-05-23 2018-11-30 中国航天系统科学与工程研究院 一种适于大规模并行数据处理任务的两级自适应调度方法
CN110109752A (zh) * 2019-04-12 2019-08-09 平安普惠企业管理有限公司 一种任务分配方法、装置、电子设备及存储介质
CN110166282A (zh) * 2019-04-16 2019-08-23 苏宁易购集团股份有限公司 资源分配方法、装置、计算机设备和存储介质
CN110362392A (zh) * 2019-07-15 2019-10-22 深圳乐信软件技术有限公司 一种etl任务调度方法、系统、设备及存储介质
CN110689265A (zh) * 2019-09-27 2020-01-14 苏州达家迎信息技术有限公司 任务分配方法、装置、设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4123449A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115081233A (zh) * 2022-06-30 2022-09-20 上海弘玑信息技术有限公司 一种流程仿真方法及电子设备

Also Published As

Publication number Publication date
EP4123449A1 (en) 2023-01-25
US20230037783A1 (en) 2023-02-09
EP4123449A4 (en) 2023-07-26
CN113535348A (zh) 2021-10-22

Similar Documents

Publication Publication Date Title
US11681547B2 (en) File operation task optimization
US10223165B2 (en) Scheduling homogeneous and heterogeneous workloads with runtime elasticity in a parallel processing environment
WO2017133351A1 (zh) 一种资源分配方法及资源管理器
US20230037783A1 (en) Resource scheduling method and related apparatus
US10503558B2 (en) Adaptive resource management in distributed computing systems
KR20140014273A (ko) 휴대용 컴퓨팅 디바이스의 스위치 패브릭들 내에서 그리고 스위치 패브릭들에 걸쳐 마스터-슬레이브 쌍들을 동적으로 생성하고 서비스하는 방법 및 시스템
WO2018018611A1 (zh) 一种任务处理方法以及网卡
US9141436B2 (en) Apparatus and method for partition scheduling for a processor with cores
CN114579285B (zh) 一种任务运行系统、方法及计算设备
CN111343288B (zh) 作业调度方法、系统及计算设备
CN110806928A (zh) 一种作业提交方法及系统
US9607275B2 (en) Method and system for integration of systems management with project and portfolio management
JP2021022379A (ja) ハードウェアアクセラレータの自律ジョブキューイングシステム
CN111586140A (zh) 一种数据交互的方法及服务器
CN112860387A (zh) 分布式任务调度方法、装置、计算机设备及存储介质
WO2019029721A1 (zh) 任务的调度方法、装置、设备及存储介质
CN109558222B (zh) 批量业务进程监控方法、装置、计算机及可读存储介质
US11388050B2 (en) Accelerating machine learning and profiling over a network
CN114675954A (zh) 任务调度方法及装置
CN113641472A (zh) 分布式应用的异转同实现方法及装置
KR102642396B1 (ko) 제한된 gpu리소스를 사용한 딥러닝 추론 모델을 위한 배치 스케줄링 장치
US11343134B1 (en) System and method for mitigating analytics loads between hardware devices
US20230195546A1 (en) Message Management Method and Apparatus, and Serverless System
CN109145015B (zh) 基于结构化查询语言的数据查询方法、装置和系统
Mou et al. Joint Task Scheduling and Container Image Caching in Edge Computing

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21792088

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021792088

Country of ref document: EP

Effective date: 20221020

NENP Non-entry into the national phase

Ref country code: DE