WO2023024410A1 - 任务调度方法、装置、设备、存储介质及计算机程序产品、计算机程序 - Google Patents

任务调度方法、装置、设备、存储介质及计算机程序产品、计算机程序 Download PDF

Info

Publication number
WO2023024410A1
WO2023024410A1 PCT/CN2022/071421 CN2022071421W WO2023024410A1 WO 2023024410 A1 WO2023024410 A1 WO 2023024410A1 CN 2022071421 W CN2022071421 W CN 2022071421W WO 2023024410 A1 WO2023024410 A1 WO 2023024410A1
Authority
WO
WIPO (PCT)
Prior art keywords
node
task
resource units
video processing
tasks
Prior art date
Application number
PCT/CN2022/071421
Other languages
English (en)
French (fr)
Inventor
许波
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2023024410A1 publication Critical patent/WO2023024410A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining

Definitions

  • the embodiments of the present application relate to but are not limited to the field of computer technology, and in particular, relate to a task scheduling method, device, device, storage medium, computer program product, and computer program.
  • a video processing system needs to process thousands of video streams if it is small, and it needs Handle video streams of 10,000 or even 100,000 channels.
  • a video processing system may be implemented in a cluster manner, and a received video processing task may be dispatched to different nodes in the cluster for execution.
  • the resolutions of video streams processed in different video processing tasks are various, common resolutions are 1080P (ie 1920*1080), 2K (ie 2560*1440), 4K (ie 4096*2048), etc.;
  • Various video processing systems propose multiple resolution requirements, but the same video processing system in the related art can only process video streams with a specific resolution, and video streams with different resolutions need to be converted into the same specific resolution before processing .
  • each node in the cluster of the video processing system may have different hardware resources; each node uses a different performance graphics card (ie, Graphics Processing Unit, GPU), central processing unit (Central Processing Unit, CPU), memory, etc.
  • graphics card ie, Graphics Processing Unit, GPU
  • CPU Central Processing Unit
  • memory etc.
  • the processing capabilities of video streams are different, and in related technologies, video processing tasks cannot be uniformly scheduled on nodes with different hardware resources in a video processing system cluster.
  • embodiments of the present application provide a task scheduling method, device, device, storage medium, computer program product, and computer program.
  • An embodiment of the present application provides a task scheduling method, the method including:
  • An embodiment of the present application provides a task scheduling device, the device comprising:
  • the first acquisition part is configured to acquire the task type of the video processing task to be executed and the resolution of the video stream to be processed in the video processing task;
  • a first determining part configured to determine the number of resource units required to perform the video processing task based on the task type and the resolution
  • the second determining part is configured to determine resource unit usage information on at least one node in the cluster executing the video processing task based on the task type; wherein, the number of resource units on each node is based on the The hardware type of the node is determined;
  • a first scheduling section configured to schedule a target node from the at least one node to perform the video processing based on the number of resource units required to perform the video processing task and usage information of the resource units of each of the nodes task; wherein, the number of idle resource units in the target node is not less than the number of resource units required to execute the video processing task.
  • An embodiment of the present application provides a task scheduling device, including a memory and a processor, the memory stores a computer program that can run on the processor, and the processor implements some or all of the steps in the above method when executing the program .
  • An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented.
  • An embodiment of the present application provides a computer program, including computer readable codes.
  • a processor in the computer device executes some or all of the steps in the above method.
  • An embodiment of the present application provides a computer program product.
  • the computer program product includes a non-transitory computer-readable storage medium storing a computer program.
  • the computer program is read and executed by a computer, part or all of the above methods are implemented. All steps.
  • the task type of the video processing task to be executed and the resolution of the video stream to be processed in the video processing task are obtained; based on the task type and resolution, the number of resource units required to execute the video processing task is determined, And determine the use information of resource units on at least one node in the cluster that executes the video processing task based on the task type, wherein the number of resource units on each node is determined based on the hardware type of the node; based on the resources required to perform the video processing task
  • the number of units and the usage information of the resource units of each node, and the target node is scheduled from at least one node to perform the video processing task, wherein the number of idle resource units in the target node is not less than the resource required by the video processing task the number of units.
  • FIG. 1 is a schematic diagram of an implementation flow of a task scheduling method provided in an embodiment of the present application
  • FIG. 2 is a schematic diagram of an implementation flow of a task scheduling method provided in an embodiment of the present application
  • FIG. 3 is a schematic diagram of an implementation flow of a task scheduling method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of an implementation flow of a task scheduling method provided in an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an implementation flow of a task scheduling method provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of an implementation flow of a task scheduling method provided by an embodiment of the present application.
  • FIG. 7A is a schematic diagram of an implementation architecture of a task scheduling method provided by an embodiment of the present application.
  • FIG. 7B is a schematic diagram of the implementation flow of a task allocation method provided by the embodiment of the present application.
  • FIG. 8 is a schematic diagram of the composition and structure of a task scheduling device provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a hardware entity of a task scheduling device provided in an embodiment of the present application.
  • references to “some embodiments” describe a subset of all possible embodiments, but it is understood that “some embodiments” may be the same subset or a different subset of all possible embodiments, and Can be combined with each other without conflict.
  • the term “first/second/third” is only used to distinguish similar objects, and does not represent a specific order for objects. Understandably, “first/second/third” is used in Where permitted, the specific order or sequence may be interchanged such that the embodiments of the application described herein can be practiced in other sequences than illustrated or described herein.
  • Video analysis refers to the use of analysis methods such as target detection, feature extraction, object recognition, and deep learning to organize video content into information that can be recognized, understood, and retrieved by computers and/or humans.
  • Algorithm application element In practical applications, different types of tasks can be defined according to different business objectives. For different types of tasks, different applications can be used to process different algorithms to obtain corresponding processing results.
  • An application that processes a certain type of task is an algorithm application.
  • the tasks for which the algorithm applies meta-processing may include, but are not limited to, video parsing tasks, audio parsing tasks, image processing tasks, artificial general intelligence (AGI) tasks, and the like.
  • Algorithm application units can be applications running on cluster nodes. The same cluster can include at least two types of nodes running different algorithm application units. Through the task scheduling service of the cluster, different types of tasks can be scheduled to the corresponding types of tasks. On the node, the corresponding algorithm application element is used to process the task.
  • An embodiment of the present application provides a task scheduling method, which can be executed by a task scheduling service running on a computer device.
  • computer equipment refers to servers, notebook computers, tablet computers, desktop computers, smart TVs, set-top boxes, mobile devices (such as mobile phones, portable video players, personal digital assistants, dedicated messaging devices, portable game devices), etc. Devices with data processing capabilities.
  • Fig. 1 is a schematic diagram of the implementation flow of a task scheduling method provided in the embodiment of the present application. As shown in Fig. 1, the method includes:
  • Step S101 acquiring the task type of the video processing task to be executed and the resolution of the video stream to be processed in the video processing task;
  • the video processing task may be any suitable task for processing video streams, for example, tasks such as parsing, encrypting, intelligent analysis, target detection, audio analysis, and general artificial intelligence processing for video streams.
  • the video processing task to be executed can be actively obtained by the task scheduling service from an external system, can also be issued by an external system to the scheduling service, or can be automatically generated by the scheduling service based on preset task configuration information, which is not limited here .
  • the external system can write the video processing tasks to be executed into the message queue, and the scheduling service can read the video processing tasks to be executed from the message queue according to a specific message consumption strategy; the scheduling service can also provide a
  • the external system can send the video processing task to be executed to the scheduling service by calling this interface.
  • the task types of video processing tasks may include, but are not limited to, video stream processing types (such as video parsing, encryption, intelligent analysis, target detection, anomaly detection, etc.), algorithm types used to process video streams, and the like. During implementation, those skilled in the art may classify video processing tasks in an appropriate manner according to actual conditions.
  • the task type of the video processing task and the resolution of the video stream to be processed in the video processing task may be determined based on configuration information of the video processing task to be executed.
  • the task content of the video processing task to be executed can be analyzed to determine the task type of the video processing task, and the resolution detection of the video stream to be processed in the video processing task can be performed to obtain the video stream resolution.
  • Step S102 based on the task type and the resolution, determine the number of resource units required to execute the video processing task.
  • the resource unit is a pre-defined basic unit used to measure hardware resources required to execute video processing tasks, and is an abstract definition of the task processing capabilities that specific hardware resources can provide.
  • a resource unit can correspond to a certain task processing capability, and then can correspond to a certain hardware resource.
  • a resource unit can correspond to a specific running memory, a specific number of CPU cores, and/or a specific number of graphics cards. capability, and thus the specific operating memory, the specific number of CPU cores, and/or the specific number of graphics cards, etc. may be defined as a resource unit.
  • those skilled in the art may define resource units in an appropriate manner according to actual conditions, which is not limited here.
  • the task processing capability corresponding to a resource unit can be pre-defined according to expert experience, and then the hardware resources corresponding to a resource unit can be determined, or the distribution of task types and video stream resolutions of historically executed video processing tasks can also be used
  • An appropriate basic unit for measuring hardware resources required to execute video processing tasks, that is, a resource unit, is abstracted.
  • the hardware resources required to perform video processing tasks of different task types are different, and the hardware resources required to process video streams with different resolutions are also different, so based on the task types of the video processing tasks to be performed and the The resolution of the video stream to be processed in the video processing task can determine the hardware resources required to execute the video processing task. Since resource units are used to measure the hardware resources required to execute the video processing task, based on the task type and the resolution, the number of resource units required to execute the video processing task can be determined.
  • the computing workload required to perform the video processing task will be different, that is, the required computing resources will also be different, so it can be based on the video processing task
  • the resolution of the video stream to be processed determines the number of resource units required to perform video processing tasks.
  • the task types of video processing tasks are different, and the computing workload and storage space required to execute the video processing tasks will be different. Therefore, the execution of video processing tasks can be determined based on the number of frames of the video stream to be processed in the video processing tasks. The number of resource units required.
  • the video processing task can be analyzed based on the task type of the video processing task to be executed and the resolution of the video stream to be processed in the video processing task, and the hardware resources required for executing the video processing task can be estimated, and then Determine the number of resource units required to perform the video processing task, for example, determine the number of CPU cores, the number of GPUs, and the memory required for the video processing task based on the resolution and task type of the video stream to be processed in the video processing task Space, and based on the number of CPU cores, number of GPUs, memory space required for video processing tasks, and the number of CPU cores, number of GPUs, and memory space that a single resource unit has, determine the number of resource units required for video processing tasks ; It is also possible to detect in advance the hardware resources occupied by video processing tasks of different task types and resolutions of different video streams when running in computer equipment, and to calibrate the requirements of video processing tasks of different task types and different video stream resolutions.
  • the number of resource units and then obtain the corresponding relationship between the task type of the video processing task and the resolution of the video stream and the number of resource units. Detect the hardware resources occupied by each video processing task when running in the computer equipment, calibrate the number of resource units required to process video streams of different resolutions and/or video processing tasks of different task types, and then obtain the resolution. , the corresponding relationship between the task type and the number of resource units.
  • Step S103 based on the task type, determine resource unit usage information on at least one node in the cluster that executes the video processing task; wherein, the number of resource units on each node is determined based on the hardware type of the node of.
  • the cluster performing the video processing task may include at least one node, and each node may be a server or a terminal.
  • the hardware resources on each node can be abstracted into a specific number of resource units.
  • the number of resource units on each node may be determined based on the hardware type of the node. Nodes of different hardware types may have different resource units. For example, since graphics cards produced by different manufacturers or graphics cards of different specifications usually have different performances, the hardware resources that can be provided are different, so different manufacturers are used. Or nodes with graphics cards with different specifications may have different resource unit quantities.
  • the hardware type of the node may include but not limited to one or more of the node's graphics card model, graphics card specification, processor type, processor model, memory type, memory capacity, disk type, and disk capacity.
  • the total amount of hardware resources owned by the node can be determined according to the hardware type of the node, and the total amount of hardware resources owned by the node can be measured by resource units to obtain the number of resource units on the node; Analyze the hardware resources owned by the node, calibrate the number of resource units owned by nodes of different hardware types, and then obtain the correspondence between the hardware type and the number of resource units. Using this correspondence, the node can be determined based on the hardware type of the node. The number of resource units.
  • the usage information of resource units on a node may include but not limited to one or more of the number of occupied resource units on a node, the number of idle resource units, tasks occupying resource units, the number of resource units occupied by each task, etc. Various. During implementation, the usage information of resource units on each node can be determined by monitoring the running status of each task on each node and the number of resource units owned by each node, or each node can count the usage of its own resource units information, and report the usage information of resource units to the task scheduling service.
  • the cluster for executing video processing tasks may include at least one type of node, and each type of node is used to process video processing tasks of different task types. Based on the task type of the video processing task to be executed, usage information of resource units on at least one node processing the video processing task of the task type in the cluster may be determined. In some implementation manners, all nodes in the cluster are used to execute the video processing task of the task type, and usage information of resource units on all nodes in the cluster can be obtained. In some implementation manners, a correspondence between task types and nodes may be predetermined, and based on the correspondence, nodes for processing video processing tasks of different task types may be determined.
  • video processing tasks of different task types need to be processed by different application programs, and the corresponding relationship between task types and application programs can be determined in advance. Based on the corresponding relationship, video processing tasks of different task types can be determined.
  • the application program of the task so that according to the application programs running on different nodes, the nodes that process the video processing tasks of different task types are determined.
  • Step S104 based on the number of resource units required to execute the video processing task and the resource unit usage information of each node, schedule a target node from the at least one node to execute the video processing task; wherein, the The number of idle resource units in the target node is not less than the number of resource units required to execute the video processing task.
  • the number of idle resource units in at least one node is not less than the number of resource units required by the video processing task the target node, and dispatch the video processing task to the target node to execute the video processing task.
  • the idle resource units on the scheduled target node may be unoccupied resource units, or resource units released after moving or unloading tasks running on at least one node, which is not limited here.
  • a policy determines a target node from among the plurality of nodes.
  • the above step S102 may include: based on the task type and the resolution, query the preset correspondence between the task type, resolution and the number of resource units, and determine to execute the video processing task The number of resource units required.
  • the correspondence between the preset resolution and the number of resource units may be pre-calibrated for the hardware resources occupied by multiple video processing tasks that process video streams of different resolutions and task types when running on a computer device owned.
  • the processing resolutions are 1080P, 2K, and 4K respectively.
  • the number of resource units required for the video processing task of the video stream will increase sequentially.
  • the task type of the video processing task to be executed and the resolution of the video stream to be processed in the video processing task are acquired; then, based on the task type and resolution, the resources required to execute the video processing task are determined The number of units, and determine the use information of resource units on at least one node in the cluster performing video processing tasks based on the task type, wherein the number of resource units on each node is determined based on the hardware type of the node; finally, based on the execution video
  • the number of resource units required to process the task and the usage information of the resource units of each node schedule the target node from at least one node to perform the video processing task, wherein the number of idle resource units in the target node is not less than the video The number of resource units required to process the task.
  • An embodiment of the present application provides a task scheduling method, which can be executed by a task scheduling service running on a computer device. As shown in Figure 2, the method includes:
  • Step S201 acquiring the task type of the video processing task to be executed and the resolution of the video stream to be processed in the video processing task.
  • Step S202 based on the task type and the resolution, determine the number of resource units required to execute the video processing task.
  • step S201 to step S202 respectively correspond to the aforementioned step S101 to step S102, and the specific implementation manners of the aforementioned step S101 to step S102 can be referred to for implementation.
  • Step S203 based on the task type, determine an application program for processing the video processing task of the task type.
  • video processing tasks of different task types need to be processed by different application programs.
  • the correspondence between task types and application programs may be determined in advance, and based on the correspondence, application programs for processing video processing tasks of different task types may be determined.
  • the application program may be a process or thread running on the node, which is not limited here.
  • Step S204 determining at least one node running the application program from the cluster, wherein the number of resource units on each node is determined based on the hardware type of the node.
  • the cluster is used to execute video processing tasks of at least two task types, and each video processing task of the task type is processed on at least one node.
  • the same application can run on at least one node in the cluster.
  • at least one node running the application program in the cluster may be determined according to the configuration information of the cluster or the corresponding relationship between a specific node and the application program.
  • Step S205 determining usage information of resource units on each node in the at least one node.
  • Step S206 based on the number of resource units required to execute the video processing task and the resource unit usage information of each node, schedule a target node from the at least one node to execute the video processing task; wherein, The number of idle resource units in the target node is not less than the number of resource units required to execute the video processing task.
  • step S206 corresponds to the aforementioned step S104, and the specific implementation manner of the aforementioned step S104 can be referred to for implementation.
  • the cluster is used to execute video processing tasks of at least two task types, each task type of video processing task is processed on at least one node, and the video processing task of the task type is determined based on the task type of the video processing task
  • the application program of the task determines at least one node running the application program from the cluster, and determines usage information of resource units on each node of the at least one node. In this way, video processing tasks of different task types can be scheduled to nodes where corresponding application programs are located, so that unified scheduling of video processing tasks of different task types in the same cluster can be realized.
  • An embodiment of the present application provides a task scheduling method, which can be executed by a task scheduling service running on a computer device. As shown in Figure 3, the method includes:
  • Step S301 acquiring the task type of the video processing task to be executed and the resolution of the video stream to be processed in the video processing task.
  • Step S302 based on the task type and the resolution, determine the number of resource units required to execute the video processing task.
  • Step S303 based on the task type, determine the number of idle resource units on at least one node in the cluster that executes the video processing task; wherein, the number of resource units on each node is based on the hardware type of the node definite.
  • steps S301 to S303 correspond to the aforementioned steps S101 to S103 respectively, and for implementation, reference may be made to the specific implementation manners of the aforementioned steps S101 to S103.
  • Step S304 Determine at least one available idle node from the at least one node based on the number of resource units required to execute the video processing task and the number of idle resource units in each node.
  • an available idle node is a node whose number of idle resource units is not less than the number of resource units required to perform video processing tasks.
  • at least one available idle node may be determined by checking the number of idle resource units on each node of the at least one node and comparing it with the number of resource units required to perform video processing tasks.
  • the node may be determined as an available idle node.
  • the running tasks in at least one node may be performed Unloading or moving, so that at least one node has a node whose number of idle resource units is not less than the number of resource units required for performing video processing tasks, and determining this node as an available idle node.
  • Step S305 determining a target node from at least one available idle node.
  • any suitable selection strategy may be adopted according to the actual situation, and at least one node may be selected from at least one available idle node as the target node, which is not limited here.
  • at least one node may be randomly determined as the target node from at least one available idle node, or the available idle node with the lowest load may be determined as the target node according to the current load condition of each available idle node.
  • Step S306 scheduling the target node to execute the video processing task.
  • the number of the at least one available idle node is at least two, and the above step S305 may include: determining a target idle node from the at least two available idle nodes in a load balancing manner.
  • an appropriate load balancing strategy may be adopted according to actual conditions to determine a target idle node from at least two available idle nodes, which is not limited in this embodiment of the present application.
  • the method may further include: when determining that there are multiple available idle nodes among the at least one node, determining the node with the lowest load among the multiple available idle nodes as the final available idle node .
  • the load of a node may be determined by the number of tasks being executed and/or waiting to be executed on the node, and the larger the number of tasks being executed and/or waiting to be executed, the higher the load of the node.
  • the load of a node can also be determined by the resource occupancy rate of the node. The higher the resource occupancy rate, the higher the load of the node.
  • the above step S304 may include:
  • Step S341 based on the number of resource units required to execute the video processing task and the number of idle resource units in each node, determine whether there is an available idle node in the at least one node;
  • the number of idle resource units in each node is not less than the number of resource units required to perform video processing tasks, so as to determine whether there is an available idle node in at least one node.
  • Step S342 when it is determined that there is no available idle node in the at least one node, perform task scheduling adjustment on the at least one node to obtain at least one available idle node.
  • adjusting the task scheduling of at least one node may include moving a task on one node to another node for execution, so as to concentrate idle resource units on each node, and may also include offloading tasks on at least one node, thereby
  • the number of free resource units in at least one node can be increased. For example, if there is 1 free resource unit on node 1 and 1 free resource unit on node 2, you can move task 1, which occupies 1 resource unit on node 1, to node 2 for execution to increase the idle time on node 1.
  • task 1 occupying 1 resource unit on node 1 may be unloaded, so that the number of free resource units on node 1 is increased to 2.
  • the number of idle resource units in at least one node can be increased, so that the number of idle resource units that may appear in at least one node after the task scheduling adjustment is not less than the resource units required to perform video processing tasks
  • the number of nodes that is, available idle nodes.
  • the task can also be further moved or unloaded to the adjusted at least one node to obtain an available idle node, for example, the adjusted at least one node can be At least one currently running task is unloaded to release the occupied resource units.
  • the same video processing system can only process video streams with a specific resolution, and different nodes may not necessarily have the same processing capabilities for video streams, so it is impossible to have different hardware in the video processing system cluster
  • the video processing tasks are scheduled uniformly on the nodes of the resources, so it is impossible to perform unified task scheduling adjustments on the video processing tasks among the nodes.
  • those skilled in the art have no motivation to think of organizing and concentrating idle resource capabilities distributed on different nodes to make full use of hardware resources, let alone adjusting task scheduling on at least one node in the cluster , to at least increase the number of idle resource units in one node to obtain at least one available idle node. Therefore, the above steps S341 to S342 are not obvious to those skilled in the art.
  • performing task scheduling adjustment on the at least one node described in step S342 above to obtain at least one available idle node may include:
  • Step S3421 based on the number of resource units required by the video processing task, determine at least one first node from the at least one node;
  • At least one first node may be determined from the at least one node, and the tasks running on the first node may be scheduled and adjusted, so as to obtain from the at least one first node A node releases at least one occupied resource unit.
  • At least one first node may be determined from at least one node in an appropriate manner according to actual conditions, which is not limited here. For example, at least one node may be determined as the first node with the number of resource units greater than the number of resource units required by the video processing task, or the priority of the task running in at least one node may be lower than the preset The priority node is determined as the first node, and at least one node whose number of idle resource units is greater than a set number threshold may also be determined as the first node.
  • Step S3422 unload at least one running task from the at least one first node
  • the task to be unloaded may be determined according to the priority of the task, or randomly determined, or determined according to the number of resource units occupied by the task, which is not limited here.
  • Unloading tasks can include suspending or ending the execution of the task, executing the task again after new resource units are released on the node, and loading the task if there are enough free resource units on other nodes to run on the other node.
  • Step S3423 taking the at least one first node after the task is offloaded as the at least one available idle node.
  • the method may further include: loading the unloaded at least one running task onto the second node of the at least one node.
  • the second node is a node different from the first node among the at least one node.
  • the above step S3421 may include:
  • Step S3421a determining at least one low-priority task whose priority is lower than the video processing task from the tasks already running on the at least one node;
  • Step S3421b based on the number of resource units required by the video processing task and the number of resource units occupied by each of the low priority tasks, determine at least one task with the lowest priority among the at least one low priority task For the task to be uninstalled.
  • At least one task to be unloaded may be determined based on the number of resource units required by the video processing task and the number of resource units occupied by each low-priority task, and the determined at least one task to be unloaded is at least one low-priority task At least one task with the lowest priority, so that after the at least one task to be unloaded is unloaded, the number of idle resource units in at least one node is not less than the number of resource units required by the video processing task, or after the task is unloaded After the idle resource units on at least one node are concentrated, the number of idle resource units in at least one node is not less than the number of resource units required by the video processing task.
  • the above step S3422 may include: offloading the at least one task to be offloaded from the at least one first node.
  • the Tasks to be unloaded are unloaded; it is also possible to unload at least one low-priority task in sequence according to the priority of the low-priority tasks from low to high, and after each unloading of a low-priority task, determine the current status of at least one node.
  • Whether there is an available idle node if there is currently an available idle node, directly determine the available idle node, if there is no available idle node, perform task scheduling adjustment on at least one node after the task is offloaded, and determine the adjusted Whether there is an available idle node in at least one node of the adjusted node, if there is an available idle node in at least one node after adjustment, determine the available idle node, if there is no available idle node in the adjusted at least one node, continue The next low-priority task is offloaded until there is an available idle node in at least one node after the task is unloaded or an available idle node is obtained after task scheduling adjustment is performed on at least one node after the task is unloaded.
  • At least one available idle node is determined from at least one node, and at least one available idle node is determined from target node, and schedule video processing tasks to the available idle node.
  • the load balancing method can be used to determine the target node from the multiple available idle nodes, so that the load among the nodes can be better balanced, and the Overall performance of video processing task execution.
  • the task scheduling adjustment may be performed on at least one node, so as to at least increase the number of idle resource units in one of the nodes, and from the adjusted task scheduling
  • An available idle node is determined in at least one node, so that multiple idle resource units distributed on different nodes can be concentrated on one node, so that hardware resources can be more fully utilized, and the overall performance of task execution can be further improved.
  • at least one low-priority task whose priority is lower than the current video processing task can be offloaded from at least one node, and enough resource units can be released to execute the current video processing task.
  • Video processing tasks in this way, can better meet the hardware resource requirements of higher priority tasks.
  • An embodiment of the present application provides a task scheduling method, which can be executed by a task scheduling service running on a computer device. As shown in Figure 4, the method includes:
  • Step S401 acquiring the task type of the video processing task to be executed and the resolution of the video stream to be processed in the video processing task.
  • Step S402 based on the task type and the resolution, determine the number of resource units required to execute the video processing task.
  • Step S403 based on the task type, determine resource unit usage information on at least one node in the cluster that executes the video processing task; wherein, the number of resource units on each node is determined based on the hardware type of the node of.
  • Step S404 based on the number of resource units required to execute the video processing task and the resource unit usage information of each node, schedule a target node from the at least one node to execute the video processing task; wherein, The number of idle resource units in the target node is not less than the number of resource units required to execute the video processing task.
  • steps S401 to S404 respectively correspond to the foregoing steps S101 to S104, and for implementation, reference may be made to the specific implementation manners of the foregoing steps S101 to S104.
  • Step S405 if the preset load balancing trigger condition is satisfied, based on the preset load threshold of each node, perform load balancing adjustment on the at least one node.
  • the preset load balancing trigger conditions may be determined according to actual business requirements, and may include but not limited to one or more of preset timing trigger conditions, task addition and deletion frequency reaching a preset frequency threshold, and the like.
  • stricter load balancing trigger conditions can be set, for example, when the load balancing trigger condition is a preset timing trigger condition , you can set a larger timing trigger interval, and the load balancing trigger condition is that when the frequency of adding and deleting tasks reaches the preset frequency threshold, you can set a higher frequency threshold; for tasks that require less stability and task rebalancing
  • you can set looser load balancing trigger conditions for business scenarios with low cost.
  • the load balancing trigger condition is a preset timing trigger condition
  • the load balancing trigger condition is the task.
  • a lower frequency threshold may be set.
  • the load threshold is a preset value, which can represent the sensitivity of triggering load balancing. Different load thresholds can be set for each node, or the same load threshold can be set for at least one node, which is not limited here. When the load threshold is set higher, less load balancing adjustments can be performed, but the stability of the currently running tasks can be guaranteed. When the load threshold is set low, load balancing adjustments can be performed more frequently to better ensure that the load among the nodes is more balanced.
  • the preset load threshold may include at least one of the highest load water level and the lowest load water level. When the preset load balancing trigger condition is met, at least one node may be adjusted for load balancing, so that between nodes Keep the load balanced.
  • At least one task running on a node with a higher load among at least one node may be moved to at least one node with a lower load for execution, so that the load balance among the nodes is maintained.
  • at least one task running on a node whose load exceeds the first load threshold among at least one node can be moved to at least one node with the lowest load among at least one node for execution, or at least one task whose load exceeds the first load threshold among at least one node can also be executed.
  • At least one task running on the node is moved to at least one node whose load is lower than the second load threshold for execution, wherein the first load threshold is higher than the second load threshold, and the task running on the node with the highest load among the at least one node can also be moved to At least one task is moved to execute on at least one node whose load is lower than a second load threshold.
  • the load threshold includes a first load threshold and a second load threshold, the first load threshold is not lower than the second load threshold, and the above step S405 is based on each preset
  • the load threshold of the node, and adjusting the load balance of the at least one node may include: determining that there is a third node with a load higher than the first load threshold among the at least one node, and there is a third node with a load lower than the first load threshold.
  • the fourth node having a load threshold of two, moving a preset number of tasks in the third node to the fourth node.
  • the preset quantity can be preset according to actual conditions.
  • the first load threshold may be higher than or equal to the second load threshold, which is not limited here.
  • the fourth node after moving a preset number of tasks in the third node to the fourth node, it is also possible to detect whether there are both the third node and the fourth node in at least one node after the task is moved. If the third node and the fourth node exist at the same time in the at least one node after adjustment, continue to perform load balancing adjustment on the at least one node until the third node or the fourth node does not exist in the adjusted at least one node.
  • the preset load balancing trigger condition includes at least one of the following: a preset timing trigger condition; the frequency of adding and deleting tasks reaches a preset frequency threshold.
  • a preset timing trigger condition the frequency of adding and deleting tasks reaches a preset frequency threshold.
  • both the timing trigger condition and the frequency threshold can be set in advance according to actual needs.
  • Timing trigger conditions may include, but not limited to, timing trigger time, timing trigger times, and the like.
  • the frequency threshold can be a positive integer. If the frequency threshold is equal to 1, the load balancing adjustment is performed every time a task is added or deleted.
  • load balancing adjustment is performed on at least one node based on a preset load threshold of each node. In this way, the load among the nodes can be better balanced, and the situation that the nodes are fully loaded can be reduced, thereby further improving the overall performance of video processing task execution.
  • An embodiment of the present application provides a task scheduling method, which can be executed by a task scheduling service running on a computer device. As shown in Figure 5, the method includes:
  • Step S501 acquiring the task type of the video processing task to be executed and the resolution of the video stream to be processed in the video processing task.
  • Step S502 based on the task type and the resolution, determine the number of resource units required to execute the video processing task.
  • Step S503 based on the task type, determine resource unit usage information on at least one node in the cluster that executes the video processing task; wherein, the number of resource units on each node is determined based on the hardware type of the node of.
  • Step S504 based on the number of resource units required to execute the video processing task and the resource unit usage information of each node, schedule a target node from the at least one node to execute the video processing task; wherein, The number of idle resource units in the target node is not less than the number of resource units required to execute the video processing task.
  • steps S501 to S504 respectively correspond to the foregoing steps S101 to S104, and for implementation, reference may be made to the specific implementation manners of the foregoing steps S101 to S104.
  • Step S505 if it is detected that there is an abnormal node in the at least one node, acquire tasks to be rescheduled running on the abnormal node.
  • the abnormal node may include but not limited to a node with at least one abnormality such as downtime, crash, system crash, etc., and tasks cannot be run normally on the abnormal node.
  • the task to be rescheduled is the task that needs to be executed on the abnormal node.
  • the task to be rescheduled on the abnormal node can be obtained, and the task to be rescheduled can be scheduled to On other nodes except abnormal nodes.
  • it may be determined whether there is an abnormal node in at least one node by any suitable means such as log monitoring, heartbeat detection, and error reporting.
  • Step S506 based on the number of resource units required by the task to be rescheduled and the resource unit usage information of each node, from the at least one node except the abnormal node, The scheduling target node executes the task to be rescheduled.
  • step S506 corresponds to the above-mentioned step S104, and the specific implementation manner of the above-mentioned step S104 can be referred to for implementation.
  • the number of tasks to be rescheduled is multiple, and the above step S506 may include: sequentially targeting each of the multiple tasks to be rescheduled in order of priority from high to low The scheduled task schedules a target node from the other nodes to execute the task to be rescheduled based on the number of resource units required by the task to be rescheduled and usage information of resource units of each node.
  • the task to be rescheduled running on the abnormal node is obtained, and based on the number of resource units required by the task to be rescheduled and each Use information of the resource unit of the node, and schedule the target node from other nodes except the abnormal node in at least one node, so as to execute the task to be rescheduled.
  • a node in the cluster is abnormal, it can ensure that the tasks running on the abnormal node can be scheduled to be executed on the normal node, so that the self-healing of the abnormal task caused by the abnormal node can be realized, and the task execution in the cluster can be improved. stability.
  • the embodiments of the present application can be applied to video processing systems, including but not limited to smart security, smart city and other scenarios, and are used to dispatch tasks including video streams issued by users to nodes in the cluster for processing.
  • the embodiment of the present application provides a task scheduling method, which is executed by the task scheduling service.
  • the video processing tasks for processing video streams with different resolutions can be uniformly scheduled.
  • it can prioritize high-priority tasks, and has load balancing capabilities. It can schedule tasks on high-load nodes to low-load nodes for execution.
  • it has self-healing capabilities for tasks running on abnormal nodes. When a node in the cluster is abnormal, the tasks running on the node can be scheduled to other nodes.
  • the task scheduling method provided by the embodiment of the present application can be implemented through the following steps:
  • Step S601 receiving a task delivered by an external system by calling a task delivery interface
  • Step S602 according to the issued task, determine the application program corresponding to the task, and determine multiple nodes running the application program in the cluster;
  • Step S603 when it is determined that at least one of the plurality of nodes has sufficient resources to execute the task, schedule the task to the node with the lowest load among the at least one node to execute the task;
  • Step S604 when it is determined that none of the multiple nodes have enough resources to execute the task, adjust the tasks running on the multiple nodes, and concentrate the idle resources on each node to one node;
  • Step S605 in the case that at least one node among the adjusted multiple nodes has enough resources to execute the task, schedule the task to the node with the lowest load among the at least one node to execute the task;
  • Step S606 in the case that none of the adjusted multiple nodes has enough resources to execute the task, perform task priority scheduling, and at least one task running on multiple nodes has the lowest priority and the highest priority A low-priority task lower than the task is offloaded from the node to obtain a node with sufficient resources for executing the task, and the task is scheduled to the node to execute the task;
  • Step S607 after the task scheduling is completed, monitor the change of the state of the task on multiple nodes running the application program, and automatically perform load balancing, and schedule the new task after receiving the new task;
  • step S608 when a node is abnormal, the task on the node is rescheduled to other nodes according to the task scheduling policy in steps S602 to S605.
  • Figure 7A shows a schematic diagram of the implementation architecture of the task scheduling method provided by the embodiment of the present application.
  • the method can provide the following functions: providing task operation interface 71, task allocation 72, load balancing 73 and task self-healing 74 .
  • the method can be executed by a task scheduling service, and the scheduling service can provide various interfaces for external system calls to perform task operations.
  • multiple dispatching services can be deployed in the mode of one master and multiple slaves.
  • the slave dispatching service 700-2 can automatically switch to the new master dispatching service, so that Ensure high availability of scheduling services.
  • Scheduling service can dispatch the task issued by the external system to a process of the application program corresponding to the type of the task for processing. According to the type of the issued task, different types of tasks can be dispatched to different application programs ( Figure exemplarily shows execution on application programs 710-1, 710-2, 710-N).
  • This function can provide interfaces such as task delivery, deletion, and query for external system calls, and realize corresponding task operations.
  • Task allocation This function can normally allocate tasks to the nodes running the application program corresponding to the task.
  • the resources occupied when processing video streams with different resolutions are different.
  • the occupation of video processing tasks for processing video streams with different resolutions is calibrated.
  • the resource of is represented by the resource unit occupied by the node.
  • the same application program can start a node on different computer devices, and the number of resource units on the computer devices with different hardware resources is also different. The number of resource units on different hardware types of computer devices can be pre-marked.
  • the resources required to perform video processing tasks and the hardware resources of nodes are uniformly measured by resource units, so that video processing tasks of different task types and video stream resolutions can be implemented between nodes with different hardware types unified scheduling.
  • Fig. 7B shows a schematic flow chart of a task allocation method provided in the embodiment of the present application, which can be used to implement the task allocation function in the task scheduling method provided in the embodiment of the present application, and the method includes the following steps:
  • Step S701 receiving a video processing task issued by an external system
  • Step S702 analyzing the video processing task to obtain the number of resource units required to execute the video processing task and the task type of the video processing task;
  • Step S703 judging whether there is a node running an application matching the task type
  • Step S704 if there is no node running the application program, report an error and return an error message indicating that there is no node;
  • Step S705 if there is a node running the application program, determine whether each node is an available idle node in order of all available nodes running the application program in the cluster in the order of load from low to high; In the case of an available idle node, the video processing task is scheduled to be executed on the available idle node;
  • the number of idle resource units in the available idle nodes is not less than the number of resource units required to execute the video processing task, so that the available idle nodes have a sufficient number of idle resource units for executing the video processing task.
  • Step S706 in the case that there is no available idle node in each available node, adjust the tasks occupying different numbers of resource units on each available node, and if there is an available idle node in each available node after adjustment, set The video processing task is scheduled to be executed on the available idle node;
  • the tasks occupying different numbers of resource units on each available node are adjusted, that is, the idle resource units on each available node are sorted out. For example, node 1 has 1 free resource unit left, node 2 has 1 free resource unit left, and 2 resource units are needed to perform video processing tasks, then the task occupying 1 resource unit on node 1 can be moved to node 2 , so that node 1 has 2 idle resource units left, and node 2 has no idle resource units, so the video processing task can be scheduled to node 1 for execution.
  • Step S707 in the case that there is no available idle node among the adjusted available nodes, execute a priority scheduling process to obtain available idle nodes;
  • the priority scheduling process may include: obtaining at least one low-priority task whose priority is lower than the video processing task among the tasks running on each available node; level tasks to get available idle nodes.
  • the priority scheduling process it may be necessary to adjust the tasks running on each available node. For example, the lowest priority low-priority task 1 is running on node 1, and the low-priority task 1 occupies 1 resource unit, and node 2 has 1 idle resource unit, and it needs to occupy 1 resource unit to execute the newly issued video processing task 2 resource units, the low-priority task 1 running on node 1 can be offloaded, and task 2, which occupies 1 resource unit on node 1, can be moved to node 2 for execution. In this way, there are 2 idle resources left on node 1 The unit can schedule the video processing task to be executed on node 1.
  • Step S708 if there is no low-priority task with a priority lower than the video processing task among the tasks running on each available node, report an error and return an error message indicating that there is no resource.
  • the load balancing function can be used to adjust the load of each node, effectively reducing the situation that the node is fully loaded, so that the tasks on each node can run more smoothly.
  • the load balancing function may be triggered based on a preset trigger condition.
  • Trigger conditions can be determined according to actual business scenarios. For business scenarios with high task stability requirements or high task rebalancing costs, stricter load balancing trigger conditions can be set.
  • the trigger conditions may include at least the following two types: one is to trigger load balancing at regular intervals, and the other is to trigger load balancing by adding or deleting tasks.
  • the scheduling service can be triggered to realize the load balancing function.
  • the load balancing function can be realized through the following steps:
  • Step S711 traversing all the application programs running in the cluster in turn, and sorting the nodes running the application program in the cluster according to the load from high to low for the currently accessed application program;
  • Step S712 judging whether the maximum load among the loads of the nodes running the application program exceeds a preset load water level, and whether the minimum load is lower than the load water level;
  • the load water level may be the aforementioned load threshold;
  • Step S713 when the maximum load does not exceed the load water level or the minimum load does not fall below the load water level, access the next application program;
  • Step S714 when the maximum load exceeds the load water level and the minimum load is lower than the load water level, move a task on the node with the highest load to the node with the lowest load for execution, and return to step S712.
  • the load water level is a preset value, which can represent the sensitivity of triggering load balancing. When the water level is set higher, less load balancing adjustments can be performed, but the current For the stability of running tasks, when the water level is set to be low, load balancing adjustments can be performed more frequently to better ensure that the load among nodes is more balanced. In some embodiments, the load water level can be set to 0.8.
  • Task self-healing This function provides a task self-healing mechanism. When a node is abnormal, the tasks running on the abnormal node can be assigned to other normal nodes for execution.
  • the task self-healing function can realize the self-healing of abnormal tasks caused by single machine failure, node abnormality, etc.
  • the tasks on the abnormal node can be rescheduled to other nodes for execution.
  • at least one task on the abnormal node may be sorted from high to low in priority, and the at least one task may be assigned to other nodes for execution through the task assignment function in sequence.
  • the task scheduling method provided by the embodiment of the present application has at least the following improvements:
  • the same video processing system can only process video streams with a specific resolution, and video streams with different resolutions need to be converted into the same specific resolution before processing, and it is impossible to have different hardware in the cluster of the video processing system
  • the video processing tasks are uniformly scheduled on the resource nodes, and in the embodiment of the present application, the hardware resources required to execute the video processing tasks and the hardware resources provided by the nodes are uniformly measured by resource units, so that nodes with different hardware types can Realize unified scheduling of video processing tasks of different task types and video stream resolutions;
  • the embodiment of the present application can allocate multiple idle hardware resources distributed on different nodes Resource units are concentrated on one node, so that hardware resources can be more fully utilized;
  • load balancing adjustment can be performed based on the preset load water level, and the task can be realized. self-healing mechanism.
  • the task scheduling method provided by the embodiment of the present application has at least the following beneficial effects: 1) The hardware resources required to execute the video processing tasks and the hardware resources provided by the nodes are uniformly measured in resource units, and before the application program is released, the processing The resources occupied by video processing tasks of video streams with different resolutions are calibrated when running on different hardware types of computer equipment, so that video processing of different task types and video stream resolutions can be realized between nodes with different hardware types Unified scheduling of tasks; 2) It realizes the functions of task allocation, centralized resource arrangement, task self-healing, and load balancing, which can meet the basic scene requirements of video processing task scheduling, and can improve the overall performance of video processing task execution, thus supporting scheduling Processing tasks of video streams of 10,000-channel level and above.
  • Fig. 8 is a schematic diagram of the composition and structure of a task scheduling device provided by the embodiment of the present application.
  • the task scheduling device 800 includes: a first acquiring part 810, a first determining part 820, a second A scheduling part 840, wherein:
  • the first acquiring part 810 is configured to acquire the task type of the video processing task to be executed and the resolution of the video stream to be processed in the video processing task;
  • the first determining part 820 is configured to determine the number of resource units required to execute the video processing task based on the task type and the resolution;
  • the second determining part 830 is configured to determine, based on the task type, usage information of resource units on at least one node in the cluster that executes the video processing task; wherein, the number of resource units on each node is based on the determined by the hardware type of the above nodes;
  • the first scheduling part 840 is configured to schedule a target node from the at least one node to execute the video processing task based on the number of resource units required to execute the video processing task and the resource unit usage information of each node. Processing tasks; wherein, the number of idle resource units in the target node is not less than the number of resource units required to execute the video processing task.
  • the cluster is configured to execute video processing tasks of at least two task types, each video processing task of the task type is processed on at least one node; the second determining part is further configured to : Based on the task type, determine an application program for processing the video processing task of the task type; determine at least one node running the application program from the cluster; determine resource units on each node in the at least one node usage information.
  • the first determination part is further configured to: based on the task type and the resolution, query the preset correspondence between the task type, the resolution and the number of resource units, and determine the execution The number of resource units required by the video processing task.
  • the resource unit usage information of each node includes the number of idle resource units in the node
  • the first scheduling part is further configured to: based on the The number of resource units and the number of idle resource units in each of the nodes, determining at least one available idle node from the at least one node; determining a target node from at least one of the available idle nodes; scheduling the target node The video processing task is performed.
  • the number of the at least one available idle node is at least two; the first scheduling part is further configured to: use load balancing to determine the target idle node from the at least two available idle nodes node.
  • the first scheduling part is further configured to: determine the at least one Whether there is an available idle node in the nodes; if it is determined that there is no available idle node in the at least one node, perform task scheduling adjustment on the at least one node to obtain at least one available idle node.
  • the first scheduling part is further configured to: determine at least one first node from the at least one node based on the number of resource units required by the video processing task; Unloading at least one running task from the first node; using the at least one first node after the task is offloaded as the at least one available idle node.
  • the first scheduling part is further configured to load the offloaded at least one running task onto a second node of the at least one node.
  • the first scheduling part is further configured to: determine at least one low-priority task whose priority is lower than the video processing task from the tasks already running on the at least one node; based on the The number of resource units required by video processing tasks and the number of resource units occupied by each of the low-priority tasks, determining at least one task to be unloaded with the lowest priority from the at least one low-priority task; The node where the task to be offloaded is determined to be a first node; and the at least one task to be offloaded is offloaded from the at least one first node.
  • the apparatus further includes: a balancing part configured to load the at least one node based on a preset load threshold of each node when a preset load balancing trigger condition is met.
  • Balance adjustment the preset load balancing trigger condition includes at least one of the following: preset timing trigger condition; task addition and deletion frequency reaches a preset frequency threshold.
  • the load threshold includes a first load threshold and a second load threshold, the first load threshold is not lower than the second load threshold, and the balancing part is further configured to: when determining the If there is a third node whose load is higher than the first load threshold in at least one node and a fourth node whose load is lower than the second load threshold, assign a preset number of tasks to the third node Move to the fourth node.
  • the apparatus further includes: a second acquisition part configured to acquire tasks to be rescheduled running on the abnormal node when detecting that there is an abnormal node in the at least one node; A second scheduling section configured to exclude the abnormal node from the at least one node based on the number of resource units required by the task to be rescheduled and usage information of resource units of each of the nodes Among other nodes, the scheduling target node executes the task to be rescheduled.
  • the number of tasks to be rescheduled is multiple
  • the second scheduling part is further configured to: sequentially target the multiple tasks to be rescheduled in order of priority from high to low For each task to be rescheduled, based on the number of resource units required by the task to be rescheduled and the resource unit usage information of each node, schedule the target node from the other nodes to execute the to-be-rescheduled task Rescheduled tasks.
  • a "part" may be a part of a circuit, a part of a processor, a part of a program or software, etc., of course it may also be a unit, a module or a non-modular one.
  • the above task scheduling method is implemented in the form of software function modules and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
  • the software product is stored in a storage medium, and includes several instructions to make a
  • a computer device which may be a personal computer, a server, or a network device, etc.
  • the aforementioned storage medium includes: various media that can store program codes such as U disk, mobile hard disk, read-only memory (Read Only Memory, ROM), magnetic disk or optical disk.
  • embodiments of the present application are not limited to any specific combination of hardware and software.
  • An embodiment of the present application provides a task scheduling device, including a memory and a processor, the memory stores a computer program that can run on the processor, and the processor implements the steps in the above method when executing the program.
  • An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method are implemented.
  • An embodiment of the present application provides a computer program product.
  • the computer program product includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, part or all of the above methods are implemented. All steps.
  • the computer program product can be specifically realized by means of hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in other embodiments, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.
  • An embodiment of the present application provides a computer program, including computer readable codes.
  • a processor in the computer device executes some or all of the steps in the above method.
  • FIG. 9 is a schematic diagram of a hardware entity of the task scheduling device in the embodiment of the present application.
  • the hardware entity of the task scheduling device 900 includes: a processor 901, a communication interface 902, and a memory 903.
  • the processor 901 generally controls the overall operation of the computer device 900 .
  • the communication interface 902 enables the computer device to communicate with other terminals or servers through the network.
  • the memory 903 is configured to store instructions and applications executable by the processor 901, and can also cache data to be processed or processed by the processor 901 and various modules in the computer device 900 (for example, image data, audio data, voice communication data and Video communication data) can be realized by flash memory (FLASH) or random access memory (Random Access Memory, RAM). Data transmission may be performed between the processor 901 , the communication interface 902 and the memory 903 through the bus 904 .
  • FLASH flash memory
  • RAM Random Access Memory
  • the disclosed devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division.
  • the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms of.
  • the units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed to multiple network units; Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be used as a single unit, or two or more units can be integrated into one unit; the above-mentioned integration
  • the unit can be realized in the form of hardware or in the form of hardware plus software functional unit.
  • the above-mentioned integrated units of the present application are realized in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium.
  • the computer software product is stored in a storage medium and includes several instructions to make a A computer device (which may be a personal computer, a server, or a network device, etc.) executes all or part of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes various media capable of storing program codes such as removable storage devices, ROMs, magnetic disks or optical disks.
  • An embodiment of the present application provides a task scheduling method, device, device, storage medium, computer program product, and computer program.
  • the method includes: based on the task type of the video processing task and the resolution of the video stream in the video processing task, determine the execution The number of resource units required by the video processing task; determine the use information of the resource units on at least one node in the cluster performing the video processing task based on the task type, and the number of resource units on each node is determined based on the hardware type of the node; The number of resource units required to execute video processing tasks and the usage information of each node's resource units, at least one node is scheduled to execute the target node to execute the video processing task, and the number of idle resource units in the target node is not less than the execution video The number of resource units required to process the task.
  • unified scheduling of video processing tasks of different task types and video stream resolutions can be implemented among nodes of different hardware types.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请实施例公开了一种任务调度方法、装置、设备、存储介质及计算机程序产品、计算机程序，该方法包括：基于视频处理任务的任务类型和视频处理任务中视频流的分辨率，确定执行视频处理任务所需的资源单元的数量；基于任务类型确定执行视频处理任务的集群中至少一个节点上资源单元的使用信息，每一节点上资源单元的数量是基于节点的硬件类型确定的；基于执行视频处理任务所需的资源单元的数量和每一节点的资源单元的使用信息，从至少一个节点中调度目标节点执行视频处理任务，目标节点中空闲的资源单元的数量不少于执行视频处理任务所需的资源单元的数量。

Description

任务调度方法、装置、设备、存储介质及计算机程序产品、计算机程序
相关申请的交叉引用
本申请基于申请号为202110997563.2、申请日为2021年08月27日、申请名称为“任务调度方法、装置、设备、存储介质及计算机程序产品”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本申请实施例涉及但不限于计算机技术领域，尤其涉及一种任务调度方法、装置、设备、存储介质及计算机程序产品、计算机程序。
背景技术
随着人工智能的兴起，海量的视频处理任务逐渐产生，对视频处理系统处理视频流的路数要求也越来越高，一个视频处理系统少则需要处理千路级别的视频流，多则需要处理万路乃至十万路级别的视频流。
相关技术中，视频处理系统可以采用集群的方式实现，可以将接收的视频处理任务调度到集群中不同的节点上执行。不同视频处理任务中处理的视频流的分辨率是各种各样的，常见的分辨率有1080P(即1920*1080)、2K(即2560*1440)、4K(即4096*2048)等；各种各样的视频处理系统提出多种分辨率需求，但相关技术中同一视频处理系统仅能处理特定分辨率的视频流，对于不同分辨率的视频流需要转换成同一特定的分辨率后才能处理。此外，视频处理系统的集群中各节点可能具有不同的硬件资源；每个节点由于使用不同性能的显卡(即Graphics Processing Unit，GPU)、中央处理器(Central Processing Unit，CPU)、内存等，对视频流的处理能力各不相同，相关技术中无法在视频处理系统的集群中具有不同硬件资源的节点上对视频处理任务进行统一调度。
发明内容
有鉴于此，本申请实施例提供一种任务调度方法、装置、设备、存储介质及计算机程序产品、计算机程序。
本申请实施例的技术方案是这样实现的：
本申请实施例提供一种任务调度方法，所述方法包括：
获取待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率；
基于所述任务类型和所述分辨率，确定执行所述视频处理任务所需的资源单元的数量；
基于所述任务类型，确定执行所述视频处理任务的集群中至少一个节点上资源单元的使用信息；其中，每一所述节点上资源单元的数量是基于所述节点的硬件类型确定的；
基于执行所述视频处理任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中调度目标节点执行所述视频处理任务；其中，所述目标节点中空闲的资源单元的数量不少于执行所述视频处理任务所需的资源单元的数量。
本申请实施例提供一种任务调度装置，所述装置包括：
第一获取部分，被配置为获取待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率；
第一确定部分，被配置为基于所述任务类型和所述分辨率，确定执行所述视频处理 任务所需的资源单元的数量；
第二确定部分，被配置为基于所述任务类型，确定执行所述视频处理任务的集群中至少一个节点上资源单元的使用信息；其中，每一所述节点上资源单元的数量是基于所述节点的硬件类型确定的；
第一调度部分，被配置为基于执行所述视频处理任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中调度目标节点执行所述视频处理任务；其中，所述目标节点中空闲的资源单元的数量不少于执行所述视频处理任务所需的资源单元的数量。
本申请实施例提供一种任务调度设备，包括存储器和处理器，所述存储器存储有可在处理器上运行的计算机程序，所述处理器执行所述程序时实现上述方法中的部分或全部步骤。
本申请实施例提供一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述方法中的部分或全部步骤。
本申请实施例提供一种计算机程序，包括计算机可读代码，在所述计算机可读代码在计算机设备中运行的情况下，所述计算机设备中的处理器执行上述方法中的部分或全部步骤。
本申请实施例提供一种计算机程序产品，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序被计算机读取并执行时，实现上述方法中的部分或全部步骤。
本申请实施例中，获取待执行的视频处理任务的任务类型以及视频处理任务中待处理的视频流的分辨率；基于任务类型和分辨率，确定执行视频处理任务所需的资源单元的数量，并且基于任务类型确定执行视频处理任务的集群中至少一个节点上资源单元的使用信息，其中，每一节点上资源单元的数量是基于节点的硬件类型确定的；基于执行视频处理任务所需的资源单元的数量和每一节点的资源单元的使用信息，从至少一个节点中调度目标节点，以执行视频处理任务，其中，目标节点中空闲的资源单元的数量不少于视频处理任务所需的资源单元的数量。这样，通过将执行视频处理任务所需的硬件资源和节点提供的硬件资源统一通过资源单元来度量，可以在具有不同硬件类型的节点之间实现不同任务类型及视频流分辨率的视频处理任务的统一调度。
附图说明
为了更清楚地说明本申请一个或多个实施例或相关技术中的技术方案，下面将对本申请实施例或相关技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请一个或多个实施例中记载的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的一种任务调度方法的实现流程示意图；
图2为本申请实施例提供的一种任务调度方法的实现流程示意图；
图3为本申请实施例提供的一种任务调度方法的实现流程示意图；
图4为本申请实施例提供的一种任务调度方法的实现流程示意图；
图5为本申请实施例提供的一种任务调度方法的实现流程示意图；
图6为本申请实施例提供的一种任务调度方法的实现流程示意图；
图7A为本申请实施例提供的一种任务调度方法的实现架构示意图；
图7B为本申请实施例提供的一种任务分配方法的实现流程示意图；
图8为本申请实施例提供的一种任务调度装置的组成结构示意图；
图9为本申请实施例提供的一种任务调度设备的硬件实体示意图。
具体实施方式
为了使本申请的目的、技术方案和优点更加清楚，下面结合附图和实施例对本申请的技术方案进一步详细阐述，所描述的实施例不应视为对本申请的限制，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。
在以下的描述中，涉及到“一些实施例”，其描述了所有可能实施例的子集，但是可以理解，“一些实施例”可以是所有可能实施例的相同子集或不同子集，并且可以在不冲突的情况下相互结合。在以下的描述中，所涉及的术语“第一/第二/第三”仅仅是区别类似的对象，不代表针对对象的特定排序，可以理解地，“第一/第二/第三”在允许的情况下可以互换特定的顺序或先后次序，以使这里描述的本申请实施例能够以除了在这里图示或描述的以外的顺序实施。
除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中所使用的术语只是为了描述本申请的目的，不是旨在限制本申请。
对本申请实施例进行进一步详细说明之前，对本申请实施例中涉及的名词和术语进行说明，本申请实施例中涉及的名词和术语适用于如下的解释。
1)视频解析：视频解析指的是采用目标检测、特征提取、对象识别、深度学习等分析手段，将视频内容组织成可被计算机和/或人识别、理解、检索的信息。
2)算法应用元：在实际应用中，可以根据不同的业务目标定义不同类型的任务。对于不同类型的任务，可以通过不同的应用程序采用不同的算法进行处理，得到相应的处理结果。处理某一类型的任务的应用程序即为一个算法应用元。算法应用元处理的任务可以包括但不限于视频解析任务、音频解析任务、图像处理任务、通用人工智能(Artificial general intelligence，AGI)任务等。算法应用元可以是运行在集群节点上的应用程序，同一个集群可以包括运行不同算法应用元的至少两类节点，通过集群的任务调度服务可以将不同类型的任务调度至处理相应类型的任务的节点上，以采用相应的算法应用元进行任务处理。
本申请实施例提供一种任务调度方法，该方法可以由运行在计算机设备上的任务调度服务执行。其中，计算机设备指的可以是服务器、笔记本电脑、平板电脑、台式计算机、智能电视、机顶盒、移动设备(例如移动电话、便携式视频播放器、个人数字助理、专用消息设备、便携式游戏设备)等具备数据处理能力的设备。图1为本申请实施例提供的一种任务调度方法的实现流程示意图，如图1所示，该方法包括：
步骤S101，获取待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率；
这里，视频处理任务可以是任意合适的对视频流进行处理的任务，例如对视频流进行解析、加密、智能分析、目标检测、音频分析、通用人工智能处理等处理的任务。待执行的视频处理任务可以是任务调度服务从外部系统主动获取的，也可以是外部系统向调度服务下发的，还可以是调度服务基于预设的任务配置信息自动生成的，这里并不限定。例如，外部系统可以将待执行的视频处理任务写入消息队列，调度服务可以按照特定的消息消费策略从消息队列中读取待执行的视频处理任务；调度服务也可以提供一个用于下发任务的接口，外部系统可以通过调用该接口将待执行的视频处理任务下发给调度服务。
视频处理任务的任务类型可以包括但不限于对视频流的处理类型(如视频解析、加密、智能分析、目标检测、异常检测等类型)、处理视频流采用的算法类型等。在实施时，本领域技术人员可以根据实际情况采用合适的方式对视频处理任务进行分类。
在一些实施方式中，可以基于待执行的视频处理任务的配置信息确定该视频处理任务的任务类型以及该视频处理任务中待处理的视频流的分辨率。在另一些实施方式中，可以对待执行的视频处理任务的任务内容进行解析，确定该视频处理任务的任务类型，可以对该视频处理任务中待处理的视频流进行分辨率检测，得到该视频流的分辨率。
步骤S102，基于所述任务类型和所述分辨率，确定执行所述视频处理任务所需的资源单元的数量。
这里，资源单元为预先自定义的用于度量执行视频处理任务所需要的硬件资源的基本单位，是对特定的硬件资源所能提供的任务处理能力的抽象定义。一个资源单元可以对应一定的任务处理能力，进而可以对应一定的硬件资源，例如，一个资源单元可以对应特定的运行内存、特定的CPU核的数量和/或特定的显卡数量等能提供的任务处理能力，因而可以将该特定的运行内存、特定的CPU核的数量和/或特定的显卡数量等定义为一个资源单元。在实施时，本领域技术人员可以根据实际情况采用合适的方式定义资源单元，这里并不限定。例如，可以根据专家经验预先定义一个资源单元对应的任务处理能力，进而确定一个资源单元对应的硬件资源，也可以根据历史执行的视频处理任务的任务类型的分布情况及视频流的分辨率分布情况抽象出合适的用于度量执行视频处理任务所需要的硬件资源的基本单位，也即资源单元。
执行不同任务类型的视频处理任务所需的硬件资源是不同的，且对不同分辨率的视频流进行处理所需的硬件资源也是不同的，因而基于待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率，可以确定执行视频处理任务所需的硬件资源。由于采用资源单元来度量执行视频处理任务所需要的硬件资源，因而基于该任务类型和该分辨率，可以确定执行该视频处理任务所需的资源单元的数量。例如，视频处理任务中待处理的视频流的分辨率不同，执行该视频处理任务所需的计算工作量就会不同，也即所需的算力资源也会不同，因而可以基于视频处理任务中待处理的视频流的分辨率，确定执行视频处理任务所需的资源单元的数量。再如，视频处理任务的任务类型不同，执行该视频处理任务所需的计算工作量和存储空间就会不同，因而可以基于视频处理任务中待处理的视频流的帧数，确定执行视频处理任务所需的资源单元的数量。
在实施时，可以基于待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率对视频处理任务进行分析，预估执行视频处理任务所需的硬件资源，进而确定执行所述视频处理任务所需的资源单元的数量，比如可以基于视频处理任务中待处理的视频流的分辨率和任务类型确定视频处理任务所需的CPU核的数量、GPU的数量、内存空间，并基于视频处理任务所需的CPU核的数量、GPU的数量、内存空间和单个资源单元具有的CPU核的数量、GPU的数量、内存空间，确定视频处理任务所需的资源单元的数量；也可以预先对不同任务类型及不同视频流的分辨率的视频处理任务在计算机设备中运行时所占用的硬件资源进行检测，标定出不同任务类型及不同视频流分辨率的视频处理任务所需的资源单元的数量，进而得到视频处理任务的任务类型及视频流的分辨率与资源单元的数量之间的对应关系，比如可以预先对处理不同分辨率的视频流和/或不同任务类型的多个视频处理任务在计算机设备中运行时分别所占用的硬件资源进行检测，标定出处理不同分辨率的视频流和/或不同任务类型的视频处理任务所需的资源单元的数量，进而得到分辨率、任务类型与资源单元的数量之间的对应关系。
步骤S103，基于所述任务类型，确定执行所述视频处理任务的集群中至少一个节点上资源单元的使用信息；其中，每一所述节点上资源单元的数量是基于所述节点的硬件类型确定的。
这里，执行视频处理任务的集群中可以包括至少一个节点，每个节点可以是一个服务器或者一个终端。根据预先定义的资源单元，可以将每一节点上具有的硬件资源抽象 为特定数量的资源单元。每一节点上具有的资源单元的数量可以基于节点的硬件类型确定。不同硬件类型的节点具有的资源单元的数量可能会不同，例如，由于不同厂商生产的显卡或者不同规格的显卡通常具有不同的性能，因而能够提供的硬件资源是不同的，从而采用不同厂商生产的或者不同规格的显卡的节点会具有不同的资源单元的数量。节点的硬件类型可以包括但不限于节点的显卡型号、显卡规格、处理器类型、处理器型号、内存类型、内存容量、磁盘类型、磁盘容量等中的一种或多种。在实施时，可以根据节点的硬件类型确定节点具有的硬件资源总量，利用资源单元对节点具有的硬件资源总量进行度量，可以得到节点上资源单元的数量；也可以预先对不同硬件类型的节点具有的硬件资源进行分析，标定出不同硬件类型的节点具有的资源单元的数量，进而得到硬件类型与资源单元的数量之间的对应关系，利用该对应关系可以基于节点的硬件类型确定节点上资源单元的数量。
节点上资源单元的使用信息可以包括但不限于节点上已占用的资源单元的数量、空闲的资源单元的数量、占用资源单元的任务、每一任务占用的资源单元的数量等中的一种或多种。在实施时，可以通过监测每一节点上各任务的运行状态和各节点具有的资源单元的数量，来确定各节点上资源单元的使用信息，也可以由各节点各自统计自身的资源单元的使用信息，并将统计的资源单元的使用信息上报给任务调度服务。
执行视频处理任务的集群中可以包括至少一类节点，每一类节点用于处理不同任务类型的视频处理任务。基于待执行的视频处理任务的任务类型，可以确定该集群中处理该任务类型的视频处理任务的至少一个节点上的资源单元的使用信息。在一些实施方式中，集群中的全部节点均用于执行该任务类型的视频处理任务，可以获取该集群中全部节点上的资源单元的使用信息。在一些实施方式中，可以预先确定任务类型与节点之间的对应关系，基于该对应关系，可以确定处理不同任务类型的视频处理任务的节点。在一些实施方式中，不同任务类型的视频处理任务需要由不同的应用程序进行处理，可以预先确定任务类型与应用程序之间的对应关系，基于该对应关系，可以确定处理不同任务类型的视频处理任务的应用程序，从而根据不同节点上运行的应用程序，确定处理不同任务类型的视频处理任务的节点。
步骤S104，基于执行所述视频处理任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中调度目标节点执行所述视频处理任务；其中，所述目标节点中空闲的资源单元的数量不少于执行所述视频处理任务所需的资源单元的数量。
这里，可以基于执行视频处理任务所需的资源单元的数量和每一节点的资源单元的使用信息，确定至少一个节点中空闲的资源单元的数量不少于视频处理任务所需的资源单元的数量的目标节点，并将视频处理任务调度至该目标节点，以执行该视频处理任务。在实施时，调度的目标节点上空闲的资源单元可以是未被占用的资源单元，也可以是对至少一个节点上运行的任务进行移动或卸载之后释放出来的资源单元，这里并不限定。
需要说明的是，在至少一个节点中存在多个节点中空闲的资源单元的数量不少于执行所述视频处理任务所需的资源单元的数量的情况下，可以根据实际情况采用合适的负载均衡策略从该多个节点中确定目标节点。
在一些实施例中，上述步骤S102可以包括：基于所述任务类型和所述分辨率，查询预设的任务类型、分辨率与资源单元的数量之间的对应关系，确定执行所述视频处理任务所需的资源单元的数量。这里，预设的分辨率与资源单元的数量之间的对应关系可以是预先针对处理不同分辨率的视频流和任务类型的多个视频处理任务在计算机设备中运行时所占用的硬件资源进行标定得到的。视频处理任务中待处理的视频流的分辨率越高，执行该视频处理任务所需要的资源单元的数量越高，例如，在任务类型相同的情 况下，处理分辨率分别为1080P、2K、4K的视频流的视频处理任务所需的资源单元的数量会依次增多。
本申请实施例中，首先，获取待执行的视频处理任务的任务类型以及视频处理任务中待处理的视频流的分辨率；然后，基于任务类型和分辨率，确定执行视频处理任务所需的资源单元的数量，并且基于任务类型确定执行视频处理任务的集群中至少一个节点上资源单元的使用信息，其中，每一节点上资源单元的数量是基于节点的硬件类型确定的；最后，基于执行视频处理任务所需的资源单元的数量和每一节点的资源单元的使用信息，从至少一个节点中调度目标节点，以执行视频处理任务，其中，目标节点中空闲的资源单元的数量不少于视频处理任务所需的资源单元的数量。这样，通过将执行视频处理任务所需的硬件资源和节点提供的硬件资源统一通过资源单元来度量，可以在具有不同硬件类型的节点之间实现不同任务类型及视频流分辨率的视频处理任务的统一调度。
本申请实施例提供一种任务调度方法，该方法可以由运行在计算机设备上的任务调度服务执行。如图2所示，该方法包括：
步骤S201，获取待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率。
步骤S202，基于所述任务类型和所述分辨率，确定执行所述视频处理任务所需的资源单元的数量。
这里，步骤S201至步骤S202分别对应于前述步骤S101至步骤S102，在实施时可以参照前述步骤S101至步骤S102的具体实施方式。
步骤S203，基于所述任务类型，确定处理所述任务类型的视频处理任务的应用程序。
这里，不同任务类型的视频处理任务需要由不同的应用程序进行处理。在实施时，可以预先确定任务类型与应用程序之间的对应关系，基于该对应关系，可以确定处理不同任务类型的视频处理任务的应用程序。在实施时，应用程序可以是节点上运行的一个进程或线程等，这里并不限定。
步骤S204，从集群中确定运行所述应用程序的至少一个节点，其中，每一所述节点上资源单元的数量是基于所述节点的硬件类型确定的。
这里，所述集群用于执行至少两种任务类型的视频处理任务，每一所述任务类型的视频处理任务在至少一个节点上处理。
可以在集群中的至少一个节点上运行同一个应用程序。在实施时，可以根据集群的配置信息或者特定的节点与应用程序之间的对应关系，确定集群中运行该应用程序的至少一个节点。
步骤S205，确定所述至少一个节点中每一节点上资源单元的使用信息。
这里，节点上资源单元的使用信息的确定可以参照前述步骤S103的具体实施方式。
步骤S206，基于执行所述视频处理任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中调度目标节点执行所述视频处理任务；其中，所述目标节点中空闲的资源单元的数量不少于执行所述视频处理任务所需的资源单元的数量。
这里，步骤S206对应于前述步骤S104，在实施时可以参照前述步骤S104的具体实施方式。
本申请实施例中，集群用于执行至少两种任务类型的视频处理任务，每一任务类型的视频处理任务在至少一个节点上处理，基于视频处理任务的任务类型确定处理该任务类型的视频处理任务的应用程序，从集群中确定运行该应用程序的至少一个节点，并确 定至少一个节点中每一节点上资源单元的使用信息。这样，可以将不同任务类型的视频处理任务调度到对应的应用程序所在的节点，从而可以实现不同任务类型的视频处理任务在同一集群的统一调度。
本申请实施例提供一种任务调度方法，该方法可以由运行在计算机设备上的任务调度服务执行。如图3所示，该方法包括：
步骤S301，获取待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率。
步骤S302，基于所述任务类型和所述分辨率，确定执行所述视频处理任务所需的资源单元的数量。
步骤S303，基于所述任务类型，确定执行所述视频处理任务的集群中至少一个节点上空闲的资源单元的数量；其中，每一所述节点上资源单元的数量是基于所述节点的硬件类型确定的。
这里，步骤S301至步骤S303分别对应于前述步骤S101至步骤S103，在实施时可以参照前述步骤S101至步骤S103的具体实施方式。
步骤S304，基于执行所述视频处理任务所需的资源单元的数量和每一所述节点中空闲的资源单元的数量，从所述至少一个节点中确定至少一个可用空闲节点。
这里，可用空闲节点为空闲的资源单元的数量不少于执行视频处理任务所需的资源单元的数量的节点。在实施时，通过检查至少一个节点中每一节点上空闲的资源单元的数量，并与执行视频处理任务所需的资源单元的数量进行比较，可以确定至少一个可用空闲节点。
在一些实施方式中，在至少一个节点中存在空闲的资源单元的数量不少于执行视频处理任务所需的资源单元的数量的节点的情况下，可以将该节点确定为可用空闲节点。
在一些实施方式中，在至少一个节点中存在空闲的资源单元的数量不少于执行视频处理任务所需的资源单元的数量的节点的情况下，可以通过对至少一个节点中已运行的任务进行卸载或移动，使得至少一个节点中存在空闲的资源单元的数量不少于执行视频处理任务所需的资源单元的数量的节点，并将该节点确定为可用空闲节点。
步骤S305，从至少一个所述可用空闲节点中确定目标节点。
这里，可以根据实际情况采用任意合适的选择策略，从至少一个可用空闲节点中选择至少一个节点作为目标节点，这里并不限定。例如，可以从至少一个可用空闲节点中随机确定至少一个节点作为目标节点，也可以根据每一可用空闲节点当前的负载情况，将负载最低的可用空闲节点确定为目标节点。
步骤S306，调度所述目标节点执行所述视频处理任务。
在一些实施例中，所述至少一个可用空闲节点的数量为至少两个，上述步骤S305可以包括：采用负载均衡的方式，从所述至少两个可用空闲节点中确定目标空闲节点。这里，可以根据实际情况采用合适的负载均衡策略，从至少两个可用空闲节点中确定目标空闲节点，本申请实施例对此并不限定。
在一些实施例中，该方法还可以包括：在确定所述至少一个节点中存在多个可用空闲节点的情况下，将所述多个可用空闲节点中负载最低的节点确定为最终的可用空闲节点。这里，节点的负载可以由节点上正在执行和/或等待执行的任务数量确定，正在执行和/或等待执行的任务数量越多，说明节点的负载越高。节点的负载也可以由节点的资源占用率确定，资源占用率越高，说明节点的负载越高。
在一些实施例中，上述步骤S304可以包括：
步骤S341，基于执行所述视频处理任务所需的资源单元的数量和每一所述节点中空闲的资源单元的数量，确定所述至少一个节点中是否存在可用空闲节点；
这里，可以分别比较每一节点中空闲的资源单元的数量是否不少于执行视频处理任务所需的资源单元的数量，从而确定至少一个节点中是否存在可用空闲节点。
步骤S342，在确定所述至少一个节点中不存在可用空闲节点的情况下，对所述至少一个节点进行任务调度调整，得到至少一个可用空闲节点。
这里，对至少一个节点进行任务调度调整可以包括将一个节点上的任务移动到另一个节点上执行，以将各节点上空闲的资源单元集中，也可以包括将至少一个节点上的任务卸载，从而可以至少增加一个节点中空闲的资源单元的数量。例如，节点1上有1个空闲的资源单元，节点2上有1个空闲的资源单元，可以将节点1上占用资源单元为1的任务1移动到节点2上执行，以增加节点1上空闲的资源单元的数量，任务调度调整后，节点1上空闲的资源单元为2个，节点2上无空闲的资源单元。又如，节点1上有1个空闲的资源单元，可以将节点1上占用资源单元为1的任务1卸载，从而节点1上空闲的资源单元增加为2个。
任务调度调整后，可以至少增加一个节点中空闲的资源单元的数量，这样，任务调度调整后的至少一个节点中可能会出现空闲的资源单元的数量不少于执行视频处理任务所需的资源单元的数量的节点，也即可用空闲节点。在确定调整后的至少一个节点中不存在可用空闲节点的情况下，也可以对调整后的至少一个节点进一步进行任务移动或卸载，以得到可用空闲节点，例如，可以将调整后的至少一个节点上当前运行的至少一个任务卸载，以释放所占用的资源单元。
需要说明的是，在相关技术中，同一视频处理系统仅能处理特定分辨率的视频流，且不同节点对视频流的处理能力并不一定相同，从而无法在视频处理系统的集群中具有不同硬件资源的节点上对视频处理任务进行统一调度，进而无法对节点之间的视频处理任务进行统一的任务调度调整。在此基础上，本领域技术人员没有动机想到对分布在不同节点上的空闲的资源能力进行整理集中，以更充分地利用硬件资源，更不会想到对集群中的至少一个节点进行任务调度调整，以至少增加一个节点中空闲的资源单元的数量，得到至少一个可用空闲节点。因此，对于本领域技术人员而言，上述步骤S341至步骤S342是非显而易见的。
在一些实施例中，上述步骤S342中所述的对所述至少一个节点进行任务调度调整，得到至少一个可用空闲节点，可以包括：
步骤S3421，基于所述视频处理任务所需的资源单元的数量，从所述至少一个节点中确定至少一个第一节点；
这里，在确定至少一个节点中不存在可用空闲节点的情况下，可以从至少一个节点上中确定至少一个第一节点，并对第一节点上运行的任务进行调度调整，以从该至少一个第一节点上释放至少一个被占用的资源单元。
在实施时，可以根据实际情况采用合适的方式从至少一个节点中确定至少一个第一节点，这里并不限定。例如，可以将至少一个节点中具有的资源单元的数量大于视频处理任务所需的资源单元的数量的节点确定为第一节点，也可以将至少一个节点中运行的任务的优先级低于预设优先级的节点确定为第一节点，还可以将至少一个节点中空闲的资源单元的数量大于设定的数量阈值的节点确定为第一节点。
步骤S3422，从所述至少一个第一节点上卸载至少一个已运行的任务；
这里，可以根据任务的优先级确定要卸载的任务，也可以随机确定要卸载的任务，还可以根据任务占用的资源单元的数量来确定要卸载的任务，这里并不限定。
卸载任务可以包括暂停或结束执行该任务，在节点上有新的资源单元被释放后还可以再次执行该任务，在其他节点上有足够的空闲的资源单元的情况下，还可以将该任务加载至该其他节点上运行。
步骤S3423，将卸载掉任务后的所述至少一个第一节点作为所述至少一个可用空闲节点。
在一些实施例中，在上述步骤S3422之后，还可以包括：将已卸载的所述至少一个已运行的任务加载到所述至少一个节点中的第二节点上。这里，第二节点为该至少一个节点中与第一节点不同的节点。
在一些实施例中，上述步骤S3421可以包括：
步骤S3421a，从所述至少一个节点上已运行的任务中确定优先级低于所述视频处理任务的至少一个低优先级任务；
步骤S3421b，基于所述视频处理任务所需的资源单元的数量和每一所述低优先级任务占用的资源单元的数量，将所述至少一个低优先级任务中优先级最低的至少一个任务确定为待卸载任务。
这里，可以基于视频处理任务所需的资源单元的数量和每一低优先级任务占用的资源单元的数量，确定至少一个待卸载任务，且确定的至少一个待卸载任务为至少一个低优先级任务中优先级最低的至少一个任务，使得该至少一个待卸载任务被卸载后，至少一个节点中空闲的资源单元的数量不少于该视频处理任务所需的资源单元的数量，或者将任务卸载后的至少一个节点上空闲的资源单元集中后，至少一个节点中空闲的资源单元的数量不少于该视频处理任务所需的资源单元的数量。
上述步骤S3422可以包括：从所述至少一个第一节点上卸载所述至少一个待卸载任务。
在一些实施例中，可以预先基于视频处理任务所需的资源单元的数量和每一低优先级任务占用的资源单元的数量，从至少一个低优先级任务中确定至少一个待卸载任务后，将待卸载任务卸载；也可以按照低优先级任务的优先级从低到高的顺序，依次卸载至少一个低优先级任务，并在每次卸载完一个低优先级任务后，判断至少一个节点中当前是否存在可用空闲节点，在当前存在可用空闲节点的情况下，直接确定可用空闲节点，在当前不存在可用空闲节点的情况下，对任务卸载后的至少一个节点进行任务调度调整，并确定调整后的至少一个节点中是否存在可用空闲节点，在调整后的至少一个节点中存在可用空闲节点的情况下，确定可用空闲节点，在调整后的至少一个节点中不存在可用空闲节点的情况下，继续卸载下一个低优先级任务，直至任务卸载后的至少一个节点中存在可用空闲节点或者对任务卸载后的至少一个节点进行任务调度调整后得到可用空闲节点。
本申请实施例中，基于执行视频处理任务所需的资源单元的数量和每一节点中空闲的资源单元的数量，从至少一个节点中确定至少一个可用空闲节点，从至少一个可用空闲节点中确定目标节点，并将视频处理任务调度至该可用空闲节点。这样，可以简单快速地实现任务调度。此外，在确定至少一个节点中存在多个可用空闲节点的情况下，可以采用负载均衡的方式，从多个可用空闲节点中确定目标节点，从而可以更好地均衡各节点之间的负载，提高视频处理任务执行的整体性能。此外，还可以在确定至少一个节点中不存在可用空闲节点的情况下，对至少一个节点进行任务调度调整，以至少增加一个所述节点中空闲的资源单元的数量，并从任务调度调整后的至少一个节点中确定可用空闲节点，从而可以将分布在不同节点上的多个空闲的资源单元集中至一个节点上，从而可以更充分地利用硬件资源，进一步提高任务执行的整体性能。在一些实施例中，还可以基于视频处理任务所需的资源单元的数量，从至少一个节点上卸载优先级低于当前视频处理任务的至少一个低优先级任务，释放足够的资源单元以执行当前视频处理任务，这样，可以更好地满足较高优先级任务的硬件资源需求。
本申请实施例提供一种任务调度方法，该方法可以由运行在计算机设备上的任务调 度服务执行。如图4所示，该方法包括：
步骤S401，获取待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率。
步骤S402，基于所述任务类型和所述分辨率，确定执行所述视频处理任务所需的资源单元的数量。
步骤S403，基于所述任务类型，确定执行所述视频处理任务的集群中至少一个节点上资源单元的使用信息；其中，每一所述节点上资源单元的数量是基于所述节点的硬件类型确定的。
步骤S404，基于执行所述视频处理任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中调度目标节点执行所述视频处理任务；其中，所述目标节点中空闲的资源单元的数量不少于执行所述视频处理任务所需的资源单元的数量。
这里，步骤S401至步骤S404分别对应于前述步骤S101至步骤S104，在实施时可以参照前述步骤S101至步骤S104的具体实施方式。
步骤S405，在满足预设的负载均衡触发条件的情况下，基于预设的每个节点的负载阈值，对所述至少一个节点进行负载均衡调整。
这里，预设的负载均衡触发条件可以是根据实际业务需求确定的，可以包括但不限于预设的定时触发条件、任务的增删频率达到预设的频率阈值等中的一种或多种。在实施时，对于任务的稳定性要求比较高或者任务重新均衡的代价较高的业务场景，可以设置较为严格的负载均衡触发条件，例如，负载均衡触发条件为预设的定时触发条件的情况下，可以设置较大的定时触发间隔，负载均衡触发条件为任务的增删频率达到预设的频率阈值的情况下，可以设置较高的频率阈值；对于任务的稳定性要求比较低且任务重新均衡的代价较低的业务场景，可以设置较为宽松的负载均衡触发条件，例如，负载均衡触发条件为预设的定时触发条件的情况下，可以设置较小的定时触发间隔，负载均衡触发条件为任务的增删频率达到预设的频率阈值的情况下，可以设置较低的频率阈值。
负载阈值为预先设定的一个值，可以表征触发负载均衡的敏感度，可以为每一节点设置不同的负载阈值，也可以为至少一个节点设置相同的负载阈值，这里并不限定。负载阈值设置的较高的情况下，可以较少地执行负载均衡调整，而是保证当前已运行的任务的稳定。负载阈值设置较低的情况下，可以较频繁地执行负载均衡调整，以更好地保证各节点之间负载较为均衡。预设的负载阈值可以包括最高负载水位线、最低负载水位线中的至少一种，在满足预设的负载均衡触发条件的情况下，可以对至少一个节点进行负载均衡调整，使得各节点之间保持负载均衡。在实施时，可以通过将至少一个节点中负载更高的节点上运行的至少一个任务移动至负载更低的至少一个节点上执行，以使得各节点之间保持负载均衡。例如，可以将至少一个节点中负载超过第一负载阈值的节点上运行的至少一个任务移动至少一个节点中负载最低的至少一个节点上执行，也可以将至少一个节点中负载超过第一负载阈值的节点上运行的至少一个任务移动至负载低于第二负载阈值的至少一个节点上执行，其中，第一负载阈值高于第二负载阈值，还可以将至少一个节点中负载最高的节点上运行的至少一个任务移动至负载低于第二负载阈值的至少一个节点上执行。
在一些实施例中，所述负载阈值包括第一负载阈值和第二负载阈值，所述第一负载阈值不低于所述第二负载阈值，上述步骤S405中所述的基于预设的每个节点的负载阈值，对所述至少一个节点进行负载均衡调整，可以包括：在确定所述至少一个节点中存在负载高于所述第一负载阈值的第三节点，且存在负载低于所述第二负载阈值的第四节点的情况下，将所述第三节点中预设数量的任务移动至所述第四节点上。这里，预设数 量可以是根据实际情况预先设定的。第一负载阈值可以高于或等于第二负载阈值，这里并不限定。在一些实施例中，还可以在将第三节点中预设数量的任务移动至第四节点上之后，检测任务移动后的至少一个节点中是否同时存在第三节点和第四节点，在任务移动后的至少一个节点中同时存在第三节点和第四节点的情况下，对至少一个节点继续进行负载均衡调整，直至调整后的至少一个节点中不存在第三节点或不存在第四节点。
在一些实施例中，所述预设的负载均衡触发条件包括以下至少之一：预设的定时触发条件；任务的增删频率达到预设的频率阈值。这里，定时触发条件和频率阈值均可以是预先根据实际需要设定的。定时触发条件可以包括但不限于定时触发时间、定时触发次数等。频率阈值可以为正整数，频率阈值等于1的情况即为每次增加或删除任务均进行负载均衡调整。
需要说明的是，在相关技术中，无法在视频处理系统的集群中具有不同硬件资源的节点上对视频处理任务进行统一调度，进而无法对节点之间的视频处理任务进行统一的负载均衡调整。在此基础上，本领域技术人员没有动机想到对集群中的不同节点进行负载均衡调整，以提高视频处理任务执行的整体性能，更不会想到在满足预设的负载均衡触发条件的情况下，基于预设的负载阈值，对至少一个节点进行负载均衡调整。因此，对于本领域技术人员而言，上述步骤S401至步骤S405是非显而易见的。
本申请实施例中，在满足预设的负载均衡触发条件的情况下，基于预设的每个节点的负载阈值，对至少一个节点进行负载均衡调整。这样，可以更好地均衡各节点之间的负载，减少节点出现满载的情况，从而进一步提高视频处理任务执行的整体性能。
本申请实施例提供一种任务调度方法，该方法可以由运行在计算机设备上的任务调度服务执行。如图5所示，该方法包括：
步骤S501，获取待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率。
步骤S502，基于所述任务类型和所述分辨率，确定执行所述视频处理任务所需的资源单元的数量。
步骤S503，基于所述任务类型，确定执行所述视频处理任务的集群中至少一个节点上资源单元的使用信息；其中，每一所述节点上资源单元的数量是基于所述节点的硬件类型确定的。
步骤S504，基于执行所述视频处理任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中调度目标节点执行所述视频处理任务；其中，所述目标节点中空闲的资源单元的数量不少于执行所述视频处理任务所需的资源单元的数量。
这里，步骤S501至步骤S504分别对应于前述步骤S101至步骤S104，在实施时可以参照前述步骤S101至步骤S104的具体实施方式。
步骤S505，在检测到所述至少一个节点中存在异常节点的情况下，获取所述异常节点上运行的待重新调度的任务。
这里，异常节点可以包括但不限于发生宕机、夯机、系统崩溃等至少一种异常的节点，异常节点上无法正常运行任务。待重新调度的任务为异常节点上需要执行的任务，可以在检测到至少一个节点中存在异常节点的情况下，获取该异常节点上的待重新调度的任务，并将待重新调度的任务调度至除异常节点之外的其他节点上。在实施时，可以通过日志监控、心跳检测、错误上报等任意合适的方式确定至少一个节点中是否存在异常节点。
步骤S506，基于所述待重新调度的任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中除所述异常节点之外的其他节点中，调度 目标节点执行所述待重新调度的任务。
这里，上述步骤S506对应于前述步骤S104，在实施时可以参照前述步骤S104的具体实施方式。
在一些实施例中，所述待重新调度的任务的数量为多个，上述步骤S506可以包括：按照优先级从高到低的顺序，依次针对多个待重新调度的任务中的每一待重新调度的任务，基于所述待重新调度的任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述其他节点中调度目标节点执行所述待重新调度的任务。
需要说明的是，在相关技术中，无法在视频处理系统的集群中具有不同硬件资源的节点上对视频处理任务进行统一调度。在此基础上，本领域技术人员没有动机想到将异常节点上运行的任务调度到正常节点上执行，以实现节点异常导致的异常任务的自愈。因此，对于本领域技术人员而言，上述步骤S501至步骤S506是非显而易见的。
在本申请实施例中，在检测到至少一个节点中存在异常节点的情况下，获取异常节点上运行的待重新调度的任务，并基于待重新调度的任务所需的资源单元的数量和每一节点的资源单元的使用信息，从至少一个节点中除异常节点之外的其他节点中调度目标节点，以执行待重新调度的任务。这样，在集群中有节点出现异常的情况下，可以保证异常节点上运行的任务可以被调度到正常节点上执行，从而可以实现节点异常导致的异常任务的自愈，进而可以提升集群中任务执行的稳定性。
本申请实施例可以应用于视频处理系统中，包括但不限于智慧安防、智慧城市等场景，用于将用户下发的包含视频流的任务调度到集群中的节点上处理。本申请实施例提供一种任务调度方法，由任务调度服务执行，可以根据节点提供的硬件资源能力，对处理不同分辨率的视频流的视频处理任务进行统一调度，对于不同优先级的任务的调度场景，能够优先调度高优先级的任务，并且具有负载均衡的能力，能将高负载的节点上的任务调度到低负载的节点上执行，此外，对于异常节点上运行的任务具有自愈能力，当集群中一个节点出现异常时，可以将该节点上运行的任务调度到其他节点上去。
参见图6，本申请实施例提供的任务调度方法可以通过如下步骤实现：
步骤S601，接收外部系统通过调用任务下发接口下发的任务；
步骤S602，根据下发的任务，确定与该任务对应的应用程序，并确定集群中运行该应用程序的多个节点；
这里，应用程序和任务的类型是一一对应的，特定类型的任务只能调度到对应的应用程序上执行，可以根据任务的类型确定与该任务对应的应用程序。同一个应用程序可能在不同的计算机设备上启动多个进程，一个进程可以作为一个节点，任务调度的预期是可以将外部系统下发的任务调度到与该任务的类型对应的应用程序的某一个进程上去进行处理。
步骤S603，在确定该多个节点中有至少一个节点上有足够的资源用于执行该任务的情况下，将该任务调度至该至少一个节点中负载最低的节点，以执行该任务；
步骤S604，在确定该多个节点均没有足够的资源用于执行该任务的情况下，调整该多个节点上运行的任务，将各节点上空闲的资源集中至一个节点上；
步骤S605，在调整后的多个节点中有至少一个节点上有足够的资源用于执行该任务的情况下，将该任务调度至该至少一个节点中负载最低的节点，以执行该任务；
步骤S606，在调整后的多个节点中没有节点上有足够的资源用于执行该任务的情况下，进行任务的优先级调度，将多个节点上运行的至少一个任务中优先级最低且优先级低于该任务的低优先级任务从节点上卸载，以得到有足够的资源用于执行该任务的节点，并将该任务调度至该节点，以执行该任务；
步骤S607，在该任务调度完成后，监听运行该应用程序的多个节点上任务的状态 的变更，并自动进行负载均衡，以及在接收到新任务后对新任务进行调度；
步骤S608，在有节点出现异常的情况下，将该节点上的任务重新按照步骤S602至步骤S605中的任务调度策略调度至其他节点上。
图7A示出了本申请实施例提供的任务调度方法的实现架构示意图，如图7A所示，该方法可以提供如下功能：提供任务操作接口71、任务分配72、负载均衡73和任务自愈74。该方法可以由任务调度服务执行，该调度服务可以提供多种接口供外部系统调用来进行任务操作。参见图7A，在实施时，可以按一主多从的模式部署多个调度服务，当主调度服务700-1出现异常时，从调度服务700-2可以自动切换为新的主调度服务，从而可以保证调度服务的高可用性。调度服务可以将外部系统下发的任务调度到与该任务的类型对应的应用程序的一个进程上去进行处理，根据下发的任务的类型，可以将不同类型的任务调度到不同的应用程序(图中示例性示出应用程序710-1、710-2、710-N)上执行。
下面分别对本申请实施例提供的任务调度方法提供的各功能进行说明：
1)提供任务操作接口：该功能可以提供任务下发、删除、查询等接口供外部系统调用，并实现对应的任务操作。
2)任务分配：该功能可以将任务正常分配到运行该任务对应的应用程序的节点上。
在视频处理任务的执行过程中，对不同的分辨率的视频流进行处理时占用的资源不同，本申请实施例提供的任务调度方法中，首先标定处理不同分辨率的视频流的视频处理任务占用的资源，用占用节点的资源单元来表示，视频处理任务中处理的视频流的分辨率越高，执行该任务需要占用的资源单元就越多。同一个应用程序可以在不同的计算机设备上分别启动一个节点，具有不同硬件资源的计算机设备上具有的资源单元的数量也不同。可以预先标定不同硬件类型的计算机设备上具有的资源单元的数量。
在进行任务分配时，通过资源单元来统一度量执行视频处理任务所需的资源以及节点具有的硬件资源，可以在具有不同硬件类型的节点之间实现不同任务类型和视频流分辨率的视频处理任务的统一调度。
图7B示出了本申请实施例提供的一种任务分配方法的实现流程示意图，该方法可以用于实现本申请实施例提供的任务调度方法中的任务分配功能，该方法包括如下步骤：
步骤S701，接收外部系统下发的视频处理任务；
步骤S702，解析该视频处理任务，得到执行该视频处理任务所需的资源单元的数量以及该视频处理任务的任务类型；
步骤S703，判断是否存在运行与该任务类型匹配的应用程序的节点；
步骤S704，在不存在运行该应用程序的节点的情况下，报错并返回表征无节点的错误信息；
步骤S705，在存在运行该应用程序的节点的情况下，将集群中运行该应用程序的全部可用节点，按负载由低到高的顺序，依次确定各节点是否为可用空闲节点；在确定节点为可用空闲节点的情况下，将该视频处理任务调度到该可用空闲节点上执行；
这里，可用空闲节点中空闲的资源单元的数量不少于执行该视频处理任务所需的资源单元的数量，从而可用空闲节点有足够数量的空闲的资源单元用于执行该视频处理任务。
步骤S706，在各可用节点中不存在可用空闲节点的情况下，对各可用节点上占用不同数量的资源单元的任务进行调整，在调整后的各可用节点中存在可用空闲节点的情况下，将该视频处理任务调度到该可用空闲节点上执行；
这里，对各可用节点上占用不同数量的资源单元的任务进行调整，即对各可用节点上空闲的资源单元进行整理。例如，节点1剩余1个空闲的资源单元，节点2剩余1个 空闲的资源单元，执行视频处理任务需要2个资源单元，则可以将节点1上占用1个资源单元的任务移动至节点2上，这样节点1则剩余2个空闲的资源单元，节点2上无空闲的资源单元，可以将该视频处理任务调度到节点1上执行。
步骤S707，在调整后的各可用节点中不存在可用空闲节点的情况下，执行进行优先级调度流程，得到可用空闲节点；
这里，优先级调度流程可以包括：获取各可用节点上运行的任务中优先级低于该视频处理任务的至少一个低优先级任务；将按照优先级从低到高的顺序，卸载至少一个低优先级任务，得到可用空闲节点。在执行优先级调度流程的过程中，可能会需要对各可用节点上运行的任务进行调整。例如，节点1上运行了优先级最低的低优先级任务1，该低优先级任务1占用1个资源单元，节点2有1个空闲的资源单元，而执行新下发的视频处理任务需要占用2个资源单元，则可以卸载节点1上运行的低优先级任务1，同时将节点1上占用1个资源单元的任务2移动到节点2上执行，这样，节点1上剩余2个空闲的资源单元，可以将该视频处理任务调度到节点1上执行。
步骤S708，在各可用节点上运行的任务中没有优先级低于该视频处理任务的低优先级任务的情况下，则报错并返回表征无资源的错误信息。
3)负载均衡：可以对已存在的任务和节点进行负载均衡调整；
随着接收的视频处理任务的数量的增长，在对大量的视频处理任务进行调度的过程中，可能会由于任务的频繁增加以及删除，导致集群中各节点上运行的任务出现不均衡，存在部分节点负载很高，部分节点负载很低的情况。在本申请实施例提供的任务调度方法中，通过负载均衡功能，可以对各节点进行负载均衡调整，有效减少节点出现满载的情况，从而让各节点上的任务运行更流畅。
本申请实施例提供的任务调度方法中，可以基于预设的触发条件来触发负载均衡功能。触发条件可以根据实际的业务场景来定。对于任务的稳定性要求比较高或者任务重新均衡的代价较高的业务场景，可以设置较为严格的负载均衡触发条件。在实施时，触发条件可以至少包括如下两种：一是定时触发负载均衡，二是通过任务的增删触发负载均衡。在满足触发条件的情况下，可以触发调度服务实现负载均衡功能，负载均衡功能可以通过如下步骤实现：
步骤S711，依次遍历集群中运行的所有的应用程序，针对当前访问的应用程序，将集群中运行该应用程序的各节点按负载从高到低排序；
步骤S712，判断运行该应用程序的各节点的负载中最大负载是否超过预设的负载水位线，以及最小负载是否低于该负载水位线；这里，负载水位线可以是前述的负载阈值；
步骤S713，在最大负载未超过该负载水位线或者最小负载未低于该负载水位线的情况下，访问下一个应用程序；
步骤S714，在最大负载超过该负载水位线且最小负载低于该负载水位线的情况下，将最高负载的节点上的一个任务移动至最低负载节点上执行，并回到步骤S712。
需要说明的是，负载水位线为预先设定的一个值，可以表征触发负载均衡的敏感度，在水位线设置的较高的情况下，可以较少地执行负载均衡调整，而是保证当前已运行的任务的稳定，在水位线设置较低的情况下，可以较频繁地执行负载均衡调整，以更好地保证各节点之间负载较为均衡。在一些实施例中，负载水位线可以设置为0.8。
4)任务自愈：该功能提供任务自愈机制，当出现节点异常时，可以将异常节点上运行的任务分配到其他正常节点上执行。
在应用程序运行的过程中，单机故障、节点异常是不可避免的。本申请实施例提供的任务调度方法中，任务自愈功能可以实现单机故障、节点异常等导致的异常任务的自 愈。在出现单机故障或节点异常等问题的情况下，可以将异常节点上的任务重新调度到其他节点上执行。节点出现异常后，可以会将该异常节点上的至少一个任务按优先级从高到低排序后，通过任务分配功能依次将该至少一个任务分配到其他节点上执行。
本申请实施例提供的任务调度方法与相关技术相比，至少存在以下改进点：
1)相关技术中同一视频处理系统仅能处理特定分辨率的视频流，对于不同分辨率的视频流需要转换成同一特定的分辨率后才能处理，且无法在视频处理系统的集群中具有不同硬件资源的节点上对视频处理任务进行统一调度，而本申请实施例通过将执行视频处理任务所需的硬件资源和节点提供的硬件资源统一通过资源单元来度量，可以在具有不同硬件类型的节点之间实现不同任务类型和视频流分辨率的视频处理任务的统一调度；
2)相关技术中对于不同节点上剩余的空闲的硬件资源，无法进行整理集中，导致剩余碎片资源无法被充分利用，浪费算力，而本申请实施例可以将分布在不同节点上的多个空闲的资源单元集中至一个节点上，从而可以更充分地利用硬件资源；
3)相关技术中任务的负载均衡调整的策略比较粗糙，不能同时考虑系统的稳定性和负载均衡程度，而本申请实施例中可以基于预设的负载水位线进行负载均衡调整，且可以实现任务的自愈机制。
本申请实施例提供的任务调度方法，至少具有以下有益效果：1)将执行视频处理任务所需的硬件资源和节点提供的硬件资源统一通过资源单元来度量，通过在应用程序发布前，对处理不同分辨率的视频流的视频处理任务在不同硬件类型的计算机设备上运行时所占的资源进行标定，即可在具有不同硬件类型的节点之间实现不同任务类型和视频流分辨率的视频处理任务的统一调度；2)实现了任务分配、资源整理集中、任务自愈、负载均衡功能，可以满足视频处理任务调度的基本场景需求，并能提高视频处理任务执行的整体性能，从而可以支持调度万路级别及以上的视频流的处理任务。
图8为本申请实施例提供的一种任务调度装置的组成结构示意图，如图8所示，任务调度装置800包括：第一获取部分810、第一确定部分820、第二确定部分830和第一调度部分840，其中：
第一获取部分810，被配置为获取待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率；
第一确定部分820，被配置为基于所述任务类型和所述分辨率，确定执行所述视频处理任务所需的资源单元的数量；
第二确定部分830，被配置为基于所述任务类型，确定执行所述视频处理任务的集群中至少一个节点上资源单元的使用信息；其中，每一所述节点上资源单元的数量是基于所述节点的硬件类型确定的；
第一调度部分840，被配置为基于执行所述视频处理任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中调度目标节点执行所述视频处理任务；其中，所述目标节点中空闲的资源单元的数量不少于执行所述视频处理任务所需的资源单元的数量。
在一些实施例中，所述集群被配置为执行至少两种任务类型的视频处理任务，每一所述任务类型的视频处理任务在至少一个节点上处理；所述第二确定部分还被配置为：基于所述任务类型，确定处理所述任务类型的视频处理任务的应用程序；从所述集群中确定运行所述应用程序的至少一个节点；确定所述至少一个节点中每一节点上资源单元的使用信息。
在一些实施例中，所述第一确定部分还被配置为：基于所述任务类型和所述分辨率，查询预设的任务类型、分辨率与资源单元的数量之间的对应关系，确定执行所述视频处 理任务所需的资源单元的数量。
在一些实施例中，每一所述节点的资源单元的使用信息包括所述节点中空闲的资源单元的数量，所述第一调度部分还被配置为：基于执行所述视频处理任务所需的资源单元的数量和每一所述节点中空闲的资源单元的数量，从所述至少一个节点中确定至少一个可用空闲节点；从至少一个所述可用空闲节点中确定目标节点；调度所述目标节点执行所述视频处理任务。
在一些实施例中，所述至少一个可用空闲节点的数量为至少两个；所述第一调度部分还被配置为：采用负载均衡的方式，从所述至少两个可用空闲节点中确定目标空闲节点。
在一些实施例中，所述第一调度部分还被配置为：基于执行所述视频处理任务所需的资源单元的数量和每一所述节点中空闲的资源单元的数量，确定所述至少一个节点中是否存在可用空闲节点；在确定所述至少一个节点中不存在可用空闲节点的情况下，对所述至少一个节点进行任务调度调整，得到至少一个可用空闲节点。
在一些实施例中，所述第一调度部分还被配置为：基于所述视频处理任务所需的资源单元的数量，从所述至少一个节点中确定至少一个第一节点；从所述至少一个第一节点上卸载至少一个已运行的任务；将卸载掉任务后的所述至少一个第一节点作为所述至少一个可用空闲节点。
在一些实施例中，所述第一调度部分还被配置为：将已卸载的所述至少一个已运行的任务加载到所述至少一个节点中的第二节点上。
在一些实施例中，所述第一调度部分还被配置为：从所述至少一个节点上已运行的任务中确定优先级低于所述视频处理任务的至少一个低优先级任务；基于所述视频处理任务所需的资源单元的数量和每一所述低优先级任务占用的资源单元的数量，从所述至少一个低优先级任务中确定优先级最低的至少一个待卸载任务；将每一所述待卸载任务所在的节点确定为第一节点；从所述至少一个第一节点上卸载所述至少一个待卸载任务。
在一些实施例中，所述装置还包括：均衡部分，被配置为在满足预设的负载均衡触发条件的情况下，基于预设的每个节点的负载阈值，对所述至少一个节点进行负载均衡调整；所述预设的负载均衡触发条件包括以下至少之一：预设的定时触发条件；任务的增删频率达到预设的频率阈值。
在一些实施例中，所述负载阈值包括第一负载阈值和第二负载阈值，所述第一负载阈值不低于所述第二负载阈值，所述均衡部分还被配置为：在确定所述至少一个节点中存在负载高于所述第一负载阈值的第三节点，且存在负载低于所述第二负载阈值的第四节点的情况下，将所述第三节点中预设数量的任务移动至所述第四节点上。
在一些实施例中，所述装置还包括：第二获取部分，被配置为在检测到所述至少一个节点中存在异常节点的情况下，获取所述异常节点上运行的待重新调度的任务；第二调度部分，被配置为基于所述待重新调度的任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中除所述异常节点之外的其他节点中，调度目标节点执行所述待重新调度的任务。
在一些实施例中，所述待重新调度的任务的数量为多个，所述第二调度部分还被配置为：按照优先级从高到低的顺序，依次针对多个待重新调度的任务中的每一待重新调度的任务，基于所述待重新调度的任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述其他节点中调度目标节点执行所述待重新调度的任务。
以上装置实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本申请装置实施例中未披露的技术细节，请参照本申请方法实施例的描述而理解。
在本申请实施例以及其他的实施例中，“部分”可以是部分电路、部分处理器、部分程序或软件等等，当然也可以是单元，还可以是模块也可以是非模块化的。
需要说明的是，本申请实施例中，如果以软件功能模块的形式实现上述的任务调度方法，并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本申请各个实施例所述方法的全部或部分。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。这样，本申请实施例不限制于任何特定的硬件和软件结合。
本申请实施例提供一种任务调度设备，包括存储器和处理器，所述存储器存储有可在处理器上运行的计算机程序，所述处理器执行所述程序时实现上述方法中的步骤。
本申请实施例提供一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现上述方法中的步骤。
本申请实施例提供一种计算机程序产品，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序被计算机读取并执行时，实现上述方法中的部分或全部步骤。该计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一些实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一些实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
本申请实施例提供一种计算机程序，包括计算机可读代码，在所述计算机可读代码在计算机设备中运行的情况下，所述计算机设备中的处理器执行上述方法中的部分或全部步骤。
这里需要指出的是：以上存储介质、计算机程序、计算机程序产品和任务调度设备实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本申请存储介质、计算机程序产品和设备实施例中未披露的技术细节，请参照本申请方法实施例的描述而理解。
需要说明的是，图9为本申请实施例中任务调度设备的一种硬件实体示意图，如图9所示，该任务调度设备900的硬件实体包括：处理器901、通信接口902和存储器903，其中：处理器901通常控制计算机设备900的总体操作。通信接口902可以使计算机设备通过网络与其他终端或服务器通信。存储器903配置为存储由处理器901可执行的指令和应用，还可以缓存待处理器901以及计算机设备900中各模块待处理或已经处理的数据(例如，图像数据、音频数据、语音通信数据和视频通信数据)，可以通过闪存(FLASH)或随机访问存储器(Random Access Memory，RAM)实现。处理器901、通信接口902和存储器903之间可以通过总线904进行数据传输。
应理解，说明书通篇中提到的“一个实施例”或“一实施例”意味着与实施例有关的特定特征、结构或特性包括在本申请的至少一个实施例中。因此，在整个说明书各处出现的“在一个实施例中”或“在一实施例中”未必一定指相同的实施例。此外，这些特定的特征、结构或特性可以任意适合的方式结合在一个或多个实施例中。应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要 素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
在本申请所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元；既可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。
另外，在本申请各实施例中的各功能单元可以全部集成在一个处理单元中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：移动存储设备、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。
或者，本申请上述集成的单元如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本申请各个实施例所述方法的全部或部分。而前述的存储介质包括：移动存储设备、ROM、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。
工业实用性
本申请实施例提供了一种任务调度方法、装置、设备、存储介质及计算机程序产品、计算机程序，该方法包括：基于视频处理任务的任务类型和视频处理任务中视频流的分辨率，确定执行视频处理任务所需的资源单元的数量；基于任务类型确定执行视频处理任务的集群中至少一个节点上资源单元的使用信息，每一节点上资源单元的数量是基于节点的硬件类型确定的；基于执行视频处理任务所需的资源单元的数量和每一节点的资源单元的使用信息，从至少一个节点中调度目标节点以执行视频处理任务，目标节点中空闲的资源单元的数量不少于执行视频处理任务所需的资源单元的数量。本申请实施例中，可以在不同硬件类型的节点之间实现不同任务类型及视频流分辨率的视频处理任务的统一调度。

Claims (30)

  1. 一种任务调度方法，所述方法包括：
    获取待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率；
    基于所述任务类型和所述分辨率，确定执行所述视频处理任务所需的资源单元的数量；
    基于所述任务类型，确定执行所述视频处理任务的集群中至少一个节点上资源单元的使用信息；其中，每一所述节点上资源单元的数量是基于所述节点的硬件类型确定的；
    基于执行所述视频处理任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中调度目标节点执行所述视频处理任务；其中，所述目标节点中空闲的资源单元的数量不少于执行所述视频处理任务所需的资源单元的数量。
  2. 根据权利要求1所述的方法，其中，所述集群用于执行至少两种任务类型的视频处理任务，每一所述任务类型的视频处理任务在至少一个节点上处理；
    所述基于所述任务类型，确定执行所述视频处理任务的集群中至少一个节点上资源单元的使用信息，包括：
    基于所述任务类型，确定处理所述任务类型的视频处理任务的应用程序；
    从所述集群中确定运行所述应用程序的至少一个节点；
    确定所述至少一个节点中每一节点上资源单元的使用信息。
  3. 根据权利要求1或2所述的方法，其中，所述基于所述任务类型和所述分辨率，确定执行所述视频处理任务所需的资源单元的数量，包括：
    基于所述任务类型和所述分辨率，查询预设的任务类型、分辨率与资源单元的数量之间的对应关系，确定执行所述视频处理任务所需的资源单元的数量。
  4. 根据权利要求1至3中任一项所述的方法，其中，每一所述节点的资源单元的使用信息包括所述节点中空闲的资源单元的数量；
    所述基于执行所述视频处理任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中调度目标节点执行所述视频处理任务，包括：
    基于执行所述视频处理任务所需的资源单元的数量和每一所述节点中空闲的资源单元的数量，从所述至少一个节点中确定至少一个可用空闲节点；
    从至少一个所述可用空闲节点中确定目标节点；
    调度所述目标节点执行所述视频处理任务。
  5. 根据权利要求4所述的方法，其中，所述至少一个可用空闲节点的数量为至少两个；所述从至少一个所述可用空闲节点中确定目标节点，包括：
    采用负载均衡的方式，从所述至少两个可用空闲节点中确定目标空闲节点。
  6. 根据权利要求4或5所述的方法，其中，所述基于执行所述视频处理任务所需的资源单元的数量和每一所述节点中空闲的资源单元的数量，从所述至少一个节点中确定至少一个可用空闲节点，包括：
    基于执行所述视频处理任务所需的资源单元的数量和每一所述节点中空闲的资源单元的数量，确定所述至少一个节点中是否存在可用空闲节点；
    在确定所述至少一个节点中不存在可用空闲节点的情况下，对所述至少一个节点进行任务调度调整，得到至少一个可用空闲节点。
  7. 根据权利要求6所述的方法，其中，所述对所述至少一个节点进行任务调度调整，得到至少一个可用空闲节点，包括：
    基于所述视频处理任务所需的资源单元的数量，从所述至少一个节点中确定至少一个第一节点；
    从所述至少一个第一节点上卸载至少一个已运行的任务；
    将卸载掉任务后的所述至少一个第一节点作为所述至少一个可用空闲节点。
  8. 根据权利要求7所述的方法，其中，所述对所述至少一个节点进行任务调度调整，得到至少一个可用空闲节点，还包括：
    将已卸载的所述至少一个已运行的任务加载到所述至少一个节点中的第二节点上。
  9. 根据权利要求7或8所述的方法，其中，所述基于所述视频处理任务所需的资源单元的数量，从所述至少一个节点中确定至少一个第一节点，包括：
    从所述至少一个节点上已运行的任务中确定优先级低于所述视频处理任务的至少一个低优先级任务；
    基于所述视频处理任务所需的资源单元的数量和每一所述低优先级任务占用的资源单元的数量，将所述至少一个低优先级任务中优先级最低的至少一个任务确定为待卸载任务；
    将每一所述待卸载任务所在的节点确定为第一节点；
    所述从所述至少一个第一节点上卸载至少一个已运行的任务，包括：
    从所述至少一个第一节点上卸载所述至少一个待卸载任务。
  10. 根据权利要求1至9中任一项所述的方法，其中，所述方法还包括：
    在满足预设的负载均衡触发条件的情况下，基于预设的每个节点的负载阈值，对所述至少一个节点进行负载均衡调整；所述预设的负载均衡触发条件包括以下至少之一：预设的定时触发条件；任务的增删频率达到预设的频率阈值。
  11. 根据权利要求10所述的方法，其中，所述负载阈值包括第一负载阈值和第二负载阈值，所述第一负载阈值不低于所述第二负载阈值；
    所述基于预设的每个节点的负载阈值，对所述至少一个节点进行负载均衡调整，包括：
    在确定所述至少一个节点中存在负载高于所述第一负载阈值的第三节点，且存在负载低于所述第二负载阈值的第四节点的情况下，将所述第三节点中预设数量的任务移动至所述第四节点上。
  12. 根据权利要求1至11中任一项所述的方法，其中，所述方法还包括：
    在检测到所述至少一个节点中存在异常节点的情况下，获取所述异常节点上运行的待重新调度的任务；
    基于所述待重新调度的任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中除所述异常节点之外的其他节点中，调度目标节点执行所述待重新调度的任务。
  13. 根据权利要求12所述的方法，其中，所述待重新调度的任务的数量为多个，所述基于所述待重新调度的任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中除所述异常节点之外的其他节点中，调度目标节点执行所述待重新调度的任务，包括：
    按照优先级从高到低的顺序，依次针对多个待重新调度的任务中的每一待重新调度的任务，基于所述待重新调度的任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述其他节点中调度目标节点执行所述待重新调度的任务。
  14. 一种任务调度装置，包括：
    第一获取部分，被配置为获取待执行的视频处理任务的任务类型以及所述视频处理任务中待处理的视频流的分辨率；
    第一确定部分，被配置为基于所述任务类型和所述分辨率，确定执行所述视频处理任务所需的资源单元的数量；
    第二确定部分，被配置为基于所述任务类型，确定执行所述视频处理任务的集群中至少一个节点上资源单元的使用信息；其中，每一所述节点上资源单元的数量是基于所述节点的硬件类型确定的；
    第一调度部分，被配置为基于执行所述视频处理任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中调度目标节点执行所述视频处理任务；其中，所述目标节点中空闲的资源单元的数量不少于执行所述视频处理任务所需的资源单元的数量。
  15. 根据权利要求14所述的装置，其中，所述集群被配置为执行至少两种任务类型的视频处理任务，每一所述任务类型的视频处理任务在至少一个节点上处理；所述第二确定部分还被配置为：基于所述任务类型，确定处理所述任务类型的视频处理任务的应用程序；从所述集群中确定运行所述应用程序的至少一个节点；确定所述至少一个节点中每一节点上资源单元的使用信息。
  16. 根据权利要求14或15所述的装置，其中，所述第一确定部分还被配置为：基于所述任务类型和所述分辨率，查询预设的任务类型、分辨率与资源单元的数量之间的对应关系，确定执行所述视频处理任务所需的资源单元的数量。
  17. 根据权利要求14至16中任一项所述的装置，其中，每一所述节点的资源单元的使用信息包括所述节点中空闲的资源单元的数量，所述第一调度部分还被配置为：基于执行所述视频处理任务所需的资源单元的数量和每一所述节点中空闲的资源单元的数量，从所述至少一个节点中确定至少一个可用空闲节点；从至少一个所述可用空闲节点中确定目标节点；调度所述目标节点执行所述视频处理任务。
  18. 根据权利要求17所述的装置，其中，所述至少一个可用空闲节点的数量为至少两个；所述第一调度部分还被配置为：采用负载均衡的方式，从所述至少两个可用空闲节点中确定目标空闲节点。
  19. 根据权利要求17或18所述的装置，其中，所述第一调度部分还被配置为：基于执行所述视频处理任务所需的资源单元的数量和每一所述节点中空闲的资源单元的数量，确定所述至少一个节点中是否存在可用空闲节点；在确定所述至少一个节点中不存在可用空闲节点的情况下，对所述至少一个节点进行任务调度调整，得到至少一个可用空闲节点。
  20. 根据权利要求19所述的装置，其中，所述第一调度部分还被配置为：基于所述视频处理任务所需的资源单元的数量，从所述至少一个节点中确定至少一个第一节点；从所述至少一个第一节点上卸载至少一个已运行的任务；将卸载掉任务后的所述至少一个第一节点作为所述至少一个可用空闲节点。
  21. 根据权利要求20所述的装置，其中，所述第一调度部分还被配置为：将已卸载的所述至少一个已运行的任务加载到所述至少一个节点中的第二节点上。
  22. 根据权利要求20或21所述的装置，其中，所述第一调度部分还被配置为：从所述至少一个节点上已运行的任务中确定优先级低于所述视频处理任务的至少一个低优先级任务；基于所述视频处理任务所需的资源单元的数量和每一所述低优先级任务占用的资源单元的数量，从所述至少一个低优先级任务中确定优先级最低的至少一个待卸载任务；将每一所述待卸载任务所在的节点确定为第一节点；从所述至少一个第一节点上卸载所述至少一个待卸载任务。
  23. 根据权利要求14至22中任一项所述的装置，其中，所述装置还包括：均衡部分，被配置为在满足预设的负载均衡触发条件的情况下，基于预设的每个节点的负载 阈值，对所述至少一个节点进行负载均衡调整；所述预设的负载均衡触发条件包括以下至少之一：预设的定时触发条件；任务的增删频率达到预设的频率阈值。
  24. 根据权利要求23所述的装置，其中，所述负载阈值包括第一负载阈值和第二负载阈值，所述第一负载阈值不低于所述第二负载阈值，所述均衡部分还被配置为：在确定所述至少一个节点中存在负载高于所述第一负载阈值的第三节点，且存在负载低于所述第二负载阈值的第四节点的情况下，将所述第三节点中预设数量的任务移动至所述第四节点上。
  25. 根据权利要求14至24中任一项所述的装置，其中，所述装置还包括：第二获取部分，被配置为在检测到所述至少一个节点中存在异常节点的情况下，获取所述异常节点上运行的待重新调度的任务；第二调度部分，被配置为基于所述待重新调度的任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述至少一个节点中除所述异常节点之外的其他节点中，调度目标节点执行所述待重新调度的任务。
  26. 根据权利要求25所述的装置，其中，所述待重新调度的任务的数量为多个，所述第二调度部分还被配置为：按照优先级从高到低的顺序，依次针对多个待重新调度的任务中的每一待重新调度的任务，基于所述待重新调度的任务所需的资源单元的数量和每一所述节点的资源单元的使用信息，从所述其他节点中调度目标节点执行所述待重新调度的任务。
  27. 一种任务调度设备，包括存储器和处理器，所述存储器存储有可在处理器上运行的计算机程序，所述处理器执行所述程序时实现权利要求1至13任一项所述方法中的步骤。
  28. 一种计算机可读存储介质，其上存储有计算机程序，该计算机程序被处理器执行时实现权利要求1至13任一项所述方法中的步骤。
  29. 一种计算机程序产品，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序被计算机读取并执行时，实现权利要求1至13中任一项所述方法中的步骤。
  30. 一种计算机程序，包括计算机可读代码，在所述计算机可读代码在计算机设备中运行的情况下，所述计算机设备中的处理器执行权利要求1至13中任一项所述方法中的步骤。
PCT/CN2022/071421 2021-08-27 2022-01-11 任务调度方法、装置、设备、存储介质及计算机程序产品、计算机程序 WO2023024410A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110997563.2 2021-08-27
CN202110997563.2A CN113742068A (zh) 2021-08-27 2021-08-27 任务调度方法、装置、设备、存储介质及计算机程序产品

Publications (1)

Publication Number Publication Date
WO2023024410A1 true WO2023024410A1 (zh) 2023-03-02

Family

ID=78733481

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/071421 WO2023024410A1 (zh) 2021-08-27 2022-01-11 任务调度方法、装置、设备、存储介质及计算机程序产品、计算机程序

Country Status (2)

Country Link
CN (1) CN113742068A (zh)
WO (1) WO2023024410A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116680086A (zh) * 2023-07-25 2023-09-01 联通沃音乐文化有限公司 一种基于离线渲染引擎的调度管理系统

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113742068A (zh) * 2021-08-27 2021-12-03 深圳市商汤科技有限公司 任务调度方法、装置、设备、存储介质及计算机程序产品
CN113946431B (zh) * 2021-12-22 2022-03-04 北京瑞莱智慧科技有限公司 一种资源调度方法、系统、介质及计算设备
CN114884948A (zh) * 2022-05-05 2022-08-09 零氪科技（北京）有限公司 一种数据处理系统
CN114995974A (zh) * 2022-05-26 2022-09-02 壹沓科技(上海)有限公司 任务调度方法、装置、存储介质及计算机设备
CN115086438B (zh) * 2022-08-19 2022-11-11 南京芯驰半导体科技有限公司 任务处理方法、视频处理单元、部件及交通设备

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101986272A (zh) * 2010-11-05 2011-03-16 北京大学 一种云计算环境下的任务调度方法
CN109213593A (zh) * 2017-07-04 2019-01-15 阿里巴巴集团控股有限公司 用于全景视频转码的资源分配方法、装置和设备
CN110427270A (zh) * 2019-08-09 2019-11-08 华东师范大学 一种面向rdma网络下分布式连接算子的动态负载均衡方法
CN112422977A (zh) * 2020-09-01 2021-02-26 上海哔哩哔哩科技有限公司 音视频转码任务的分配方法和分配装置
CN112565774A (zh) * 2020-11-26 2021-03-26 北京达佳互联信息技术有限公司 视频转码资源调度方法及装置
CN112905350A (zh) * 2021-03-22 2021-06-04 北京市商汤科技开发有限公司 任务调度方法及装置、电子设备和存储介质
CN113238848A (zh) * 2021-05-27 2021-08-10 上海商汤科技开发有限公司 一种任务调度方法、装置、计算机设备和存储介质
CN113742068A (zh) * 2021-08-27 2021-12-03 深圳市商汤科技有限公司 任务调度方法、装置、设备、存储介质及计算机程序产品

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101986272A (zh) * 2010-11-05 2011-03-16 北京大学 一种云计算环境下的任务调度方法
CN109213593A (zh) * 2017-07-04 2019-01-15 阿里巴巴集团控股有限公司 用于全景视频转码的资源分配方法、装置和设备
CN110427270A (zh) * 2019-08-09 2019-11-08 华东师范大学 一种面向rdma网络下分布式连接算子的动态负载均衡方法
CN112422977A (zh) * 2020-09-01 2021-02-26 上海哔哩哔哩科技有限公司 音视频转码任务的分配方法和分配装置
CN112565774A (zh) * 2020-11-26 2021-03-26 北京达佳互联信息技术有限公司 视频转码资源调度方法及装置
CN112905350A (zh) * 2021-03-22 2021-06-04 北京市商汤科技开发有限公司 任务调度方法及装置、电子设备和存储介质
CN113238848A (zh) * 2021-05-27 2021-08-10 上海商汤科技开发有限公司 一种任务调度方法、装置、计算机设备和存储介质
CN113742068A (zh) * 2021-08-27 2021-12-03 深圳市商汤科技有限公司 任务调度方法、装置、设备、存储介质及计算机程序产品

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116680086A (zh) * 2023-07-25 2023-09-01 联通沃音乐文化有限公司 一种基于离线渲染引擎的调度管理系统
CN116680086B (zh) * 2023-07-25 2024-04-02 联通沃音乐文化有限公司 一种基于离线渲染引擎的调度管理系统

Also Published As

Publication number Publication date
CN113742068A (zh) 2021-12-03

Similar Documents

Publication Publication Date Title
WO2023024410A1 (zh) 任务调度方法、装置、设备、存储介质及计算机程序产品、计算机程序
US11567795B2 (en) Minimizing impact of migrating virtual services
US10866840B2 (en) Dependent system optimization for serverless frameworks
US8185905B2 (en) Resource allocation in computing systems according to permissible flexibilities in the recommended resource requirements
US8402466B2 (en) Practical contention-free distributed weighted fair-share scheduler
US20060069761A1 (en) System and method for load balancing virtual machines in a computer network
CN112650576B (zh) 资源调度方法、装置、设备、存储介质及计算机程序产品
CN109960575B (zh) 一种计算能力共享方法、系统及相关设备
CN110196770B (zh) 云系统内存数据处理方法、装置、设备及存储介质
CN111464659A (zh) 节点的调度、节点的预选处理方法、装置、设备及介质
KR20200080458A (ko) 클라우드 멀티-클러스터 장치
US9384050B2 (en) Scheduling method and scheduling system for multi-core processor system
CN111930516B (zh) 一种负载均衡方法及相关装置
US9736023B2 (en) Apparatus and method for changing status of cluster nodes, and recording medium having the program recorded therein
CN112118314A (zh) 负载均衡方法和装置
CN114461407B (zh) 数据处理方法、装置、分发服务器、系统及存储介质
US20230185631A1 (en) Embedded capacity-computer module for microservice load balancing and distribution
US20230297439A1 (en) Mapping logical partitions for event data to a plurality of queues
CN114675973A (zh) 资源管理方法、设备、存储介质及程序产品
US11474868B1 (en) Sharded polling system
CN111459651B (zh) 一种负载均衡方法、装置、存储介质及调度系统
GB2504812A (en) Load balancing in a SAP (RTM) system for processors allocated to data intervals based on system load
US11843548B1 (en) Resource scaling of microservice containers
CN114546279B (zh) Io请求预测方法、装置、存储节点及可读存储介质
US20230155964A1 (en) Dynamic queue management of network traffic

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22859771

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE