TW202121407A - Resource management method and apparatus, electronic device, and storage medium - Google Patents

Abstract

Examples of the present application disclose a resource management method and apparatus, an electronic device, and a storage medium. The method includes: reading a configuration file, where the configuration file includes a configuration parameter of a virtual GPU in a guest device; generating at least one available virtual GPU in the guest device based on the configuration file; in response to receiving a to-be-processed task from a target client, allocating a physical GPU to a target virtual GPU corresponding to the to-be-processed task, where the at least one available virtual GPU includes the target virtual GPU.

Description

Resource management method and device, electronic equipment and recording medium

This application relates to the field of computer technology, in particular to a resource management method and device, electronic equipment and recording media.

With the widespread application of graphics processing units (GPUs) in scientific computing, big data, finance and other fields, a large number of small-scale and light-load computing tasks make GPU resources face the challenges of low computing load and high frequency of use. On the one hand, it is difficult for a single small-scale application task to give full play to the computing performance advantages of the GPU. On the other hand, a large number of small tasks increase the power consumption of the GPU cluster and reduce the overall resource utilization.

In order to solve the above problems, the industry has proposed GPU virtualization technology. That is, each physical GPU resource is divided into multiple fine-grained virtual GPU computing resources, so that small tasks can be directly processed using virtual GPUs to improve the parallelism of applications and the resource utilization efficiency of GPU clusters.

The embodiments of the present application provide a method and device for resource management and allocation, electronic equipment, and recording media.

The first aspect of the embodiments of the present application provides a resource management allocation method, including: reading a configuration file, the configuration file including virtual image processing unit (GPU) configuration parameters of the guest device; generating the visitor based on the configuration file At least one available virtual GPU of the device; in response to receiving a task to be processed by a target user, a physical GPU is allocated to a target virtual GPU corresponding to the task to be processed, and the at least one available virtual GPU includes the target virtual GPU.

In an optional implementation manner, the allocating a physical GPU to the target virtual GPU corresponding to the task to be processed includes: setting the target virtual GPU to the target virtual GPU based on the memory capacity required by the target virtual GPU and a resource state table. Allocating physical GPUs, wherein the resource state table includes current state information of each physical GPU in the plurality of physical GPUs.

In an optional implementation manner, the current state information of the physical GPU includes: the video memory capacity of the physical GPU, user information of each virtual GPU corresponding to the video memory occupied by the physical GPU, and The video memory capacity of each virtual GPU corresponding to the video memory already occupied by the physical GPU.

In an optional implementation manner, the allocating a physical GPU to the target virtual GPU based on the required video memory capacity of the target virtual GPU and the resource state table includes: based on the required video memory capacity of the target virtual GPU With the resource state table, physical GPUs are allocated to the target virtual GPU, so that the video memory of the multiple physical GPUs is used to a minimum.

In an optional implementation manner, the allocating a physical GPU to the target virtual GPU based on the required video memory capacity and resource state table of the target virtual GPU includes: based on the resource state table, according to multiple physical GPUs The order of arrangement, the remaining video memory capacity of each physical GPU is determined in turn until the determined remaining video memory capacity meets the required video memory capacity of the target virtual GPU; the remaining video memory capacity meets the required physical memory capacity of the target virtual GPU The GPU determines the physical GPU allocated to the target virtual GPU.

In an optional implementation manner, the method further includes: in a case where it is determined that the remaining video memory capacity of the physical GPU ranked last among the multiple physical GPUs does not meet the required video memory capacity of the target virtual GPU, It is determined that the virtual GPU has no available physical resources.

In an optional implementation manner, the allocating a physical GPU to the target virtual GPU based on the display memory capacity and resource state table of the target virtual GPU includes: based on the required display memory capacity and resources of the target virtual GPU The state table allocates physical GPUs to the target virtual GPU, so that at least one task of the target user is allocated to the multiple physical GPUs as evenly as possible, wherein the at least one task includes the task to be processed .

In an optional implementation manner, the allocating a physical GPU to the target virtual GPU based on the memory capacity and resource status table of the target virtual GPU includes: determining the current target user based on the resource status table The number of tasks allocated by each physical GPU in the plurality of physical GPUs, and the remaining video memory capacity of each physical GPU; the minimum number of tasks and the remaining video memory capacity meet the required video memory capacity of the target virtual GPU The GPU determines the physical GPU allocated to the target virtual GPU.

In an optional implementation manner, the method further includes: when the number of tasks is the smallest and the remaining video memory capacity meets the required video memory capacity of the target virtual GPU, the number of physical GPUs is more than one. The foremost physical GPU among the multiple physical GPUs with the least number of tasks and the remaining video memory capacity meeting the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated by the target virtual GPU.

In an optional implementation manner, the method further includes: updating the resource state table based on the physical GPU allocated to the target virtual GPU; or, in the case that the task processing state changes, based on the The change of task processing status updates the resource status table.

In a second aspect, an embodiment of the present application provides a resource management allocation method applied to a guest device, including: reading a configuration file of a host, where the configuration file includes virtual GPU configuration parameters of the guest device; based on the configuration File, generate at least one available virtual GPU of the guest device; in the case of receiving a task to be processed by a target user, send a resource allocation request to the host based on the target virtual GPU corresponding to the task to be processed, wherein the resource The allocation request is used to request the host to allocate a physical GPU to the target virtual GPU, and the at least one available virtual GPU includes the target virtual GPU.

In an optional implementation manner, before the target virtual GPU corresponding to the task to be processed sends a resource allocation request to the host, the method further includes: determining at least one of the guest devices based on the resource status table A virtual GPU in an idle state among the available virtual GPUs, wherein the resource state table includes current state information of each of the multiple physical GPUs; the target virtual GPU is determined from the virtual GPU in the idle state.

In an optional implementation manner, the method further includes: receiving a first update instruction on the resource state table from the host, where the first update instruction is when a physical GPU is detected on the host Sent when the task processing status changes; based on the first update instruction, the resource status table is updated.

In an optional implementation manner, the at least one physical GPU includes a first physical GPU with a newly added task, and the first update instruction carries at least one of the following information: User information corresponding to the added task, information of the first physical GPU, the video memory capacity of the virtual GPU corresponding to the newly added task; and/or the at least one physical GPU includes a second physical with a newly completed task GPU, the first update command carries at least one of the following: user information corresponding to the newly completed task, and information of the second physical GPU.

In an optional implementation manner, the method further includes: scheduling the target virtual GPU to execute the task to be processed; and receiving a processing result of the task to be processed from the host.

In a third aspect, an embodiment of the present application provides a resource management allocation method applied to a host, including: receiving a resource allocation request from a guest device, the resource allocation request carrying information about the task to be processed and information about the target virtual GPU; The information of the target virtual GPU allocates a target physical GPU to the target virtual GPU.

In an optional implementation manner, the allocating a target physical GPU to the target virtual GPU based on the information of the target virtual GPU includes: setting the target virtual GPU to the target virtual GPU based on the required video memory capacity of the target virtual GPU. The GPU allocates physical GPUs.

In an optional implementation manner, the allocating a physical GPU to the target virtual GPU based on the video memory capacity of the target virtual GPU includes: based on the required video memory capacity, resource allocation strategy, and resource status of the target virtual GPU The table allocates physical GPUs to the target virtual GPU, wherein the resource state table includes current state information of each physical GPU among the multiple physical GPUs.

In an optional implementation manner, the allocating a physical GPU to the target virtual GPU based on the video memory capacity of the target virtual GPU, the resource allocation strategy, and the resource state table includes: based on the resource state table, according to The sequence of the multiple physical GPUs is to determine the remaining video memory capacity of each physical GPU in turn until the determined remaining video memory capacity meets the required video memory capacity of the target virtual GPU; and the remaining video memory capacity meets the target virtual GPU’s The physical GPU with the required video memory capacity is determined as the physical GPU allocated to the target virtual GPU.

In an optional implementation manner, in a case where it is determined that the remaining video memory capacity of the physical GPU ranked last among the multiple physical GPUs does not meet the required video memory capacity of the target virtual GPU, it is determined that the virtual GPU has no memory capacity. Available physical resources.

In an optional implementation manner, the allocating a physical GPU to the target virtual GPU based on the video memory capacity of the target virtual GPU, the resource allocation strategy, and the resource state table includes: determining based on the resource state table The target user’s current number of tasks allocated by each physical GPU among multiple physical GPUs, and the remaining video memory capacity of each physical GPU, where the target user is the user corresponding to the target virtual GPU; and the number of tasks The physical GPU with the least and remaining video memory meeting the required video memory capacity of the target virtual GPU is determined as the physical GPU of the target virtual GPU.

In an optional implementation manner, the method further includes: when the number of tasks is the smallest and the remaining video memory capacity meets the required video memory capacity of the target virtual GPU, the number of physical GPUs is more than one. The foremost physical GPU among the physical GPUs with the least number of tasks and the remaining video memory capacity meeting the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated to the target virtual GPU.

In an optional implementation manner, the method further includes: updating the resource state table based on the physical GPU allocated to the target virtual GPU; or, in the case that the task processing state changes, based on the The change of task processing status updates the resource status table.

In an optional implementation manner, the method further includes: performing task scheduling based on the task queue of each physical GPU in the plurality of physical GPUs.

In a fourth aspect, an embodiment of the present application provides a resource management system, including: a host and a guest device, wherein multiple physical GPUs are provided on the host, the guest device includes a virtual machine or a container; the guest device is used for : Read a configuration file, the configuration file including the virtual image processing unit GPU configuration parameters of the guest device; generate at least one available virtual GPU of the guest device based on the configuration file; In the case of processing tasks, a resource allocation request is sent to the host based on the target virtual GPU corresponding to the task to be processed, where the resource allocation request is used to request the host to allocate a physical GPU to the target virtual GPU, and The at least one available virtual GPU includes the target virtual GPU; the host is configured to: receive a resource allocation request from the guest device, the resource allocation request carrying information about the task to be processed and the target virtual GPU的信息; Based on the information of the target virtual GPU, assign a target physical GPU to the target virtual GPU.

In a fifth aspect, an embodiment of the present application provides a resource management device, including a communication unit and a processing unit, wherein the processing unit is configured to: obtain a configuration file through the communication unit; read the configuration file, the configuration file Including virtual image processing unit (GPU) configuration parameters of the guest device; generating at least one available virtual GPU of the guest device based on the configuration file; The target virtual GPU corresponding to the task is allocated a physical GPU, and the at least one available virtual GPU includes the target virtual GPU.

In a sixth aspect, an embodiment of the present application provides a resource management device, including a communication unit and a processing unit, wherein the processing unit is configured to: obtain a host configuration file through the communication unit; read the host configuration file, wherein: The configuration file includes virtual GPU configuration parameters of the guest device; based on the configuration file, at least one available virtual GPU of the guest device is generated; and when the task to be processed by the target user is received, the communication unit Send a resource allocation request to the host based on the target virtual GPU corresponding to the task to be processed, where the resource allocation request is used to request the host to allocate a physical GPU to the target virtual GPU, and the at least one available virtual GPU Including the target virtual GPU.

In a seventh aspect, an embodiment of the present application provides a resource management device, including a communication unit and a processing unit, wherein the processing unit is configured to: receive a resource allocation request from a guest device through the communication unit, and the resource allocation request carries Information of the task to be processed and information of the target virtual GPU; and based on the information of the target virtual GPU, a target physical GPU is allocated to the target virtual GPU.

In an eighth aspect, an embodiment of the present application provides a resource management system, including a host and a guest device, wherein the host is provided with multiple physical GPUs; the guest device is a device according to any one of the sixth aspects of the embodiments of the present application ; The host is a device according to any one of the seventh aspects of the embodiments of this application.

In a ninth aspect, an embodiment of the present application provides an electronic device including a processor and a storage device, the storage device is used to store a computer program, the computer program is configured to be executed by the processor, so that the processor executes Any method described in the first aspect, the second aspect, or the third aspect of the embodiments of the present application.

In a tenth aspect, an embodiment of the present application provides a computer-readable recording medium, wherein the computer-readable recording medium is used to store a computer program, wherein the computer program enables the computer to execute the first aspect, Any of the methods described in the second or third aspect.

In an eleventh aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable recording medium storing a computer program, and the computer program is operable to make the computer execute the application Embodiments any one of the methods described in the first aspect, the second aspect, or the third aspect. The computer program product can be a software installation package.

The embodiment of the application first reads the configuration file, where the configuration file includes the configuration parameters of the virtual GPU of the guest device, and then based on the configuration file, generates at least one available virtual GPU of the guest device, and after receiving the target customer In the case of a task to be processed, a physical GPU is allocated to a target virtual GPU corresponding to the task to be processed, and the at least one available virtual GPU includes the target virtual GPU. In the embodiment of the present application, a fixed mapping relationship is not established for the virtual GPU and the physical GPU in advance, but after the virtual GPU receives the task to be processed, the virtual GPU is dynamically assigned the corresponding physical GPU, which can be flexibly based on the current task The characteristics of the physical GPU resources are managed and allocated to improve the resource utilization of the physical GPU and improve the overall performance of the system.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

The term "and/or" in this application is only an association relationship that describes associated objects, indicating that there can be three types of relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, exist alone B these three situations. In addition, the term "at least one" in this document means any one or any combination of at least two of the multiple, for example, including at least one of A, B, and C, and may mean including those formed from A, B, and C. Any one or more elements selected in the set. The terms "first", "second", etc. in the description of the application, the scope of the patent application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations of them are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes unlisted steps or units, or optionally also includes Other steps or units inherent to these processes, methods, products or equipment.

The reference to "embodiments" herein means that a specific feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art clearly and implicitly understand that the embodiments described herein can be combined with other embodiments.

The resource management and distribution device involved in the embodiments of the present application is a device that can perform resource management and distribution, and may be an electronic device. The above-mentioned electronic device includes a terminal device. In a specific implementation, the above-mentioned terminal device includes, but is not limited to, a touch-sensitive surface. (For example, touch screen display and/or touch pad) other portable devices such as mobile phones, laptops or tablets. It should also be understood that, in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (for example, a touch screen display and/or a touch pad), a servo器等。

Fig. 1 is a schematic diagram of a resource management system provided by an embodiment of the application. Among them, the GPU virtualization solution involves a guest device (Guest) 101 and a host (Host) 102. The host 102 refers to a physical machine node of a device with a physical GPU (pGPU) 140. The user can directly perform GPU tasks on the host. In addition, the host also contains a service daemon, which is responsible for starting the physical GPU and parsing and forwarding data with guest devices, etc. Communication operation. The guest device 101 refers to a virtual machine or container that can use a virtualized GPU. The guest device 101 does not have any GPU device and cannot directly perform GPU tasks. The guest device 101 contains a process for GPU virtualization, which is responsible for the dynamic linking of the encapsulated GPU The library and data are forwarded so that tasks on the guest device 101 can be transparently operated using the physical GPU of the host.

In the resource management system provided by the embodiment of this application, the guest's virtual GPU (vGPU) resource management strategy can be selected as follows: the guest's virtual GPU resource management strategy is divided into two stages: virtual display according to configuration options and actual allocation according to task start. Among them, Guest can be a virtual machine. As shown in Figure 1, the main allocation process is as follows: First, start a virtual machine (Virtual Machine, VM) of the guest device, where there may be multiple virtual machines, such as virtual machines 110-1, 110-n, and so on. By reading the configuration file (Config) 121, obtain the virtual GPU related parameters on the current VM, including the number of virtual GPUs, the memory capacity of each virtual GPU, the use time of each virtual GPU, and the size of each virtual GPU time slice Then, a virtual virtual GPU image 122 is generated on the virtual GPU guest device of the VM. At this time, the virtual GPU is not mapped with the physical GPU, and only the software level is virtually allocated to the virtual GPU information observed by the user; when the user on the VM When the task 123 that uses GPU resources is started, the virtual machine sends out a request to initialize GPU information. At this time, the VM will establish a connection with the Host, and establish virtual GPUs (vGPUs) 124 and physical GPUs (vGPUs) through the virtual GPU-physical GPU resource allocation strategy 130. pGPU) 140 actual association. Among them, one virtual GPU 124 may be associated with one physical GPU, or may be associated with multiple physical GPUs. Host 102 returns the information of the physical GPU 140 to the VM, such as the address ID of the GPU, the device model of the physical GPU, the remaining video memory capacity of the physical GPU, the start time of applying for the use of the physical GPU, and other parameters. The virtual GPU updates the above information and follows the configuration The parameter calls the GPU virtualization process to forward the data of task 123 to the Host. The host's service daemon starts the associated physical GPU and processes task 123 according to the forwarded data, and sends the task processing result back to the VM after the task processing is completed. It should be noted that although the user on the VM looks like the task 123 is completed on the virtual GPU 124 on the VM, in fact, the task 123 is completed on the physical GPU 140 of the host.

It is understandable that the aforementioned Guest and Host may be set on the same device; for example, the physical GPU on the host and the guest device installed on the host may also be set on different devices. For example, guest devices installed on user equipment and various physical GPU node devices installed on the network.

FIG. 2 is a schematic flowchart of a resource management and allocation method disclosed in an embodiment of the present application. As shown in FIG. 2, the resource management and allocation method includes the following steps.

201: Read a configuration file, where the configuration file includes virtual GPU configuration parameters of the guest device.

Wherein, the configuration file includes virtual GPU-related parameters on the current guest device. The virtual GPU-related parameters may include, but are not limited to, the number of virtual GPUs, the memory capacity of each virtual GPU, the use time of each virtual GPU, and each One or more of the virtual GPU time slice size, etc.

Optionally, the configuration file may be preset by the system or generated according to the needs of the user.

202: Based on the configuration file, generate at least one available virtual GPU of the guest device.

In the embodiment of the present application, after reading the above configuration file, the guest device can generate at least one available virtual GPU of the guest device based on the configuration file. The above virtual GPU is a software level used to present to the user for viewing. For example, after generating at least one available virtual GPU, the user can view the number of virtual GPUs on the guest device through the user device, as well as the memory capacity of each virtual GPU, the usage time of each virtual GPU, and the Parameters such as the size of a virtual GPU time slice. At this time, the virtual GPU is not mapped with the physical GPU.

203: In response to receiving the to-be-processed task of the target user, allocate a physical GPU to the target virtual GPU corresponding to the to-be-processed task, where the at least one available virtual GPU includes the target virtual GPU.

In the embodiment of the present application, after the guest device receives the task to be processed, the guest device allocates a target virtual GPU for the task to be processed to process the task to be processed. Then, a physical GPU is allocated to the target virtual GPU according to the specific information of the target virtual GPU, that is, a mapping relationship between the virtual GPU and the physical GPU is established. After the physical GPU is allocated to the target virtual GPU, the task to be processed is processed in the physical GPU allocated by the target virtual GPU, and the processing result is fed back to the guest device. That is, the above to-be-processed tasks are processed in the physical GPU allocated by the host, and the processing results are fed back to the guest device.

The embodiment of the application reads a configuration file, where the configuration file includes the parameters of the virtual GPU of the guest device, and the guest device includes at least one virtual GPU; The target virtual GPU corresponding to the task is allocated a physical GPU, and the at least one available virtual GPU includes the target virtual GPU. In the embodiment of the present application, the virtual GPU and the physical GPU do not establish a mapping relationship before receiving the task. After the task is received, the mapping relationship may be established between the virtual GPU and the physical graphics virtual unit. Furthermore, it is possible to flexibly manage and allocate GPU resources through different resource allocation strategies, so as to improve the use efficiency of physical GPUs.

As an optional implementation manner, the allocating a physical GPU to the target virtual GPU corresponding to the task to be processed includes: setting the target virtual GPU to the target virtual GPU based on the required video memory capacity and the resource state table of the target virtual GPU Allocate physical GPUs, where the resource state table includes current state information of each of the multiple physical GPUs in the host.

Wherein, the current state information includes: among the above-mentioned multiple physical GPUs, the video memory capacity of each physical GPU, the user information of each virtual GPU corresponding to the video memory occupied by the physical GPU in each physical GPU, and each The video memory capacity of each virtual GPU corresponding to the video memory that has been occupied by a physical GPU. Alternatively, the video memory capacity of each physical GPU, the user ID of each virtual GPU in each physical GPU corresponding to the video memory already occupied by the physical GPU, and the video memory capacity of each physical GPU may be derived based on the current state information. Information such as the video memory capacity of each virtual GPU corresponding to the video memory already occupied by the physical GPU. Optionally, the aforementioned current state information may also include a task identifier of each virtual GPU in each physical GPU corresponding to the video memory that the physical GPU has occupied.

Table 1 is a specific implementation manner of a resource status table provided in an embodiment of the application. Among them, pGPU stands for physical GPU, vGPU stands for virtual GPU, and ID stands for address or identification.

Table 1 Resource status table pGPU1-ID pGPU1 memory capacity pGPU1 remaining video memory capacity vGPU1-ID vGPU1 memory capacity User-ID1 Task ID 1 ... vGPU2-ID vGPU2 memory capacity User-ID2 Task ID 2 ... vGPU3-ID vGPU3 memory capacity User-ID1 Task ID 1 ... vGPU4-ID vGPU4 memory capacity User-ID3 Task ID 3 ... .... ... .... ... ... pGPU2-ID pGPU2 memory capacity pGPU2 remaining video memory capacity vGPUn-ID vGPUn-ID video memory capacity User-ID5 Task ID 5 ... vGPUn+1-ID vGPUn+1-ID video memory capacity User-ID8 Task ID 8 ... ... ... ... ... ... ... … ... ... ... ... ... …

In a specific implementation, after receiving the above-mentioned task to be processed and determining the target virtual GPU, a physical GPU that satisfies the conditions for executing the task to be processed can be determined from the resource state table according to the memory capacity of the target virtual GPU as the target virtual GPU. For example, a physical GPU whose remaining video memory is greater than the video memory of the target virtual GPU may be determined as the physical GPU allocated by the target virtual GPU.

Further, the allocating a physical GPU to the target virtual GPU based on the required video memory capacity and resource state table of the target virtual GPU may specifically include: based on the video memory capacity, resource state table, and resource allocation of the target virtual GPU The strategy allocates physical GPUs to the target virtual GPU.

Among them, the above resource allocation strategy refers to a mapping rule between the virtual GPU and the physical GPU of the resource formulated to achieve a certain effect. For example, a minimum resource allocation strategy formulated in order to achieve the least use of physical GPUs, and another example, a load balancing allocation strategy formulated in order to make multiple tasks of the same user evenly distributed among the physical GPUs. As an example, the resource allocation strategy may be located at the host, and a partial mapping relationship of the resource allocation strategy is stored in the guest device.

As an optional implementation manner, the allocating a physical GPU to the target virtual GPU based on the target virtual GPU's video memory capacity, a resource state table, and a resource allocation strategy includes: based on the resource state table, according to multiple physical GPUs The order of GPU arrangement is to determine the remaining video memory capacity of each physical GPU in turn until the determined remaining video memory capacity meets the required video memory capacity of the target virtual GPU; the remaining video memory capacity meets the required video memory capacity of the target virtual GPU The physical GPU determines the physical GPU allocated to the target virtual GPU. If it is determined that the remaining video memory capacity of the last physical GPU among the multiple physical GPUs does not meet the required video memory capacity of the target virtual GPU, that is, the remaining video memory of each physical GPU is less than the The required video memory of the target virtual GPU, and it is determined that the virtual GPU has no available physical resources. In the embodiments of the present application, the resource allocation strategy in this allocation method of virtual GPU and physical GPU may be referred to as the least resource allocation strategy.

Among them, the arrangement order of the multiple physical GPUs can be arranged in the order of the ID numbers of the physical GPUs, or the arrangement order of the multiple physical GPUs in the resource state table, such as pGPU1-pGPUn in Table 1, and in Table 1. The arrangement order of the physical GPUs can be arranged according to the ID number order of the physical GPUs, or according to the location information of the physical GPUs in the host, and can also be arranged according to the time when the physical GPUs are added to the host. This application No restrictions.

In specific implementation, after determining the video memory capacity of the target virtual GPU for the task to be processed, according to the order of the physical GPUs in the resource state table, first obtain the remaining video memory capacity of the first physical GPU, and then determine the first physical GPU Whether the remaining video memory capacity of is greater than or equal to the required video memory capacity of the target virtual GPU, if it is greater than or equal to the required video memory capacity of the target virtual GPU, the first physical GPU is determined as the physical GPU allocated by the target virtual GPU; otherwise, the second The remaining video memory capacity of a physical GPU, and then determine whether the remaining video memory capacity of the second physical GPU is greater than or equal to the required video memory capacity of the target virtual GPU, if it is greater than or equal to the required video memory capacity of the target virtual GPU, the second physical GPU The GPU determines the physical GPU allocated to the target virtual GPU; otherwise, it continues to obtain the remaining video memory capacity of the next physical GPU to perform the same judgment operation. By analogy, if the remaining video memory capacity of all physical GPUs is less than the video memory capacity of the aforementioned target virtual GPU, it is determined that the virtual GPU has no available physical resources.

FIG. 3 is a structural diagram of a minimum resource management allocation strategy provided by an embodiment of the application. Among them, the shaded rectangle with a label indicates the task to be processed, the label in the rectangle indicates the order in which the task is generated, VPT indicates the resource state table, pGPU, pGPU1 indicate physical GPU, vGPU, vGPU2 indicate virtual GPU, as can be seen from the architecture diagram Tasks are assigned to the physical GPUs arranged in front in the order of their generation.

In the embodiment of the present application, when it is determined that the target virtual GPU allocates physical GPUs, the physical GPUs arranged in the front are preferentially allocated to the target virtual GPUs, so as to maximize the utilization of each GPU to save GPU resources.

As another optional implementation manner, the allocating a physical GPU to the target virtual GPU based on the memory capacity of the target virtual GPU, a resource state table, and a resource allocation strategy includes: determining the target virtual GPU based on the resource state table The target user's current number of tasks allocated by each physical GPU among multiple physical GPUs and the remaining video memory of each physical GPU, the user ID is the identification of the target user of the target virtual GPU; the number of tasks is the smallest and the remaining The physical GPU whose video memory capacity meets the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated to the target virtual GPU. The minimum number of tasks refers to the minimum number of tasks that the target user runs on the physical GPU, rather than the minimum number of all tasks running on the physical GPU. In the embodiments of the present application, the resource allocation strategy in this allocation method of virtual GPU and physical GPU is referred to as a load balancing allocation strategy.

In a specific implementation, based on the information in the resource state table, the number of tasks of the target user corresponding to the memory occupied by each physical GPU in the multiple physical GPUs can be determined, so as to count the tasks of the target user in the above-mentioned multiple physical GPUs. According to the distribution of physical GPUs, the physical GPU with the least number of tasks and the remaining video memory meeting the required video memory capacity of the target virtual GPU is then determined as the physical GPU allocated by the target virtual GPU. In this way, the tasks of the target users mentioned above can be evenly distributed to multiple physical GPUs, so as to reduce the accumulation of multiple tasks of the same user to execute on the same physical GPU, resulting in a single user multi-task queuing phenomenon, thereby increasing the user's tasks Implementation efficiency.

Further, in the case that the number of physical GPUs with the least number of tasks and remaining video memory meeting the required video memory capacity of the target virtual GPU is multiple, the number of tasks is the least and the remaining video memory capacity meeting the target The physical GPU arranged at the forefront among the physical GPUs with the required video memory capacity of the virtual GPU is determined as the physical GPU of the target virtual GPU. The arrangement order of the multiple physical GPUs may be arranged in the order of the ID numbers of the physical GPUs, or may be the arrangement order of the multiple physical GPUs in the aforementioned resource state table.

FIG. 4 is a structural diagram of a load balancing distribution strategy provided by another embodiment of the application. Among them, the shaded rectangle with a label indicates the task to be processed, the label in the rectangle indicates the order in which the task is generated, VPT indicates the resource state table, pGPU, pGPU1 indicate physical GPU, vGPU, vGPU2 indicate virtual GPU, as can be seen from the architecture diagram Tasks with the same user ID are evenly distributed to each physical GPU.

As an optional implementation manner, after determining the physical GPU of the target virtual GPU, the method further includes: updating the resource state table based on the physical GPU allocated to the target virtual GPU.

In a specific implementation manner, after the physical GPU of the target virtual GPU is determined, information such as the physical GPU-ID of the target virtual GPU, the ID of the target virtual GPU, the user ID of the target virtual GPU, and the memory capacity of the target virtual GPU are passed. Update the above resource status table.

As an optional implementation manner, in the case where there are tasks executed and completed in the multiple physical GPUs, the resource state table is updated based on the executed tasks.

Specifically, the information of the physical GPU that has completed the task in the resource state table can be updated through the physical GPU-ID of the completed task and the ID of the virtual GPU corresponding to the completed task.

FIG. 5 is a schematic flowchart of a resource management allocation method disclosed in another embodiment of the present application, which is applied to a guest device. As shown in FIG. 5, the resource management allocation method includes the following steps.

501: Read a configuration file of the host, where the configuration file includes virtual GPU configuration parameters of the guest device.

502: Generate at least one available virtual GPU of the guest device based on the configuration file.

In the embodiment of the present application, steps 501 and 502 can refer to steps 201 and 202 in FIG. 2, which will not be repeated here.

503: In the case of receiving the task to be processed by the target user, send a resource allocation request to the host based on the target virtual GPU corresponding to the task to be processed, where the resource allocation request is used to request the host to be the target The virtual GPU allocates a physical GPU, and the at least one available virtual GPU includes the target virtual GPU.

In the embodiment of the present application, the aforementioned guest device may correspond to the Guest in FIG. 1, and the aforementioned host may correspond to the Host in FIG. 1.

In an example, the guest device may be a virtual machine, and the virtual machine includes at least one virtual GPU. When the virtual GPU in the virtual machine does not process the task to be processed, the virtual GPU in the virtual machine does not have a physical GPU corresponding to it.

After the virtual machine receives the task to be processed, the virtual machine allocates a target virtual GPU corresponding to the task to be processed, and then sends resources to the host according to the information of the allocated target virtual GPU, such as the memory capacity of the target virtual GPU An allocation request to request the host to allocate a physical GPU to the target virtual GPU.

Wherein, the resource allocation request may carry the ID of the target virtual GPU and the video memory capacity, and may also carry the user ID of the target virtual GPU.

In another example, the guest device may also be a container, and the container includes at least one virtual GPU. When the virtual GPU in the container does not process the task to be processed, the virtual GPU in the container does not have a physical GPU corresponding to it. After the container receives the task to be processed, the container allocates the target virtual GPU corresponding to the task to be processed, and then sends a resource allocation request to the host according to the information of the allocated target virtual GPU, such as the memory capacity of the target virtual GPU To request the host to allocate a physical GPU to the target virtual GPU. Wherein, the resource allocation request may carry the ID of the target virtual GPU and the video memory capacity, and may also carry the user ID of the target virtual GPU.

The embodiment of the application reads a configuration file, where the configuration file includes the parameters of the virtual GPU of the guest device, and the guest device includes at least one virtual GPU; in the case of receiving a task to be processed, based on the to-be-processed task The target virtual GPU corresponding to the task sends a resource allocation request to the host. In the embodiment of the present application, the virtual GPU and the physical GPU do not establish a mapping relationship before the task is received. After the task is received, the mapping relationship between the virtual GPU and the physical GPU is dynamically established through a resource allocation strategy. Therefore, the mapping relationship between the virtual GPU and the physical GPU can be flexibly established to improve the use efficiency of the physical GPU.

As an optional implementation manner, before the sending a resource allocation request to the host based on the target virtual GPU corresponding to the task to be processed, the method further includes: based on the number of virtual GPUs in the guest device and the resource status table Determine the number of unused virtual GPUs of the guest device, and the resource state table includes information about the current usage status of each physical GPU; The target virtual GPU is determined from the used virtual GPUs.

Wherein, the information contained in the resource status table may optionally be consistent with the information contained in the resource status table in FIG. 2, and will not be repeated here.

Optionally, the method further includes: receiving a first update instruction regarding the resource state table, the first update instruction being sent by the host when a change in the task processing state of the physical GPU is detected; Based on the first update instruction, the resource status table is updated. Among them, a change in the task processing state may be that a new task is added to the physical GPU or a task is completed.

Specifically, when the host detects that a new task is added to the physical GPU, the first update instruction carries at least one of the following information: user information corresponding to the newly added task of the first physical GPU, Information of the first physical GPU, the video memory capacity of the virtual GPU of the newly added task; in the case that the host detects that there is a newly completed task in the video memory of the physical GPU, the first update command carries the following information At least one item of: user information corresponding to the newly completed task, and information of the second physical GPU corresponding to the newly completed task. Wherein, the first physical GPU is a GPU newly added to a task, and the second physical GPU is a GPU corresponding to a newly completed task. The user information may be the ID of the user, and the information of the physical GPU may be the ID of the physical GPU.

Optionally, the method further includes: scheduling the target virtual GPU to execute the task to be processed; and receiving a processing result of the task to be processed from the host.

FIG. 6 is a schematic flowchart of a resource management and allocation method disclosed in another embodiment of the present application, which is applied to a host. As shown in FIG. 6, the resource management and allocation method includes the following steps.

601: Receive a resource allocation request from a guest device, where the resource allocation request carries information about a task to be processed and information about a target virtual GPU, where the target virtual GPU is a virtual GPU included in the guest device.

In the embodiment of the present application, the aforementioned guest device may correspond to the Guest in FIG. 1, and the aforementioned host may correspond to the Host in FIG. 1.

The aforementioned guest device includes at least one virtual GPU. When the virtual GPU in the guest device does not process the task to be processed, the virtual GPU in the guest device does not have a physical GPU corresponding to it.

The aforementioned information of the target virtual GPU may include, but is not limited to, the memory capacity of the target virtual GPU, and the user ID of the target virtual GPU.

602: Assign a target physical GPU to the target virtual GPU based on the information of the target virtual GPU.

In the embodiment of the present application, the allocating a target physical GPU to the target virtual GPU based on the information of the target virtual GPU includes: allocating a physical GPU to the target virtual GPU based on the memory capacity of the target virtual GPU. Wherein, the target physical GPU is used to process the task to be processed according to the information of the task to be processed.

Further, the allocating a physical GPU to the target virtual GPU based on the video memory capacity of the target virtual GPU includes: assigning a physical GPU to the target virtual GPU based on the required video memory capacity of the target virtual GPU, a resource allocation strategy, and a resource state table. The GPU allocates physical GPUs, wherein the resource state table includes current state information of each physical GPU in the plurality of physical GPUs.

In the embodiment of the present application, the above-mentioned specific implementation process of allocating a physical GPU to the target virtual GPU based on the memory capacity of the target virtual GPU, the resource allocation strategy, and the resource state table can refer to the "based on the target virtual GPU" in Figure 2. The required video memory capacity of the virtual GPU and the resource state table allocate operations corresponding to the “physical GPU” for the target virtual GPU. It should be noted that the operation of determining the number of tasks currently allocated by the target user to each physical GPU in the multiple physical GPUs can be understood as determining the number of tasks currently allocated by the target user to each physical GPU in the multiple physical GPUs , The target user is the user corresponding to the target virtual GPU. Among them, the "resource status table" in the embodiment of the present application corresponds to the "resource status" in the "resource status for the target virtual GPU based on the required video memory capacity and resource status table of the target virtual GPU" in Figure 2 above. table". In an example, the resource state table of the host may be called the second resource state table, and the resource state table of the guest device may be called the first resource state table, and the second resource state table is the same as the first resource state table, for example The guest device copies the resource status table on the host to the local, and the host maintains the second resource status table, and when the second resource status table is updated, the guest device is notified to update the first resource status table synchronously. For example, the guest device may be notified to update the first resource status table through the first update instruction.

As an optional implementation manner, the method further includes: updating the resource state table based on the physical GPU allocated to the target virtual GPU; or, in the case that the task processing state changes, based on the task The resource status table is updated with the change of processing status. Based on the physical GPU allocated to the target virtual GPU, the resource state table is updated, including at least one of the following: user information corresponding to the newly added task of the allocated physical GPU, information of the allocated physical GPU, and The video memory capacity of the virtual GPU corresponding to the newly added task is described. When the task processing state changes, the resource state table is updated based on the change in the task processing state, including at least one of the following: user information corresponding to the newly completed task, and information about the physical GPU that completed the task. The information of the physical GPU includes the ID of the physical GPU, and the user information includes the user ID.

As an optional implementation manner, after allocating physical GPUs to the target virtual GPU corresponding to the task to be processed, the method may further include: performing task scheduling based on the task queue of each of the multiple physical GPUs .

Wherein, the task queue refers to tasks included in the video memory occupied by the physical GPU.

This embodiment of the application first receives a resource allocation request from a guest device, the resource allocation request carries information about the task to be processed and the target virtual GPU. The target virtual GPU is a virtual GPU included in the guest device in the guest device, and then based on all the information. The information and resource management strategy of the target virtual GPU allocates a target physical GPU to the target virtual GPU. In the embodiment of the application, a fixed mapping relationship is not established for the virtual GPU and the physical GPU in advance, but after the virtual GPU receives the task, the virtual GPU is dynamically allocated the corresponding physical GPU, which can be flexibly allocated through different resources Strategies to manage and allocate GPU resources to improve the resource utilization of the physical GPU and improve the overall performance of the system.

Similar to the embodiment shown in FIG. 2 above, FIG. 7 is a schematic structural diagram of an electronic device 700 provided by an embodiment of the present application. As shown in FIG. 7, the electronic device 700 includes a processor 710, a storage device 720, and a communication device. Interface 730 and physical GPU 740. The one or more programs 721 are stored in the storage device 720 and are configured to be executed by the processor 710, and the one or more programs 721 include instructions for executing the following steps.

Read a configuration file, the configuration file including the GPU configuration parameters of the guest device; and based on the configuration file, generate at least one available virtual GPU of the guest device; The target virtual GPU corresponding to the task to be processed is allocated a physical GPU, and the at least one available virtual GPU includes the target virtual GPU.

The embodiment of the application reads a configuration file, where the configuration file includes configuration parameters of the virtual GPU of the guest device; and based on the configuration file, generates at least one available virtual GPU of the guest device; In the case of a task, a physical GPU is allocated to the target virtual GPU corresponding to the task to be processed, and the at least one available virtual GPU includes the target virtual GPU. In the embodiment of the present application, a fixed mapping relationship is not established for the virtual GPU and the physical GPU in advance, but after the virtual GPU receives the task to be processed, the virtual GPU is dynamically assigned the corresponding physical GPU, which can be flexibly based on the current task The characteristics of the GPU resources are allocated to improve the resource utilization of the physical GPU and improve the overall performance of the GPU system.

In an optional implementation manner, the target virtual GPU corresponding to the task to be processed is allocated a physical GPU, and the instructions in the program are specifically used to perform the following operations: based on the video memory capacity required by the target virtual GPU And a resource state table for assigning physical GPUs to the target virtual GPU, wherein the resource state table includes current state information of each of the multiple physical GPUs.

In an optional implementation manner, the current state information of the physical GPU includes: the video memory capacity of the physical GPU, user information of each virtual GPU corresponding to the video memory occupied by the physical GPU, and The video memory capacity of each virtual GPU corresponding to the video memory already occupied by the physical GPU.

In an optional implementation manner, the target virtual GPU is allocated a physical GPU based on the required video memory capacity and the resource state table of the target virtual GPU, and the instructions in the program are specifically used to perform the following operations: Based on the required video memory capacity and resource state table of the target virtual GPU, physical GPUs are allocated to the target virtual GPU, so that the video memory of the multiple physical GPUs is used to a minimum.

In an optional implementation manner, the target virtual GPU is allocated a physical GPU based on the required video memory capacity and resource state table of the target virtual GPU, and the instructions in the program are specifically used to perform the following operations: The resource state table determines the remaining video memory capacity of each physical GPU in sequence according to the arrangement order of the multiple physical GPUs, until the determined remaining video memory capacity meets the required video memory capacity of the target virtual GPU; and the remaining video memory capacity meets all requirements. The physical GPU of the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated to the target virtual GPU.

In an optional implementation manner, the instructions in the program are also used to perform the following operations: after determining that the remaining video memory capacity of the physical GPU ranked last among the multiple physical GPUs does not meet the requirements of the target virtual GPU In the case where the video memory capacity is required, it is determined that the virtual GPU has no available physical resources.

In an optional implementation manner, the target virtual GPU is allocated a physical GPU based on the memory capacity and resource state table of the target virtual GPU, and the instructions in the program are specifically used to perform the following operations: The required video memory capacity and resource state table of the target virtual GPU, and physical GPUs are allocated to the target virtual GPU, so that at least one task of the target user is allocated to the multiple physical GPUs as evenly as possible, wherein, The at least one task includes the pending task.

In an optional implementation manner, the target virtual GPU is allocated a physical GPU based on the memory capacity and resource state table of the target virtual GPU, and the instructions in the program are specifically used to perform the following operations: The resource status table determines the number of tasks currently allocated by each physical GPU of the multiple physical GPUs by the target user, and the remaining video memory capacity of each physical GPU; and minimizes the number of tasks and the remaining video memory capacity satisfies all The physical GPU of the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated to the target virtual GPU.

In an optional implementation manner, the instructions in the program are also used to perform the following operations: when the number of tasks is the smallest and the remaining video memory capacity meets the required video memory capacity of the target virtual GPU, the number of physical GPUs is greater. In the case of one, the physical GPU arranged at the forefront among the multiple physical GPUs with the least number of tasks and the remaining video memory capacity meeting the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated by the target virtual GPU .

In an optional implementation manner, the instructions in the program are also used to perform the following operations: update the resource state table based on the physical GPU allocated to the target virtual GPU; or, when the task processing state changes In the case of updating the resource status table based on the change of the task processing status.

Similar to the embodiment shown in FIG. 5, FIG. 8 is a schematic structural diagram of an electronic device 800 provided by another embodiment of the present application, and the electronic device 800 may be a visitor device. As shown in FIG. 8, the electronic device 800 includes a processor 810, a storage device 820 and a communication interface 830. The one or more programs 821 are stored in the storage device 820 and are configured to be executed by the processor 810, and the one or more programs 821 include instructions for executing the following steps.

Read the main configuration file, where the configuration file includes virtual GPU configuration parameters of the guest device; generate at least one available virtual GPU of the guest device based on the configuration file; In the case of processing tasks, a resource allocation request is sent to the host based on the target virtual GPU corresponding to the task to be processed, where the resource allocation request is used to request the host to allocate a physical GPU to the target virtual GPU, and the at least One available virtual GPU includes the target virtual GPU.

The embodiment of the application reads a configuration file, where the configuration file includes the parameters of the virtual GPU of the guest device; based on the configuration file, at least one available virtual GPU of the guest device is generated; In this case, a resource allocation request is sent to the host based on the target virtual GPU corresponding to the task to be processed. In the embodiment of this application, a fixed mapping relationship is not established for the virtual GPU and the physical GPU in advance. Instead, after the virtual GPU receives the task, it dynamically sends a resource allocation request to the host to improve the resource utilization of the physical GPU, and Improve the overall performance of the GPU system.

In an optional implementation manner, before the target virtual GPU corresponding to the task to be processed sends a resource allocation request to the host, the instructions in the program are further used to perform the following operations: determine based on the resource status table The virtual GPU in the idle state among the at least one available virtual GPU of the guest device, wherein the resource state table includes current state information of each of the multiple physical GPUs; from the virtual GPU in the idle state Determine the target virtual GPU.

In an optional implementation manner, the instructions in the program are also used to perform the following operations: receiving a first update instruction on the resource status table from the host, where the first update instruction is Sent when the host detects that the task processing state of the physical GPU has changed; based on the first update instruction, the resource state table is updated.

In an optional implementation manner, the at least one physical GPU includes a first physical GPU with a newly added task, and the first update instruction carries at least one of the following information: the newly added task of the first physical GPU corresponds to User information of the first physical GPU, the video memory capacity of the virtual GPU corresponding to the newly added task; and/or at least one physical GPU includes a second physical GPU with a newly completed task, the first The update command carries at least one of the following: user information corresponding to the newly completed task, and information of the second physical GPU.

In an optional implementation manner, the instructions in the program are further used to perform the following operations: scheduling the target virtual GPU to execute the task to be processed; and receiving the processing result of the task to be processed from the host.

Similar to the embodiment shown in FIG. 6 above, FIG. 9 is a schematic structural diagram of an electronic device 900 according to another embodiment of the present application, and the electronic device 900 may be a host. As shown in FIG. 9, the electronic device 900 includes a processor 910, a storage device 920, a communication interface 930, and a physical GPU 940. The one or more programs 921 are stored in the storage device 920 and are configured to be executed by the processor 910, and the one or more programs 921 include instructions for executing the following steps.

Receiving a resource allocation request from the guest device, the resource allocation request carrying information of the task to be processed and information of a target virtual GPU; and based on the information of the target virtual GPU, a target physical GPU is allocated to the target virtual GPU.

The embodiment of the application receives a resource allocation request from a guest device, the resource allocation request carries information about the task to be processed and information about the target virtual GPU; and based on the information about the target virtual GPU, allocates a target physical to the target virtual GPU GPU. In the embodiment of this application, a fixed mapping relationship is not established for the virtual GPU and the physical GPU in advance. Instead, after receiving the resource allocation request from the guest device, the virtual GPU is dynamically allocated the corresponding physical GPU, which can be flexibly based on the current The characteristics of the task are used to allocate GPU resources to improve the resource utilization of the GPU and improve the overall performance of the GPU system.

In an optional implementation manner, the target virtual GPU is assigned a target physical GPU based on the information of the target virtual GPU, and the instructions in the program are specifically used to perform the following operations: based on the target virtual GPU The required video memory capacity is allocated to the target virtual GPU with physical GPUs.

In an optional implementation manner, the physical GPU is allocated to the target virtual GPU based on the video memory capacity of the target virtual GPU, and the instructions in the program are specifically used to perform the following operations: The required video memory capacity, the resource allocation strategy, and the resource state table allocate physical GPUs to the target virtual GPU, wherein the resource state table includes current state information of each of the multiple physical GPUs.

In an optional implementation manner, the target virtual GPU is allocated a physical GPU based on the memory capacity of the target virtual GPU, the resource allocation strategy, and the resource state table, and the instructions in the program are specifically used to execute The following operations: based on the resource state table, according to the arrangement order of the multiple physical GPUs, sequentially determine the remaining video memory capacity of each physical GPU until the determined remaining video memory capacity meets the required video memory capacity of the target virtual GPU; A physical GPU whose remaining video memory capacity meets the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated by the target virtual GPU.

In an optional implementation manner, the instructions in the program are also used to perform the following operations, in determining that the remaining memory capacity of the physical GPU ranked last among the multiple physical GPUs does not meet the requirements of the target virtual GPU. In the case where the video memory capacity is required, it is determined that the virtual GPU has no available physical resources.

In an optional implementation manner, in the aspect of allocating a physical GPU to the target virtual GPU based on the video memory capacity of the target virtual GPU, the resource allocation strategy, and the resource state table, the instructions in the program are specifically used To perform the following operations: based on the resource state table, determine the number of tasks that the target user currently has allocated to each physical GPU among the multiple physical GPUs, and the remaining video memory capacity of each physical GPU, where the target user is all The user corresponding to the target virtual GPU; the physical GPU with the least number of tasks and the remaining video memory meeting the required video memory capacity of the target virtual GPU is determined as the physical GPU of the target virtual GPU.

In an optional implementation manner, the instructions in the program are also used to perform the following operations: when the number of tasks is the smallest and the remaining video memory capacity meets the required video memory capacity of the target virtual GPU, the number of physical GPUs is greater. In the case of one, the physical GPU arranged at the forefront among the physical GPUs with the least number of tasks and the remaining video memory capacity meeting the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated by the target virtual GPU.

In an optional implementation manner, the instructions in the program are also used to perform the following operations: update the resource state table based on the physical GPU allocated to the target virtual GPU; or, when the task processing state changes In the case of updating the resource status table based on the change of the task processing status.

In an optional implementation manner, the instructions in the program are also used to perform the following operations: perform task scheduling based on the task queue of each physical GPU in the multiple physical GPUs.

FIG. 10 is a block diagram of the functional unit composition of the resource management device 1000 involved in an embodiment of the present application. The resource management apparatus 1000 is applied to an electronic device, and the electronic device includes a processing unit 1010 and a communication unit 1020.

The processing unit 1010 is configured to: obtain a configuration file through the communication unit 1020; read a configuration file, the configuration file including the GPU configuration parameters of the guest device; based on the configuration file, generate at least one available A virtual GPU; and in response to receiving a task to be processed by a target user, a physical GPU is allocated to a target virtual GPU corresponding to the task to be processed, and the at least one available virtual GPU includes the target virtual GPU.

The embodiment of the application reads a configuration file, where the configuration file includes the parameters of the virtual GPU of the guest device; based on the configuration file, generates at least one available virtual GPU of the guest device; In this case, a physical GPU is allocated to the target virtual GPU corresponding to the task to be processed, and the at least one available virtual GPU includes the target virtual GPU. In the embodiment of the present application, a fixed mapping relationship is not established for the virtual GPU and the physical GPU in advance, but after the virtual GPU receives the task to be processed, the virtual GPU is dynamically assigned the corresponding physical GPU, which can be flexibly based on the current task The characteristics of the GPU resources are allocated to improve the resource utilization of the GPU and improve the overall performance of the GPU system.

In an optional implementation manner, in the case of assigning a physical GPU to the target virtual GPU corresponding to the task to be processed, the processing unit 1010 is specifically configured to: based on the video memory capacity required by the target virtual GPU and The resource state table allocates physical GPUs to the target virtual GPU, wherein the resource state table includes current state information of each of the multiple physical GPUs.

In an optional implementation manner, the current state information of each physical GPU includes: the video memory capacity of the physical GPU, the user information of each virtual GPU corresponding to the video memory occupied by the physical GPU, and the physical GPU The video memory capacity of each virtual GPU corresponding to the video memory that the GPU has already occupied.

In an optional implementation manner, in the case where a physical GPU is allocated to the target virtual GPU based on the required video memory capacity and the resource state table of the target virtual GPU, the processing unit 1010 is specifically configured to: The required video memory capacity and resource state table of the target virtual GPU allocates physical GPUs to the target virtual GPU, so that the video memory of the multiple physical GPUs is used at a minimum.

In an optional implementation manner, in the case where a physical GPU is allocated to the target virtual GPU based on the required video memory capacity and resource state table of the target virtual GPU, the processing unit 1010 is specifically configured to: According to the resource state table, the remaining video memory capacity of each physical GPU is determined in turn according to the arrangement order of the multiple physical GPUs, until the determined remaining video memory capacity meets the required video memory capacity of the target virtual GPU; The physical GPU with the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated to the target virtual GPU.

In an optional implementation manner, the processing unit 1010 is further configured to: determine that the remaining video memory capacity of the physical GPU ranked last among the multiple physical GPUs does not meet the required video memory capacity of the target virtual GPU In this case, it is determined that the virtual GPU has no available physical resources.

In an optional implementation manner, in the case where a physical GPU is allocated to the target virtual GPU based on the display memory capacity and resource state table of the target virtual GPU, the processing unit 1010 is specifically configured to: The required video memory capacity and resource state table of the target virtual GPU, and the physical GPU is allocated to the target virtual GPU, so that at least one task of the target user is allocated to the multiple physical GPUs as evenly as possible, wherein The at least one task includes the task to be processed.

In an optional implementation manner, in the case where a physical GPU is allocated to the target virtual GPU based on the display memory capacity and resource state table of the target virtual GPU, the processing unit 1010 is specifically configured to: The resource status table determines the number of tasks currently allocated by each physical GPU of the multiple physical GPUs by the target user, and the remaining video memory capacity of each physical GPU; and minimizes the number of tasks and the remaining video memory capacity satisfies the The physical GPU with the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated to the target virtual GPU.

In an optional implementation manner, the processing unit 1010 is further configured to: when the number of tasks is the smallest and the remaining video memory capacity meets the required video memory capacity of the target virtual GPU, there are multiple physical GPUs. , Determining the foremost physical GPU among the plurality of physical GPUs with the least number of tasks and the remaining video memory capacity meeting the required video memory capacity of the target virtual GPU as the physical GPU allocated by the target virtual GPU.

In an optional implementation manner, the processing unit 1010 is further configured to: update the resource state table based on the physical GPU allocated to the target virtual GPU; or, when the task processing state changes, The resource status table is updated based on the change in the task processing status.

FIG. 11 is a block diagram of the functional unit composition of a resource management device 1100 involved in another embodiment of the present application. The resource management apparatus 1100 is applied to electronic equipment, and the electronic equipment includes a processing unit 1110 and a communication unit 1120.

The processing unit 1110 is configured to: obtain a host configuration file through the communication unit 1120; read the host configuration file, where the configuration file includes virtual GPU configuration parameters of the guest device; based on the configuration file, Generate at least one available virtual GPU of the guest device; and in the case of receiving a task to be processed by a target user, send a resource allocation to the host through the communication unit 1120 based on the target virtual GPU corresponding to the task to be processed A request, wherein the resource allocation request is used to request the host to allocate a physical GPU to the target virtual GPU, and the at least one available virtual GPU includes the target virtual GPU.

The embodiment of the application reads a configuration file, where the configuration file includes the parameters of the virtual GPU of the guest device; based on the configuration file, generates at least one available virtual GPU of the guest device; In this case, the communication unit sends a resource allocation request to the host based on the target virtual GPU corresponding to the task to be processed. In the embodiment of this application, a fixed mapping relationship is not established for the virtual GPU and the physical GPU in advance. Instead, after the virtual GPU receives the task, it sends a resource allocation request to the host to improve the resource utilization of the physical GPU and increase the GPU. The overall performance of the system.

In an optional implementation manner, before the target virtual GPU corresponding to the task to be processed sends a resource allocation request to the host, the processing unit 1110 is further configured to: determine the guest device based on the resource status table A virtual GPU in an idle state among at least one of the available virtual GPUs, wherein the resource state table includes current state information of each physical GPU in a plurality of physical GPUs; the target is determined from the virtual GPU in the idle state Virtual GPU.

In an optional implementation manner, the processing unit 1110 is further configured to: receive a first update instruction regarding the resource status table from the host, where the first update instruction is detected on the host Sent when the task processing state of the physical GPU changes; based on the first update instruction, the resource state table is updated.

In an optional implementation manner, the first update instruction carries at least one of the following information: user information corresponding to the newly added task of the first physical GPU, information of the first physical GPU, and the new The video memory capacity of the virtual GPU corresponding to the added task, the user information corresponding to the completed task, and the information of the second physical GPU corresponding to the completed task, where the first physical GPU is the GPU newly added to the task , The second physical GPU is a GPU that completes the task.

In an optional implementation manner, the processing unit 1110 is further configured to: schedule the target virtual GPU to execute the task to be processed; and receive a processing result of the task to be processed from the host.

FIG. 12 is a block diagram of the functional unit composition of a resource management device 1200 involved in another embodiment of the present application. The resource management apparatus 1200 is applied to an electronic device, and the electronic device includes a processing unit 1210 and a communication unit 1220. The processing unit 1210 is configured to receive a resource allocation request from a visitor device through the communication unit 1220. The resource allocation request carries information of the task to be processed and information of the target virtual GPU; and based on the information of the target virtual GPU, a target physical GPU is allocated to the target virtual GPU.

The embodiment of the application receives a resource allocation request from a guest device, the resource allocation request carries information about the task to be processed and information about the target virtual GPU; and based on the information about the target virtual GPU, allocates a target physical to the target virtual GPU GPU. In the embodiment of this application, a fixed mapping relationship is not established for the virtual GPU and the physical GPU in advance, but after receiving the resource allocation request from the guest device, the virtual GPU is dynamically allocated the corresponding physical GPU, which can be flexibly based on the current task The characteristics of the GPU resources are allocated to improve the resource utilization of the physical GPU and improve the overall performance of the GPU system.

In an optional implementation manner, in the case where a target physical GPU is allocated to the target virtual GPU based on the information of the target virtual GPU, the processing unit 1210 is specifically configured to: For the required video memory capacity, physical GPUs are allocated to the target virtual GPU.

In an optional implementation manner, in the case where a physical GPU is allocated to the task to be processed based on the memory capacity of the target virtual GPU, the processing unit 1210 is specifically configured to: The required video memory capacity, resource allocation strategy, and resource state table allocate physical GPUs to the target virtual GPU, where the resource state table includes current state information of each of the multiple physical GPUs.

In an optional implementation manner, in the case where a physical GPU is allocated to the target virtual GPU based on the video memory capacity of the target virtual GPU, the resource allocation strategy, and the resource state table, the processing unit 1210 specifically uses Yu: Based on the resource state table, determine the remaining video memory capacity of each physical GPU in sequence according to the arrangement order of the multiple physical GPUs, until the determined remaining video memory capacity meets the required video memory capacity of the target virtual GPU; The physical GPU whose remaining video memory capacity meets the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated to the target virtual GPU.

In an optional implementation manner, the processing unit 1210 is further configured to determine that the remaining video memory capacity of the physical GPU ranked last among the multiple physical GPUs does not meet the required video memory capacity of the target virtual GPU. In this case, it is determined that the virtual GPU has no available physical resources.

In an optional implementation manner, in the case where a physical GPU is allocated to the target virtual GPU based on the video memory capacity of the target virtual GPU, the resource allocation strategy, and the resource state table, the processing unit 1210 specifically uses Yu: Based on the resource status table, determine the number of tasks that the target user currently has allocated to each physical GPU among multiple physical GPUs, and the remaining video memory capacity of each physical GPU, where the target user is the target virtual The user corresponding to the GPU; the physical GPU with the least number of tasks and the remaining video memory capacity meeting the required video memory capacity of the target virtual GPU is determined as the physical GPU of the target virtual GPU.

In an optional implementation manner, the processing unit 1210 is further configured to: when the number of tasks is the smallest and the remaining video memory capacity meets the required video memory capacity of the target virtual GPU, there are multiple physical GPUs. , Determining the physical GPU arranged at the forefront among the physical GPUs with the least number of tasks and the remaining video memory capacity meeting the required video memory capacity of the target virtual GPU as the physical GPU allocated by the target virtual GPU.

In an optional implementation manner, the processing unit 1210 is further configured to: update the resource state table based on the physical GPU allocated to the target virtual GPU; or, when the task processing state changes, The resource status table is updated based on the change in the task processing status.

In an optional implementation manner, the processing unit 1210 is further configured to perform task scheduling based on the task queue of each physical GPU in the multiple physical GPUs.

An embodiment of the present application also provides a computer-readable recording medium, wherein the computer-readable recording medium stores a computer program for use, and the computer program enables the computer to execute part or all of any of the methods described in the above-mentioned method embodiments Step, the above-mentioned computer includes electronic equipment.

An embodiment of the application also provides a computer program product. The computer program product includes a non-transitory computer-readable recording medium storing a computer program, and the computer program is operable to cause the computer to execute any of the methods described in the foregoing method embodiments. Part or all of the steps of the method. The computer program product may be a software installation package, and the computer includes electronic equipment.

It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that this application is not limited by the described sequence of actions. Because according to this application, some steps can be performed in other order or at the same time. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided in this application, it should be understood that the disclosed device may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or integrated into Another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical or other forms.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be realized either in the form of hardware or in the form of software functional unit.

If the above-mentioned integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage device. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage device. A number of instructions are included to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above-mentioned methods in each embodiment of the present application. The aforementioned storage devices include: flash drives, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), flash drives, magnetic disks, or optical disks, etc., which can store program codes Medium.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage device. The storage device can include: Flash memory, read-only memory, random access memory, floppy disk or CD-ROM, etc.

The embodiments of the application are described in detail above, and specific examples are used in this article to illustrate the principles and implementation of the application. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the application; at the same time, for Those of ordinary skill in the art, based on the idea of the application, will have changes in the specific implementation and the scope of application. In summary, the content of this specification should not be construed as a limitation to the application.

110-1, 110-n: virtual machine 101: Guest device 102: host 121: configuration file 122: virtual GPU mirroring 123: Task 124, vGPU, vGPU2: virtual GPU 130: Virtual GPU-Physical GPU resource allocation strategy 140, pGPU, pGPU1: physical GPU 201~203, 501~503, 601~602: steps VPT: Resource Status Table 700, 800, 900: electronic equipment 710, 810, 910: Processor 720, 820, 920: storage device 721, 821, 921: One or more programs 730, 830, 930: communication interface 740, 940: Physical GPU 1000, 1100, 1200: resource management device 1010, 1110, 1210: processing unit 1020, 1120, 1220: communication unit

Fig. 1 is a schematic diagram of a resource management system provided by an embodiment of the present application. Fig. 2 is a schematic flowchart of a resource management and allocation method disclosed in an embodiment of the present application. Fig. 3 is a structural diagram of a minimum resource management allocation strategy provided by an embodiment of the present application. Fig. 4 is a structural diagram of a load balancing distribution strategy provided by an embodiment of the present application. Fig. 5 is a schematic flowchart of a resource management and allocation method disclosed in another embodiment of the present application. Fig. 6 is a schematic flowchart of a resource management and allocation method disclosed in another embodiment of the present application. Fig. 7 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. FIG. 8 is a schematic structural diagram of an electronic device provided by another embodiment of the present application. FIG. 9 is a schematic structural diagram of an electronic device according to another embodiment of the present application. FIG. 10 is a block diagram of the functional unit composition of a resource management device involved in an embodiment of the present application. FIG. 11 is a block diagram of the functional unit composition of a resource management device involved in another embodiment of the present application. FIG. 12 is a block diagram of the functional unit composition of a resource management device involved in another embodiment of the present application.

201~203: Steps

Claims (18)

A resource management method, the method includes: Reading a configuration file, the configuration file including virtual image processing unit (GPU) configuration parameters of the guest device; Generating at least one available virtual GPU of the guest device based on the configuration file; and In response to receiving the task to be processed by the target user, a physical GPU is allocated to the target virtual GPU corresponding to the task to be processed, and the at least one available virtual GPU includes the target virtual GPU. The method according to claim 1, wherein the step of allocating the physical GPU to the target virtual GPU corresponding to the task to be processed includes: Based on the required video memory capacity of the target virtual GPU and a resource state table, the physical GPU is allocated to the target virtual GPU, wherein the resource state table includes the information of each of the multiple physical GPUs. Current status information. The method according to claim 2, wherein the step of allocating the physical GPU to the target virtual GPU based on the required video memory capacity of the target virtual GPU and the resource state table includes: Based on the required video memory capacity of the target virtual GPU and the resource state table, the physical GPU is allocated to the target virtual GPU, so that the video memory of the multiple physical GPUs is used at a minimum. The method according to any one of claims 2 to 3, wherein the step of allocating the physical GPU to the target virtual GPU based on the required video memory capacity of the target virtual GPU and the resource state table includes: Based on the resource state table, according to the arrangement order of the multiple physical GPUs, determine the remaining video memory capacity of each of the multiple physical GPUs in turn until the determined remaining video memory capacity meets the required video memory capacity of the target virtual GPU ;as well as A physical GPU whose remaining video memory capacity meets the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated by the target virtual GPU. The method according to claim 2, wherein the step of allocating the physical GPU to the target virtual GPU based on the required video memory capacity of the target virtual GPU and the resource state table includes: Based on the required video memory capacity of the target virtual GPU and the resource state table, the physical GPU is allocated to the target virtual GPU, so that at least one task of the target user is allocated to the multiple tasks as evenly as possible. A physical GPU, wherein the at least one task includes the task to be processed. The method according to claim 2 or 5, wherein the step of allocating the physical GPU to the target virtual GPU based on the required video memory capacity of the target virtual GPU and the resource state table includes: Based on the resource status table, determine the number of tasks currently allocated by the target user to each of the multiple physical GPUs in the multiple physical GPUs, and the remaining video memory capacity of each of the multiple physical GPUs; and The physical GPU with the least number of tasks and the remaining video memory capacity meeting the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated by the target virtual GPU. A resource management method applied to a visitor device, the method includes: Reading a configuration file of the host, where the configuration file includes virtual GPU configuration parameters of the guest device; Generating at least one available virtual GPU of the guest device based on the configuration file; and In the case of receiving the task to be processed by the target user, based on the target virtual GPU corresponding to the task to be processed, a resource allocation request is sent to the host, where the resource allocation request is used to request the host to be the target The virtual GPU allocates a physical GPU, and the at least one available virtual GPU includes the target virtual GPU. The method according to claim 7, wherein before the step of sending the resource allocation request to the host based on the target virtual GPU corresponding to the task to be processed, the method further includes: Determine the virtual GPU in the idle state among the at least one available virtual GPU of the guest device based on the resource state table, wherein the resource state table includes current state information of each of the multiple physical GPUs; as well as The target virtual GPU is determined from the virtual GPUs in the idle state. The method according to claim 7 or 8, wherein the method further includes: Scheduling the target virtual GPU to execute the task to be processed; and Receiving the processing result of the to-be-processed task from the host. A resource management method, applied to a host, and the method includes: Receiving a resource allocation request from the guest device, the resource allocation request carrying information about the task to be processed and information about the target virtual GPU; and Based on the information of the target virtual GPU, a target physical GPU is allocated to the target virtual GPU. The method according to claim 10, wherein the step of allocating the target physical GPU to the target virtual GPU based on the information of the target virtual GPU includes: The physical GPU is allocated to the target virtual GPU based on the required video memory capacity, resource allocation strategy, and resource state table of the target virtual GPU, wherein the resource state table includes each of the plurality of physical GPUs. The current status information of the physical GPU. The method according to claim 11, wherein the step of allocating the physical GPU to the target virtual GPU based on the required video memory capacity of the target virtual GPU, the resource allocation strategy, and the resource state table includes: Based on the resource state table, according to the arrangement order of the multiple physical GPUs, determine the remaining video memory capacity of each of the multiple physical GPUs in turn until the determined remaining video memory capacity meets the required video memory capacity of the target virtual GPU ;as well as A physical GPU whose remaining video memory capacity meets the required video memory capacity of the target virtual GPU is determined as the physical GPU allocated by the target virtual GPU. The method according to claim 11, wherein the step of allocating the physical GPU to the target virtual GPU based on the required video memory capacity of the target virtual GPU, the resource allocation strategy, and the resource state table includes: Based on the resource state table, determine the number of tasks that the target user currently has allocated to each of the multiple physical GPUs in the multiple physical GPUs, and the remaining video memory capacity of each of the multiple physical GPUs, where all The target user is a user corresponding to the target virtual GPU; and The physical GPU with the smallest number of tasks and the remaining video memory capacity meeting the required video memory capacity of the target virtual GPU is determined as the physical GPU of the target virtual GPU. A resource management device includes a communication unit and a processing unit, wherein the processing unit is used for: Obtaining a configuration file through the communication unit; Reading the configuration file, the configuration file including virtual image processing unit (GPU) configuration parameters of the guest device; Generating at least one available virtual GPU of the guest device based on the configuration file; and In response to receiving the task to be processed by the target user, a physical GPU is allocated to the target virtual GPU corresponding to the task to be processed, and the at least one available virtual GPU includes the target virtual GPU. A resource management device includes a communication unit and a processing unit, wherein the processing unit is used for: Obtaining the configuration file of the host through the communication unit; Reading the configuration file, where the configuration file includes virtual GPU configuration parameters of the guest device; Generating at least one available virtual GPU of the guest device based on the configuration file; and In the case of receiving the task to be processed by the target user, the communication unit sends a resource allocation request to the host based on the target virtual GPU corresponding to the task to be processed, wherein the resource allocation request is used to request the The host allocates a physical GPU to the target virtual GPU, and the at least one available virtual GPU includes the target virtual GPU. A resource management device includes a communication unit and a processing unit, wherein the processing unit is used for: Receiving a resource allocation request from the guest device through the communication unit, the resource allocation request carrying information about the task to be processed and information about the target virtual GPU; and Based on the information of the target virtual GPU, a target physical GPU is allocated to the target virtual GPU. An electronic device, comprising a processor and a storage device, the storage device is used to store a computer program, the computer program is configured to be executed by the processor, so that the processor executes any of claims 1-6 Item or any one of 7-9 or any one of 10-13. A computer-readable recording medium, the computer-readable recording medium is used to store a computer program, wherein the computer program enables the computer to execute any one of claims 1-6 or any one of 7-9 or 10- 13. The method of any one of them.

Images