WO2022041507A1

WO2022041507A1 - 3d rendering method and system

Info

Publication number: WO2022041507A1
Application number: PCT/CN2020/130407
Authority: WO
Inventors: 刘威; 卢涛
Original assignee: 西安万像电子科技有限公司
Priority date: 2020-08-28
Filing date: 2020-11-20
Publication date: 2022-03-03
Also published as: CN112102457A

Abstract

A 3D rendering method and system, which relate to the field of a virtual cloud desktop technology, and can solve the problem that system performance is affected due to the fact that a virtual GPU needs to frequently perform operations such as rendering context switching, synchronization and queuing when a plurality of 3D rendering tasks access the same public resource. The specific technical solution comprises: a virtual machine (401) acquiring 3D rendering instructions sent by at least one 3D application, and sending each 3D rendering instruction to a virtual GPU (403) corresponding to the 3D application, wherein each 3D application corresponds to one independent virtual GPU (403); the virtual GPUs (403) parsing the received 3D rendering instructions to obtain IR data, and sending the IR data to a host computer (402); and the host computer (402) generating a target rendering instruction according to the IR data, and sending the target rendering instruction to a target physical GPU (4021), so that the target physical GPU (4021) executes the target rendering instruction. The present invention is used for 3D rendering.

Description

3D rendering method and system

technical field

The present application relates to the technical field of virtual cloud desktops, and in particular, to a 3D rendering method and system.

Background technique

With the rapid development of cloud virtualization technology, public cloud platforms and private cloud platforms have been further developed, which puts forward further segmentation requirements for virtual desktop cloud platforms. At present, according to the user's usage behavior, the virtual cloud desktop is gradually refined into cloud desktop office users, 2D/3D design and development users, and 3D game entertainment users. Under the current technical conditions, in response to the needs of cloud desktop office users, the desktop cloud has proposed many solutions, such as QEMU (virtual operating system simulator) combined with the spice protocol open source project. For 2D/3D design and development and 3D game entertainment users, virtual graphics cards that can render 3D applications need to be virtualized from the cloud.

At present, there are generally three solutions for the 3D rendering requirements of virtual machines:

The first is to install multiple physical GPU (Graphics Processing Unit, graphics processor) graphics cards in the cloud host, and use Passthrough technology to map a single physical GPU graphics card to each customer's virtual machine as the virtual machine's physical graphics card The schematic diagram of the principle can be seen in Figure 1. The virtual machine realizes the rendering of 3D instructions through Passthrough to the physical graphics card of the virtual machine.

The second is to install a single physical GPU graphics card in the cloud host, and use vGPU (virtual GPU) technology to virtualize a single physical GPU graphics card into multiple vGPU graphics cards, and map these multiple vGPUs one by one to the guest's virtual machine as The virtual graphics card is used, and its schematic diagram can be seen in Figure 2. The virtual machine realizes the rendering of 3D instructions through the vGPU graphics card.

The third is the virtual machine virtual graphics card scheme. Each virtual machine sends a rendering request to the host through its own virtual GPU graphics card, and the host uses the physical GPU graphics card to complete the real rendering. The schematic diagram can be seen in Figure 3.

In the above solutions, the first solution is simple to implement, but each virtual machine must bind the GPU hardware and use it exclusively when it is started, and the GPU may not be used at all in actual operation, resulting in GPU resources. waste. The technical implementation of the second solution is relatively complex, requiring GPU hardware support, and this function is generally limited to specific models of GPUs from some specific manufacturers, which is expensive and expensive to use. The third solution is the most flexible. The virtual machine does not need to be bound to a fixed GPU hardware during startup and operation, and the host does not need to select a specific type of GPU hardware from a specific manufacturer. It is also optimal from cost considerations. Therefore, the third solution The scheme has been widely used.

But for the third solution, when the virtual machine runs multiple 3D applications at the same time, the virtual GPU needs to manage multiple rendering contexts, each 3D application needs to use the virtual GPU exclusively, and inside the virtual GPU, all resources are public , each 3D application initiates a rendering request to the virtual GPU through its own rendering context, and the virtual GPU responds to the rendering request of different applications by identifying different rendering contexts. In this way, when multiple rendering tasks access the same common resource, the virtual GPU needs to frequently perform operations such as rendering context switching, synchronization, and queuing, thereby affecting the performance of the system.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a 3D rendering method and system, which can solve the problem that the virtual GPU needs to frequently perform operations such as rendering context switching, synchronization, and queuing when multiple 3D rendering tasks access the same common resource, which affects system performance. The technical solution is as follows:

According to a first aspect of the embodiments of the present application, a 3D rendering method is provided, the method comprising:

The virtual machine obtains a 3D rendering instruction sent by at least one 3D application;

The virtual machine sends each 3D rendering instruction to a virtual graphics processor GPU corresponding to the 3D application, wherein each 3D application corresponds to an independent virtual GPU;

The virtual GPU parses the received 3D rendering instruction, obtains intermediate representation IR data, and sends the IR data to the host;

The host computer generates a target rendering instruction according to the IR data, and sends the target rendering instruction to the target physical GPU, so that the target physical GPU executes the target rendering instruction.

In the 3D rendering method provided by the embodiment of the present application, an independent virtual GPU is set for each 3D application, the virtual machine can send the 3D rendering instruction sent by each 3D application to the virtual GPU uniquely corresponding to the 3D application, and the virtual GPU can receive the The received 3D rendering instruction is parsed, and the obtained IR data is sent to the host. The host generates a target rendering instruction according to the IR data, and sends the target rendering instruction to the target physical GPU, so that the target physical GPU can execute the target rendering. instruction. In this way, one virtual GPU is only responsible for managing the rendering context of one 3D application, thereby avoiding the problem that the virtual GPU needs to frequently perform context switching, synchronization, queuing, etc. to affect performance due to the management of multiple rendering contexts by one virtual GPU.

In one embodiment, the virtual machine sending each 3D rendering instruction to the virtual graphics processor GPU corresponding to the 3D application includes:

The virtual machine determines whether a corresponding virtual GPU is created for the 3D application;

If it has been created, the 3D rendering instruction is sent to the virtual GPU corresponding to the 3D application.

In one embodiment, after the virtual machine determines whether a corresponding virtual GPU is created for the 3D application, the method further includes:

If the corresponding virtual GPU is not created for the 3D application, create a corresponding virtual GPU for the 3D application;

Send the 3D rendering instruction to the virtual GPU corresponding to the 3D application.

In the 3D rendering method provided by the embodiment of the present application, after the virtual machine obtains the 3D rendering instruction sent by the 3D application, it first determines whether a corresponding virtual GPU has been created for the 3D application, and if so, sends the 3D rendering instruction to the 3D application. The virtual GPU corresponding to the application, if not created, first create the corresponding virtual GPU for the 3D application, and then send the 3D rendering instruction to the virtual GPU corresponding to the 3D application. It is guaranteed that each 3D application has an independent virtual GPU.

In one embodiment, before the sending the target rendering instruction to the target physical GPU, the method further includes:

The host computer selects a target physical GPU from at least one physical GPU according to a preset rule.

In one embodiment, the host computer selects a target physical GPU from at least one physical GPU according to a preset rule, including:

The host selects the physical GPU with the lowest utilization rate as the target physical GPU from at least one physical GPU managed by the host.

In the 3D rendering method provided by the embodiment of the present application, the host can select the physical GPU with the lowest utilization rate from at least one physical GPU managed by it as the target physical GPU, and then send the target rendering instruction to the target physical GPU, so that the target physical GPU can The GPU executes the target rendering instruction. In this way, 3D applications can be distinguished as the smallest particles, and the rendering instructions of each 3D can be distributed to different physical GPU hardware, thus avoiding that one virtual machine only corresponds to one physical GPU, and the virtual machine runs multiple 3D applications at the same time. The problem caused by the virtual machine corresponding to the physical GPU is overburdened.

According to a second aspect of the embodiments of the present application, a 3D rendering system is provided, the 3D rendering system includes a virtual machine, a host machine, at least one virtual graphics processing unit (GPU), and at least one physical GPU, and at least one physical GPU runs on the virtual machine. 3D applications, each 3D application corresponds to an independent virtual GPU, and the host machine includes at least one physical GPU;

The virtual machine is configured to obtain a 3D rendering instruction sent by at least one 3D application, and send each 3D rendering instruction to a virtual GPU corresponding to the 3D application;

The virtual GPU is used to parse the received 3D rendering instruction, obtain intermediate representation IR data, and send the IR data to the host;

The host machine is used to generate a target rendering instruction according to the IR data, and send the target rendering instruction to the target physical GPU;

The physical GPU is used to execute the received target rendering instruction.

The 3D rendering system provided by the embodiments of the present application includes a virtual machine, a host machine, at least one virtual GPU, and at least one physical GPU. At least one 3D application runs on the virtual machine, each 3D application corresponds to an independent virtual GPU, and the host machine Include at least one physical GPU. The system sets up an independent virtual GPU for each 3D application. The virtual machine can send the 3D rendering instructions sent by each 3D application to the unique virtual GPU corresponding to the 3D application, and the virtual GPU parses the received 3D rendering instructions. , and send the obtained IR data to the host, the host generates a target rendering instruction according to the IR data, and sends the target rendering instruction to the target physical GPU, so that the target physical GPU executes the target rendering instruction. In this way, one virtual GPU is only responsible for managing the rendering context of one 3D application, thereby avoiding the problem that the virtual GPU needs to frequently perform context switching, synchronization, queuing, etc. to affect performance due to the management of multiple rendering contexts by one virtual GPU.

In one embodiment, the virtual machine is specifically configured to acquire a 3D rendering instruction sent by at least one 3D application, determine whether a corresponding virtual GPU has been created for the 3D application, and if so, send the 3D rendering instruction Create a virtual GPU corresponding to the 3D application; if not, create a corresponding virtual GPU for the 3D application, and then send the 3D rendering instruction to the virtual GPU corresponding to the 3D application.

In this way, after obtaining the 3D rendering instruction sent by the 3D application, the virtual machine determines whether a corresponding virtual GPU has been created for the 3D application. The 3D rendering instruction is sent to the virtual GPU corresponding to the 3D application, which can ensure that each 3D application has an independent virtual GPU.

In one embodiment, the host is specifically configured to select a physical GPU from the at least one physical GPU according to a preset rule, obtain a target physical GPU, and send the target rendering instruction to the target physical GPU .

In one embodiment, the host is specifically configured to select a physical GPU with the lowest utilization rate from the at least one physical GPU as a target physical GPU.

In this way, the rendering instructions of multiple 3D applications of the same virtual machine can be distinguished by taking the 3D application as the smallest particle, and the rendering instructions of each 3D application can be distributed to different physical GPU hardware, which can prevent the virtual machine from running multiple 3D applications at the same time. The virtual machine's corresponding physical GPU is overburdened during rendering tasks for 3D applications.

In one embodiment, the virtual GPU includes a front-end driver and a back-end driver;

The front-end driver runs on the virtual machine, and is used to parse the received 3D rendering instruction, obtain intermediate representation IR data, and send the IR data to the back-end driver;

The back-end driver runs on the host, and is used for sending the IR data to the host.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not limiting of the present application.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application.

1 is a schematic diagram of 3D rendering by Passthrough technology in the prior art;

Fig. 2 is a schematic diagram of performing 3D rendering by virtualizing hardware GPU in the prior art;

Fig. 3 is a schematic diagram of performing 3D rendering through the solution of virtual machine virtual graphics card in the prior art;

4 is a schematic structural diagram of a 3D rendering system provided by an embodiment of the present application;

5 is a flowchart of a 3D rendering method provided by an embodiment of the present application;

6 is a schematic structural diagram of a specific 3D rendering system provided by an embodiment of the present application;

7 is a schematic structural diagram of another 3D rendering system provided by an embodiment of the present application;

FIG. 8 is a flowchart of a specific 3D rendering method provided by an embodiment of the present application.

detailed description

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as recited in the appended claims.

In this embodiment of the present application, the 3D application running on the virtual machine is used as the minimum rendering particle, and an independent virtual GPU is set for each 3D application, and the virtual GPU can issue the rendering instructions of the 3D application to the selected physical object. GPU to execute.

An embodiment of the present application provides a 3D rendering system. As shown in FIG. 4 , the 3D rendering system 40 may include a virtual machine 401, a host machine 402, at least one virtual GPU 403 and at least one physical GPU 4021, wherein the host machine 402 includes At least one physical GPU 4021, such as M (M is an integer greater than or equal to 1) physical GPUs 4021; at least one 3D application runs on the virtual machine 401, and each 3D application corresponds to an independent virtual GPU, as shown in FIG. 4 for example There are N (N is an integer greater than or equal to 1) 3D applications: 3D application 1, 3D application 2... 3D application N, 3D application 1 corresponds to virtual GPU1, 3D application 2 corresponds to virtual GPU2, ... 3D application N Corresponds to virtual GPU N.

Specifically, the virtual machine 401 is configured to acquire a 3D rendering instruction sent by at least one 3D application, and send each 3D rendering instruction to a virtual GPU corresponding to the 3D application. For example, in FIG. 4 , the virtual machine 401 can obtain the 3D rendering instructions sent by 3D application 1, 3D application 2... The 3D rendering instruction sent by the 3D application N is sent to the virtual GPU2, ..., and so on, the 3D rendering instruction sent by the 3D application N is sent to the virtual GPUN.

The virtual GPU 403 is used to parse the received 3D rendering instruction, obtain IR (Intermediate Representation, intermediate representation) data, and send the IR data to the host machine 402.

The host 402 is configured to generate a target rendering instruction according to the IR data, and send the target rendering instruction to the target physical GPU; the physical GPU is configured to execute the received target rendering instruction.

For example, in FIG. 4, the virtual GPU1 receives the 3D rendering instruction of the 3D application 1 sent by the virtual machine 401, then parses the 3D rendering instruction to obtain IR data, and then sends the IR data to the host machine 402, and the host machine 402 Generate a target rendering instruction according to the IR data, and send the target rendering instruction to the target physical GPU, for example, to the physical GPU1, and the physical GPU1 executes the target rendering instruction.

In one embodiment, the virtual machine 401 is specifically configured to obtain a 3D rendering instruction sent by at least one 3D application, determine whether a corresponding virtual GPU has been created for the 3D application, and if so, send the 3D rendering instruction of the 3D application. A virtual GPU corresponding to the 3D application is created; if not, a corresponding virtual GPU is created for the 3D application, and then the 3D rendering instruction of the 3D application is sent to the created virtual GPU corresponding to the 3D application.

In one embodiment, the host 402 is specifically configured to select a physical GPU from at least one physical GPU according to a preset rule, obtain a target physical GPU, and send a target rendering instruction to the target physical GPU.

In one embodiment, the host 402 may select a physical GPU with the lowest utilization rate as the target physical GPU from at least one physical GPU managed by the host machine 402 .

In one embodiment, virtual GPU 403 may include front-end drivers and back-end drivers. The front-end driver runs on the virtual machine 401 to parse the received 3D rendering instructions, obtain IR data, and send the IR data to the back-end driver; the back-end driver runs on the host machine 402 and uses to send the received IR data to the host computer 402 .

The 3D rendering system provided by the embodiments of the present application includes a virtual machine, a host machine, at least one virtual GPU, and at least one physical GPU. At least one 3D application runs on the virtual machine, each 3D application corresponds to an independent virtual GPU, and the host machine Include at least one physical GPU. The system takes the 3D application of the virtual machine as the minimum rendering particle, and sets up an independent virtual GPU for each 3D application. The virtual machine can send the 3D rendering instructions sent by each 3D application to the unique virtual GPU corresponding to the 3D application. The virtual GPU parses the received 3D rendering instruction, and sends the obtained IR data to the host. The host generates a target rendering instruction according to the IR data, and sends the target rendering instruction to the target physical GPU, so that the target physical GPU can Execute the target rendering instruction. In this way, one virtual GPU is only responsible for managing the rendering context of one 3D application, thereby avoiding the problem that the virtual GPU needs to frequently perform context switching, synchronization, queuing, etc. to affect performance due to the management of multiple rendering contexts by one virtual GPU. Further, the virtual GPU can issue the 3D rendering instructions of the 3D application to the selected physical GPU, and the selected physical GPU performs the rendering process, and can select the appropriate physical GPU for each 3D application, no longer a virtual machine. It can only correspond to one physical GPU, thereby avoiding the problem that the physical GPU corresponding to the virtual machine is overburdened when the virtual machine runs 3D rendering of multiple 3D applications at the same time.

Based on the 3D rendering system of the above embodiment corresponding to FIG. 1 , an embodiment of the present application further provides a 3D rendering method, which can be at least applied to the 3D rendering system of the above embodiment corresponding to FIG. 4 . As shown in Figure 5, the method may include the following steps:

Step 501: Obtain a 3D rendering instruction sent by at least one 3D application.

At least one 3D application may run on the virtual machine, and when the 3D application initiates a 3D rendering instruction, the virtual machine obtains the 3D rendering instruction sent by the at least one 3D application.

Step 502: Send each 3D rendering instruction to the virtual GPU corresponding to the 3D application.

After acquiring the 3D rendering instruction sent by at least one 3D application, the virtual machine sends each 3D rendering instruction to a virtual GPU corresponding to the 3D application, wherein each 3D application corresponds to an independent virtual GPU.

For example, there are three 3D applications running on the virtual machine, namely 3D application 1, 3D application 2 and 3D application 3, and the three 3D applications are in one-to-one correspondence with virtual GPU1, virtual GPU2 and virtual GPU3. These three 3D applications respectively initiate 3D rendering instructions at runtime, and the virtual machine sends the 3D rendering instructions initiated by 3D application 1 to virtual GPU1, and the 3D rendering instructions initiated by 3D application 2 to virtual GPU2, and the 3D application 3 The 3D rendering command initiated by 3 is sent to the virtual GPU3.

Step 503: Parse the 3D rendering instruction to obtain IR data.

For each virtual GPU, after receiving the 3D rendering instruction sent by the virtual machine, the 3D rendering instruction is parsed to obtain IR data, and the IR data is sent to the host computer. By parsing 3D rendering instructions into IR data, differences in operating systems and rendering environments between the virtual machine and the host can be shielded.

Step 504: Generate a target rendering instruction according to the IR data.

After receiving the IR data sent by the virtual GPU, the host computer generates a target rendering instruction according to the IR data. In this way, the IR data can be restored to the data format supported by the host, and the target rendering instruction can be obtained.

Step 505: Send the target rendering instruction to the target physical GPU.

After the host computer generates the target rendering instruction according to the IR data, it sends the target rendering instruction to the target physical GPU, so that the target physical GPU executes the target rendering instruction.

Specifically, the host can manage at least one physical GPU, and after generating the target rendering instruction according to the IR data, the host can select the target physical GPU from the at least one physical GPU according to a preset rule. For example, the target physical GPU can be selected from the at least one physical GPU. Among the GPUs, the physical GPU with the lowest utilization rate is selected as the target physical GPU. In this way, the rendering instructions of multiple 3D applications of the same virtual machine can be sent to different physical GPUs, and a virtual machine no longer corresponds to only one physical GPU, which can avoid the virtual machine running multiple heavily rendered 3D applications at the same time. This virtual machine corresponds to the problem that the physical GPU is overburdened.

In the 3D rendering method provided by the embodiment of the present application, each 3D application running on a virtual machine corresponds to an independent virtual GPU, the virtual machine obtains a 3D rendering instruction sent by at least one 3D application, and sends each 3D rendering instruction to a The virtual GPU corresponding to the 3D application; the virtual GPU parses the received 3D rendering instruction, obtains IR data, and sends the IR data to the host; the host generates a target rendering instruction according to the IR data, and renders the target Instructions are sent to the target physical GPU so that the target physical GPU executes the received target rendering instructions. Since a virtual GPU is only responsible for managing the rendering context of one 3D application, rather than managing multiple rendering contexts, it can avoid the problem that the virtual GPU needs to perform context switching, synchronization, queuing and other operations frequently and affect the system performance, and improves the 3D performance. Rendering system performance.

In order to more clearly reflect the purpose of the present application, further examples are given on the basis of the above embodiments.

Based on the 3D rendering system of the embodiment corresponding to FIG. 4 , another embodiment of the present application provides a specific 3D rendering system. As shown in FIG. 6 , the 3D rendering system 40 includes a virtual machine 401 , a host machine 402 , and at least one virtual machine. GPU 403 and at least one physical GPU 4021. Wherein, at least one 3D application runs on the virtual machine 401, and each 3D application corresponds to an independent virtual GPU 403. For example, in FIG. 6, N (N is an integer greater than or equal to 1) 3D applications are shown: 3D application 1, 3D application 2, ..., 3D application N; 3D application 1 corresponds to virtual GPU1, and 3D application 2 corresponds to virtual GPU 1. GPU2, ..., 3D application N corresponds to virtual GPUN. In practical applications, the host machine 402 may be a server, on which the virtual machine 401 may run, and may be responsible for managing the physical GPU 4021.

The virtual machine 401 may include a 3D rendering portal module 4011, and at least one 3D application running on the virtual machine 401 is connected in communication with the 3D rendering portal module 4011. The host 402 may include a 3D rendering intermediate instruction module 4022 , a GPU distribution module 4023 and at least one physical GPU 4021 . For example, the host machine 402 may include M=4 physical GPUs 4021: physical GPU1, physical GPU2, physical GPU3, and physical GPU4. This is only an example and is not used to limit the application. In practical applications, it can be any number of physical GPUs. 4021.

When each 3D application on the virtual machine 401 is running, if a 3D rendering instruction is initiated, for example, the 3D rendering instruction may be a call of an OpenGL (Open Graphics Library) API (Application Programming Interface), then It will be captured by the 3D rendering entry module 4011 of the virtual machine 401 . Specifically, the capture mode of the 3D rendering entry module 4011 may be presented in the form of a dynamic library of 3D API functions. For example, if the virtual machine 401 runs a Linux system, the capture method of the 3D rendering entry module 4011 can be to replace the libGL.so dynamic library on the Linux system. In this way, all 3D applications developed based on OpenGL API load the libGL.so dynamic library What is actually loaded is the 3D rendering entry module 4011.

Further, after acquiring the 3D rendering instruction sent by at least one 3D application, the 3D rendering entry module 4011 can first determine whether a corresponding virtual GPU has been created for the 3D application, and if so, continue the rendering process, The 3D rendering instruction of the 3D application is sent to the virtual GPU corresponding to the 3D application; otherwise, a new virtual GPU is created for it, and then the 3D rendering instruction of each 3D application is sent to the virtual GPU corresponding to the 3D application.

For example, as shown in FIG. 6 , it is assumed that 3D application 1 and 3D application 2 respectively initiate 3D rendering instructions, and the 3D rendering instructions are captured by the 3D rendering entry module 4011. At this time, the 3D rendering entry module 4011 determines that it is already 3D application 1. When the corresponding virtual GPU1 is created, the 3D rendering instruction initiated by the 3D application 1 is sent to the virtual GPU1; the 3D rendering entry module 4011 also determines that the corresponding virtual GPU has not been created for the 3D application 2, and then creates the corresponding virtual GPU for the 3D application 2 first. virtual GPU2, and then send the 3D rendering instruction initiated by 3D application 2 to virtual GPU2.

The virtual GPU 403 is a virtual device connected to and running on the virtual machine and the host machine, which may include a front-end driver 4031 and a back-end driver 4032, the front-end driver 4031 runs on the virtual machine 401, and the back-end driver 4032 runs on the host machine 403 superior. The front-end driver 4031 can parse the received 3D rendering instruction, obtain IR data, and send the IR data to the back-end driver 4032; after the back-end driver 4032 receives the IR data sent by the front-end driver 4031, it sends the IR data To the host computer 402 , specifically, it may be the 3D rendering intermediate instruction module 4022 sent to the host computer 402 . By parsing the 3D rendering instructions into IR data, differences in operating systems and rendering environments between the virtual machine 401 and the host machine 402 can be shielded.

The 3D rendering intermediate instruction module 4022 of the host machine 402 can create a corresponding rendering context for each virtual GPU 403, and is responsible for parsing the IR data received from the front-end driver 4031 of the virtual GPU 403 to the target rendering supported by the local physical GPU 4021 instruction, the target rendering instruction can be a 3D API call, such as a Vulkan API call, where Vulkan is a cross-platform 2D and 3D drawing application program interface.

The GPU distribution module 4023 is responsible for managing the physical GPU 4021. When the target rendering instruction sent by the 3D rendering intermediate instruction module 4022 arrives, a suitable physical GPU 4021 can be selected for rendering the target rendering instruction. In one embodiment, the strategy for selecting a physical GPU may take the utilization rate of the physical GPU as a reference, and preferentially select a relatively idle physical GPU as the target physical GPU, that is, select the physical GPU with the lowest utilization rate as the target physical GPU. In this way, the rendering of the same rendering context, that is, the rendering of the same 3D application, can be executed on the same physical GPU.

The 3D rendering system provided by the embodiments of the present application can take the 3D application running on the virtual machine as the smallest rendering particle, and create a dedicated virtual GPU graphics card for each 3D application, that is, set an independent virtual GPU for each 3D application , the virtual GPU can deliver the rendering instruction of the 3D application to the selected target physical GPU for execution. In this way, the rendering instructions of multiple 3D applications of the same virtual machine can be distinguished by taking the 3D application as the smallest particle, and the rendering instructions of each 3D application can be distributed to different physical GPU hardware, thus avoiding that one virtual machine corresponds to only one The physical GPU and the virtual machine running multiple rendering 3D applications at the same time cause the virtual machine's corresponding physical GPU to be overburdened; moreover, in this solution, one virtual GPU is only responsible for managing the rendering context of one 3D application, which can avoid Because one virtual GPU manages multiple rendering contexts, the virtual GPU needs to frequently perform context switching, synchronization, queuing and other issues affecting performance, which improves the rendering performance of the system.

The above embodiment is described by taking one host machine corresponding to one virtual machine as an example. In practical applications, one host machine may also correspond to multiple virtual machines. For example, as shown in FIG. 7 , one host machine may correspond to two virtual machines. The two virtual machines can respectively run at least one 3D application, and for each virtual machine, each 3D application running on the virtual machine corresponds to an independent virtual GPU. The host can manage the 3D rendering instructions from the two virtual machines at the same time, and select the target physical GPU for each 3D rendering instruction.

Based on the 3D rendering system of the above embodiment corresponding to FIG. 6 , the embodiment of the present application further provides a specific 3D rendering method, which can be at least applied to the 3D rendering system of the above embodiment corresponding to FIG. 6 . As shown in Figure 8, the method may include the following steps:

Step 801: Obtain a 3D rendering instruction sent by at least one 3D application.

At least one 3D application can run on the virtual machine 401. When the 3D application on the virtual machine initiates a 3D rendering instruction, for example, when a 3D API call is initiated, the 3D rendering entry module 4011 on the virtual machine will capture the 3D rendering instruction.

Step 802: Determine whether a corresponding virtual GPU is created for the 3D application.

After obtaining the 3D rendering instruction sent by at least one 3D application, the 3D rendering entry module 4011 determines whether a corresponding virtual GPU node, that is, a virtual GPU graphics card, has been created for the 3D application. If it has been created, execute step 804, otherwise execute step 803 .

Step 803: Create a corresponding virtual GPU for the 3D application.

If the 3D rendering entry module 4011 determines that the corresponding virtual GPU has not been created for the 3D application, it will create a new virtual GPU node for the 3D application, and then step 804 is executed.

Step 804: Send the 3D rendering instruction to the virtual GPU corresponding to the 3D application.

The 3D rendering entry module 4011 determines that after creating a corresponding virtual GPU for the 3D application, or after creating a new virtual GPU for the 3D application, the 3D rendering instruction is sent to the virtual GPU corresponding to the 3D application.

The virtual GPU is a virtual device connected to and running on the virtual machine and the host, which may include a front-end driver and a back-end driver. The front-end driver runs on the virtual machine 401 and the back-end driver runs on the host 403 . Specifically, the 3D rendering entry module 4011 can send the 3D rendering instruction to the front-end driver of the virtual GPU corresponding to the 3D application.

For example, 3D application 1 and 3D application 2 running on the virtual machine 401 respectively initiate a 3D rendering instruction, and after capturing the 3D rendering instruction, the 3D rendering entry module 4011 determines whether the corresponding 3D application 1 and 3D application 2 have been created respectively. Virtual GPU, assuming that virtual GPU1 has been created for 3D application 1, but not for 3D application 2, the 3D rendering entry module 4011 will send the 3D rendering command initiated by 3D application 1 to the front-end driver of virtual GPU1, and will After creating the virtual GPU2 for the 3D application 2, the 3D rendering instruction initiated by the 3D application 2 is sent to the front-end driver of the virtual GPU2.

Step 805: Parse the 3D rendering instruction to obtain IR data.

For each virtual GPU, after receiving the 3D rendering instruction sent by the 3D rendering entry module 4011, the 3D rendering instruction is parsed to obtain IR data, and the IR data is sent to the host. By parsing 3D rendering instructions into IR data, differences in operating systems and rendering environments between the virtual machine and the host can be shielded.

Specifically, the front-end driver of the virtual GPU parses the 3D rendering instruction sent by the 3D rendering entry module 4011, generates IR data, and then sends the IR data to the back-end driver of the virtual GPU; the back-end driver of the virtual GPU then interprets the IR data. The IR data is sent to the 3D rendering intermediate instruction module 4022 of the host computer 402 .

Step 806: Generate a target rendering instruction according to the IR data.

After the 3D rendering intermediate instruction module 4022 of the host computer 402 receives the IR data sent by the virtual GPU, it generates a target rendering instruction according to the IR data. For example, the target rendering instruction can be generated by parsing the IR data, so that the IR data can be restored to the host computer. 402 Supported data format, get the target rendering instruction. Then, the 3D rendering intermediate instruction module 4022 sends the target rendering instruction to the GPU distribution module 4023 .

Step 807: Select a target physical GPU.

After the GPU distribution module 4023 receives the target rendering instruction sent by the 3D rendering intermediate instruction module 4022, it can select a target physical GPU from at least one physical GPU managed by it according to a preset rule. Specifically, the GPU distribution module 4023 may select the physical GPU with the lowest GPU utilization rate as the target physical GPU according to the utilization rate of each physical GPU managed by the host machine 402 .

Step 808: Send the target rendering instruction to the target physical GPU.

After the GPU distribution module 4023 selects the target physical GPU, it sends the target rendering instruction to the selected target physical GPU.

Step 809: Execute the target rendering instruction.

After the target physical GPU receives the target rendering instruction, it executes the target rendering instruction.

In this application, different 3D applications of the same virtual machine can be distributed to different physical GPUs; different 3D applications of different virtual machines can be distributed to the same physical GPU. Specifically, the selection strategy for distributing the physical GPUs may be to preferentially select the physical GPUs in an idle state, which can solve the problem of uneven busyness and idleness of multiple physical GPUs in the host.

Understandably, from the perspective of rendering data flow, all rendering instructions of a 3D application run by a virtual machine are always sent to a physical GPU of the host machine, until the rendering process ends when the 3D application exits, and the corresponding virtual GPU and physical GPU is released. The virtual GPU driver and the physical GPU driver can avoid frequently performing context switching, synchronization, queuing and other operations that affect performance, thereby improving rendering efficiency.

In the 3D rendering method provided by the embodiment of the present application, when a 3D application on a virtual machine initiates a 3D rendering instruction, the 3D rendering entry module captures the 3D rendering instruction, and determines whether a corresponding virtual GPU has been created for the 3D application, and if so, Then send the 3D rendering instruction to the virtual GPU corresponding to the 3D application; if not, create a corresponding virtual GPU for the 3D application, and then send the 3D rendering instruction to the virtual GPU corresponding to the 3D application; the virtual GPU parses the 3D rendering instruction Obtain IR data and send the IR data to the 3D rendering intermediate instruction module of the host; the 3D rendering intermediate instruction module generates a target rendering instruction according to the IR data, and sends the target rendering instruction to the GPU distribution module; the GPU distribution module selects The physical GPU with the lowest utilization is the target physical GPU, and the target rendering instruction is sent to the target physical GPU, so that the target physical GPU executes the target rendering instruction. It can distinguish the 3D rendering instructions of multiple 3D applications of the same virtual machine with the 3D application as the smallest particle, and distribute the 3D rendering instructions of each 3D application to different physical GPU hardware, no longer a virtual machine only corresponds to one In this way, when the virtual machine runs multiple 3D applications with heavy rendering at the same time, the problem that the physical GPU corresponding to the virtual machine is overburdened can be avoided. At the same time, a virtual GPU is only responsible for managing the rendering context of one 3D application, which can avoid the problem that the virtual GPU driver needs to frequently perform context switching, synchronization, queuing, etc. due to the management of multiple rendering contexts by one virtual GPU, which affects performance. Efficiency of 3D rendering.

Based on the 3D rendering methods described in the embodiments corresponding to FIG. 5 and FIG. 8 , the embodiments of the present application further provide a computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be a read-only memory (English) : Read Only Memory, ROM), random access memory (English: Random Access Memory, RAM), CD-ROM, magnetic tape, floppy disk and optical data storage devices, etc. The storage medium stores computer instructions for executing the 3D rendering methods described in the embodiments corresponding to FIG. 5 and FIG. 8 , and details are not described herein again.

Other embodiments of the present application will readily occur to those skilled in the art upon consideration of the specification and practice of what is disclosed herein. This application is intended to cover any variations, uses or adaptations of this application that follow the general principles of this application and include common knowledge or conventional techniques in the technical field not disclosed in this application . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the application being indicated by the appended claims.

It is to be understood that the present application is not limited to the precise structures described above and illustrated in the accompanying drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

A 3D rendering method, characterized in that the method comprises:

The virtual machine obtains a 3D rendering instruction sent by at least one 3D application;

The virtual machine sends each 3D rendering instruction to a virtual graphics processor GPU corresponding to the 3D application, wherein each 3D application corresponds to an independent virtual GPU;

The virtual GPU parses the received 3D rendering instruction, obtains intermediate representation IR data, and sends the IR data to the host;

The host computer generates a target rendering instruction according to the IR data, and sends the target rendering instruction to the target physical GPU, so that the target physical GPU executes the target rendering instruction.
The 3D rendering method according to claim 1, wherein the virtual machine sending each 3D rendering instruction to a virtual graphics processor (GPU) corresponding to the 3D application comprises:

The virtual machine determines whether a corresponding virtual GPU is created for the 3D application;

If it has been created, the 3D rendering instruction is sent to the virtual GPU corresponding to the 3D application.
The 3D rendering method according to claim 2, wherein after the virtual machine determines whether a corresponding virtual GPU is created for the 3D application, the method further comprises:

If the corresponding virtual GPU is not created for the 3D application, create a corresponding virtual GPU for the 3D application;

Send the 3D rendering instruction to the virtual GPU corresponding to the 3D application.
The 3D rendering method according to claim 1, wherein before the sending the target rendering instruction to the target physical GPU, the method further comprises:

The host computer selects a target physical GPU from at least one physical GPU according to a preset rule.
The 3D rendering method according to claim 4, wherein the host computer selects a target physical GPU from at least one physical GPU according to a preset rule, comprising:

The host selects the physical GPU with the lowest utilization rate as the target physical GPU from at least one physical GPU managed by the host.
A 3D rendering system, characterized in that it includes a virtual machine, a host machine, at least one virtual graphics processing unit (GPU) and at least one physical GPU, wherein at least one 3D application runs on the virtual machine, and each 3D application corresponds to an independent a virtual GPU, the host machine includes at least one physical GPU;

The virtual machine is configured to obtain a 3D rendering instruction sent by at least one 3D application, and send each 3D rendering instruction to a virtual GPU corresponding to the 3D application;

The virtual GPU is used to parse the received 3D rendering instruction, obtain intermediate representation IR data, and send the IR data to the host;

The host machine is used to generate a target rendering instruction according to the IR data, and send the target rendering instruction to the target physical GPU;

The physical GPU is used to execute the received target rendering instruction.
The 3D rendering system according to claim 6, wherein the virtual machine is specifically configured to obtain a 3D rendering instruction sent by at least one 3D application, and determine whether a corresponding virtual GPU has been created for the 3D application, and if a virtual GPU has been created , then send the 3D rendering instruction to the virtual GPU corresponding to the 3D application; if not, create a corresponding virtual GPU for the 3D application, and then send the 3D rendering instruction to the corresponding virtual GPU of the 3D application Virtual GPU.
The 3D rendering system according to claim 6, wherein the host is specifically configured to select a physical GPU from the at least one physical GPU according to a preset rule, obtain a target physical GPU, and convert the target physical GPU to the target physical GPU. Rendering instructions are sent to the target physical GPU.
The rendering system according to claim 8, wherein the host is specifically configured to select a physical GPU with the lowest utilization rate from the at least one physical GPU as the target physical GPU.
The 3D rendering system according to claim 6, wherein the virtual GPU comprises a front-end driver and a back-end driver;

The front-end driver runs on the virtual machine, and is used to parse the received 3D rendering instruction, obtain intermediate representation IR data, and send the IR data to the back-end driver;

The back-end driver runs on the host, and is used for sending the IR data to the host.