WO2021228031A1

WO2021228031A1 - Rendering method, apparatus and system

Info

Publication number: WO2021228031A1
Application number: PCT/CN2021/092699
Authority: WO
Inventors: 李力
Original assignee: 华为技术有限公司
Priority date: 2020-05-09
Filing date: 2021-05-10
Publication date: 2021-11-18
Also published as: CN113628317A

Abstract

A rendering method, comprising: a remote rendering platform acquiring a virtual scene, wherein the virtual scene comprises a light source and at least one three-dimensional model; the remote rendering platform performing, according to the light source of the virtual scene, forward ray tracing on grids of each three-dimensional model of the virtual scene, wherein the surface of the three-dimensional model of the virtual scene is segmented to obtain the grids of the three-dimensional model of the virtual scene; the remote rendering platform generating pre-ray-tracing results of the grids of each three-dimensional model of the virtual scene according to forward ray tracing results of the grids of each three-dimensional model of the virtual scene; the remote rendering platform storing the pre-ray-tracing results of the grids of each three-dimensional model of the virtual scene; the remote rendering platform receiving a first rendering request and determining first observable grids, on each three-dimensional model of the virtual scene, for the first rendering request; and the remote rendering platform determining rendering results of the first observable grids from the stored pre-ray-tracing results of the grids of each three-dimensional model of the virtual scene.

Description

Rendering method, equipment and system

Technical field

This application relates to the field of computer technology, and in particular to a rendering method, device, and system.

Background technique

Rendering refers to the process of using software to generate images from a three-dimensional model. The three-dimensional model is a description of a three-dimensional object in a strictly defined language or data structure, which includes geometry, viewpoint, texture, and lighting information. The image is a digital image or a bitmap image. The term rendering is similar to "artist's rendering of a scene". In addition, rendering is also used to describe "the process of calculating the effects in a video editing file to generate the final video output." The process of rendering an image according to the model requires a large amount of calculation and consumes a lot of computing resources.

Summary of the invention

In order to solve the above problems, the present application provides a rendering method, which can effectively improve rendering efficiency.

In the first aspect, a rendering method is provided, including:

Acquiring a virtual scene by a remote rendering platform, the virtual scene including a light source and at least one three-dimensional model;

The remote rendering platform performs forward ray tracing on the grids of the three-dimensional models of the virtual scene according to the light source of the virtual scene, wherein the surface of the three-dimensional model of the virtual scene is segmented to obtain the three-dimensional model of the virtual scene Grid

The remote rendering platform generates the pre-ray tracing results of the grids of the three-dimensional models of the virtual scene according to the forward ray tracing results of the grids of the three-dimensional models of the virtual scene;

The remote rendering platform stores the pre-ray tracing results of the grids of the three-dimensional models of the virtual scene;

The remote rendering platform receives a first rendering request, and determines the first observable grid of each three-dimensional model of the virtual scene in the first rendering request;

The remote rendering platform determines the rendering result of the first observable grid from the stored pre-ray tracing results of the grids of the three-dimensional models of the virtual scene.

In some possible designs, the remote rendering platform generates a first rendered image according to the rendering result of the first observable grid, or the remote rendering platform converts the first observable grid The rendering result of is sent to the first terminal device, so that the first terminal device generates a first rendered image according to the rendering result of the first observable grid.

In some possible designs, the remote rendering platform receives a second rendering request, and determines that the second rendering request is in the second observable grid of each three-dimensional model of the virtual scene; the remote rendering platform receives From the stored pre-ray tracing results of the grids of the three-dimensional models of the virtual scene, the rendering result of the second observable grid is determined.

In the above solution, the pre-ray tracing results of the grids of each 3D model of the virtual scene are pre-calculated and stored in the remote rendering platform. When different users need to render different perspectives of the same virtual scene, they only need to separately The corresponding results can be queried in the pre-ray tracing results, and there is no need to perform separate calculations, which greatly reduces the amount of calculations.

In some possible designs, the first rendering request is issued by the first terminal according to an operation of the first user, and the first rendering request carries the perspective of the first user in the virtual scene.

In some possible designs, before the remote rendering platform performs forward ray tracing on the grids of the three-dimensional models of the virtual scene according to the light source of the virtual scene, the method further includes:

The remote rendering platform obtains the forward ray tracing parameters set by the provider of the virtual scene or the user who issued the first rendering request, and the forward ray tracing parameters include at least one of the following: the number of samples per unit area; and The number of light bounces;

The remote rendering platform performs forward ray tracing on the grids of the three-dimensional models of the virtual scene according to the light source of the virtual scene, including:

The remote rendering platform performs forward ray tracing on the grid of each three-dimensional model of the virtual scene according to the light source of the virtual scene and the forward ray tracing parameter.

In the above solution, the user can set the forward ray tracing parameters according to actual needs. When the quality of the rendered image is high, the forward ray tracing parameters can be set with higher requirements, and vice versa, the forward ray tracing parameters can be set with lower requirements. Tracking parameters.

In some possible designs, before the remote rendering platform stores the pre-ray tracing results of the grids of the three-dimensional models of the virtual scene, the method further includes:

The remote rendering platform performs reverse ray tracing of multiple preset viewing angles on part or all of the grids of each three-dimensional model of the virtual scene according to the light source of the virtual scene;

The remote rendering platform generates the pre-ray tracing results of the grids of the three-dimensional models of the virtual scene according to the forward ray tracing results of the grids of the three-dimensional models of the virtual scene, including:

The remote rendering platform according to the forward ray tracing results of the grids of the three-dimensional models of the virtual scene and the backward ray tracing results of multiple preset viewing angles of part or all of the grids of the three-dimensional models of the virtual scene A pre-ray tracing result of the grid of each three-dimensional model of the virtual scene is generated.

It can be understood that since some of the first observable grids may not have reverse ray tracing results, if there is a reverse ray tracing result for some of the first observable grids, then the first observable part of the grid has reverse ray tracing results. The rendering result of the mesh includes the reverse ray tracing result and the forward ray tracing result of the mesh. If there is no reverse ray tracing result for part of the first observable mesh, then the first observable part of the The rendered result of the mesh includes the result of forward ray tracing, but not the result of reverse ray tracing.

In the above solution, by adding the reverse ray tracing result, the rendered image can have a stronger sense of reality and a better effect.

In some possible designs, the remote rendering platform obtains preset viewing angle parameters set by the provider of the virtual scene or the user. The preset viewing angle parameter may be the number of preset viewing angles, or may be multiple preset viewing angles. In addition, the parameters set by the provider of the virtual scene or the user may also include the number of light bounces and so on.

In the above solution, the user can set preset viewing angle parameters according to actual needs. When the preset viewing angle parameter is larger, the effect of rendering the image is better.

In some possible designs, the remote rendering platform performs reverse ray tracing of multiple preset viewing angles on part or all of the meshes of each three-dimensional model of the virtual scene according to the light source of the virtual scene, including:

According to the light source of the virtual scene, the remote rendering platform performs reverse ray tracing of a plurality of preset viewing angles on a smooth grid of materials of each three-dimensional model of the virtual scene.

In the above solution, only reverse ray tracing is performed on a mesh with a smooth material, which can ensure the quality of the rendered image under the premise of controlling the amount of calculation.

In the second aspect, a rendering node is provided, including: a rendering application server and a rendering engine,

The rendering application server is configured to obtain a virtual scene, the virtual scene including a light source and at least one three-dimensional model;

The rendering engine is configured to perform forward ray tracing on the grid of each three-dimensional model of the virtual scene according to the light source of the virtual scene, wherein the surface segmentation of the three-dimensional model of the virtual scene obtains the The grid of the three-dimensional model; generating the pre-ray tracing results of the grids of the three-dimensional models of the virtual scene according to the forward ray tracing results of the grids of the three-dimensional models of the virtual scene; storing the three-dimensional models of the virtual scene Pre-ray tracing results of the mesh of the model;

The rendering application server is configured to receive a first rendering request, and determine the first observable grid of each three-dimensional model of the virtual scene in the first rendering request;

The rendering engine is configured to determine the rendering result of the first observable grid from the stored pre-ray tracing results of the grids of the three-dimensional models of the virtual scene.

In some possible designs, the rendering application server is also used to obtain the forward ray tracing parameters set by the provider of the virtual scene or the user who issued the first rendering request, and the forward ray tracing parameters include At least one of the following: the number of samples per unit area and the number of light bounces;

The rendering engine is further configured to perform forward ray tracing on the grids of each three-dimensional model of the virtual scene according to the light source of the virtual scene and the forward ray tracing parameters.

In some possible designs, the rendering engine is further configured to perform reverse ray tracing of multiple preset viewing angles on part or all of the meshes of each three-dimensional model of the virtual scene according to the light source of the virtual scene; The forward ray tracing results of the grids of the three-dimensional models of the virtual scene and the reverse ray tracing results of multiple preset viewing angles of part or all of the three-dimensional models of the virtual scene generate the virtual scene Pre-ray tracing results of the mesh of each 3D model.

In some possible designs, the rendering engine is further configured to perform reverse ray tracing of multiple preset viewing angles on a smooth grid of materials of each three-dimensional model of the virtual scene according to the light source of the virtual scene.

In a third aspect, a rendering node is provided, including a memory and a processor, and the processor executes a program in the memory to execute the method provided in the first aspect and possible designs thereof. Specifically, the rendering node may include one or more computers, and each computer executes part or all of the steps in the method provided in the first aspect and possible designs.

In a fourth aspect, a computer-readable storage medium is provided, which includes instructions that, when the instructions run on a computing node, cause the computing node to execute the method provided in the first aspect and possible designs thereof.

In a fifth aspect, a rendering system is provided, including: a terminal device, a network device, and a remote rendering platform. The terminal device communicates with the remote rendering platform through the network device, wherein the remote rendering platform is used for Implement the methods provided by the first aspect and its possible designs.

In a sixth aspect, a computer program product is provided, including instructions, which when the instructions run on a rendering node, cause the rendering node to execute the method provided in the first aspect and possible designs thereof.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the background art, the following will describe the drawings that need to be used in the embodiments of the present application or the background art.

1A and 1B are schematic diagrams of the structure of the rendering system provided by the present application;

2 is a schematic diagram of a rendered image obtained by observing a virtual scene from different angles provided by the present application;

3A to 3C are schematic diagrams of ray tracing rendering provided by this application;

FIG. 4 is another schematic diagram of ray tracing rendering provided by this application;

FIG. 5 is a schematic diagram of the comparison between the calculation amount when each user performs the calculation separately and the calculation amount when the common part is extracted for unified calculation provided by this application;

6A and 6B are schematic diagrams of the grids of various three-dimensional models provided by this application;

7A to 7D are schematic diagrams of forward ray tracing rendering provided by the present application;

FIG. 8 is another schematic diagram of forward ray tracing rendering provided by this application;

Fig. 9 is a schematic diagram of the number of light rays passing per unit area of the light source provided by the present application;

FIG. 10 is a schematic diagram of various preset viewing angles of reverse ray tracing rendering provided by the present application;

11A and 11B are schematic diagrams of reverse ray tracing rendering provided by the present application;

FIG. 12 is a schematic diagram of obtaining a set of object surfaces observable from a user's perspective provided by the present application;

13A and 13B are comparison diagrams where the number of light samples per grid in the three-dimensional model provided by the present application is 1 and n, respectively;

FIG. 14 is a schematic flowchart of a method for generating pre-ray tracing results proposed by this application;

FIG. 15 is a schematic flowchart of a rendering method proposed by this application;

FIG. 16 is a schematic diagram of the structure of a rendering node proposed in this application;

FIG. 17 is a schematic structural diagram of another rendering node proposed in this application.

Detailed ways

Referring to FIG. 1A, FIG. 1A is a schematic structural diagram of a rendering system related to the present application. The rendering system of the present application is used for a 2D image obtained by rendering a 3D model of a virtual scene by a rendering method, that is, a rendered image. The rendering system of the present application may include: one or more terminal devices 10, a network device 20, and a remote rendering platform 30. The remote rendering platform 30 may be specifically deployed on a public cloud. The remote rendering platform 30 and the terminal device 10 are generally deployed in different data centers.

The terminal device 10 may be a device that needs to display rendered images in real time, for example, it may be a virtual reality (VR) device used for flight training, a computer used for virtual games, a smart phone used for a virtual mall, etc. , There is no specific limitation here. The terminal device can be a device with high configuration and high performance (for example, multi-core, high clock speed, large memory, etc.), or a device with low configuration and low performance (for example, single core, low clock speed, small memory, etc.) equipment. In a specific embodiment, the terminal device 10 may include hardware, an operating system, and a rendering application client.

The network device 20 is used to transmit data between the terminal device 10 and the remote rendering platform 30 through a communication network of any communication mechanism/communication standard. Among them, the communication network can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof.

The remote rendering platform 30 includes one or more remote rendering nodes, and each remote rendering node includes rendering hardware, a virtualization service, a rendering engine, and a rendering application server from bottom to top. Among them, the rendering hardware includes computing resources, storage resources, and network resources. Computing resources can use heterogeneous computing architecture, for example, central processing unit (CPU) + graphics processing unit (GPU) architecture, CPU + AI chip, CPU + GPU + AI chip architecture, etc. , There is no specific limitation here. Storage resources can include storage devices such as memory and video memory. Network resources can include network cards, port resources, address resources, and so on. The virtualization service is a service that virtualizes the resources of the rendering node into vCPUs and other self through virtualization technology, and flexibly isolates independent resources to run the user's application according to the user's needs. Commonly, virtualization services may include virtual machine (VM) services and container (container) services. VMs and containers can run rendering engines and rendering application servers. The rendering engine is used to implement the rendering algorithm. The rendering application server is used to call the rendering engine to complete the rendering of the rendered image.

The rendering application client on the terminal device 10 and the rendering application server of the remote rendering platform 30 are collectively referred to as a rendering application. Common rendering applications can include: game applications, VR applications, movie special effects, animations, and so on. The user inputs operation instructions through the rendering application client, the rendering application client sends the operation instructions to the rendering application server, the rendering application server calls the rendering engine to generate rendering results, and sends the rendering results to the rendering application client. Then, the rendering application client converts the rendering result into an image and presents it to the user. In a specific implementation manner, the rendering application server and the rendering application client may be provided by a rendering application provider, and the rendering engine may be provided by a cloud service provider. For example, the rendering application can be a game application. The game developer of the game application installs the game application server on the remote rendering platform provided by the cloud service provider, and the game developer of the game application provides the game application client through the Internet Download it to the user and install it on the user's terminal device. In addition, the cloud service provider also provides a rendering engine, which can provide computing power for game applications. In another specific implementation manner, the rendering application client, the rendering application server, and the rendering engine may all be provided by a cloud service provider.

The rendering system shown in FIG. 1B further includes a management device 40. The management device 40 may be a device provided by a third party other than the user's terminal device and the remote rendering platform 30 of the cloud service provider. For example, the management device 40 may be a device provided by a game developer. The game developer can manage the rendering application through the management device 40. It can be understood that the management device 40 may be set on the remote rendering platform, and may also be set outside the remote rendering platform, which is not specifically limited here.

Take the rendering system shown in Figure 1A or Figure 1B as an example. In a virtual scene where multiple users participate, in order to enable each user to have a sense of reality, user A uses terminal device 1 and user B uses terminal device 2. Join the same virtual scene. Therefore, as shown in Figure 2, assuming a virtual scene as shown in Figure 2(a), terminal device 1 needs to display the rendered image of the virtual scene generated from the perspective of user A, and terminal device 2 needs to display the image from user B A rendered image of the virtual scene generated by the perspective. The terminal device 1 and the terminal device 2 may independently use the resources of the remote rendering platform 30 to perform ray tracing rendering on the virtual scene, so as to obtain rendered images of different angles. specifically,

The terminal device 1 sends a first rendering request to the remote rendering platform 30 through the network device 20, and the remote rendering platform 30 calls the rendering engine according to the first rendering request from the user A’s perspective and uses ray tracing rendering to separately perform ray tracing on the virtual scene, thereby obtaining A rendered image of the virtual scene generated from the perspective of user A.

The terminal device 2 sends a second rendering request to the remote rendering platform 30 through the network device 20, and the remote rendering platform 30 calls the rendering engine according to the second rendering request from the user B’s perspective and uses ray tracing rendering to separately perform ray tracing on the virtual scene, thereby obtaining A rendered image of the virtual scene generated from the perspective of user B.

The ray tracing rendering method adopted by the terminal device 1 and the terminal device 2 will be described in detail below. Ray tracing rendering is a rendering method that generates a rendered image by tracing the path of light emitted from the viewpoint of an observer (such as a camera or human eye) toward each pixel of the rendered image into the virtual scene. Among them, the virtual scene includes a light source and a three-dimensional model. In the ray tracing rendering method, starting from the observer's point of view (when the point of view is determined, the angle of view is naturally determined), the light that can reach the light source is traced backwards. Since only the rays that can finally enter the observer's point of view are useful, the reverse tracing of rays can effectively reduce the amount of data processing. There are mainly three scenes of reflection, refraction, and transmission in ray tracing rendering, which will be described below in conjunction with specific embodiments.

As shown in FIG. 3A, in the reflection scene, it is assumed that the virtual scene has only one light source 111 and one opaque sphere 112. A light ray is emitted from the viewpoint E of the camera 113 (taking the camera 113 observers as an example), and is projected onto the pixel point O ₁ of the rendered image 114, and then continues to shoot to a point P _{1 of the} opaque sphere 112, and then is reflected To the light source L, at this time, _{the light intensity and color of the point P 1} determine the light intensity and color of the pixel O ₁ . Another ray is emitted from the viewpoint E of the camera 111, and is projected to another pixel O ₂ in the rendered image 114, and then continues to be emitted to a point P _{2 of the} opaque sphere 112, and then is reflected to the light source L, and the point There is an obstacle opaque sphere 112 between P ₂ and the light source L. At this time, the point P ₂ is located in the shadow of the opaque sphere 112, _{the light intensity of the pixel point O 2} is zero, and the color is black.

As shown in FIG. 3B, in the refraction scene, it is assumed that the virtual scene has only one light source 121 and one transparent sphere 122. A light ray is emitted from the viewpoint E of the camera 123, and is projected onto the pixel point O ₃ on the rendered image 124, and then continues to be emitted to a point P _{3 of the} transparent sphere 122, and then is refracted to the light source L. At this time, the point P _{The light intensity and color of 3 determine} the light intensity and color of pixel O ₃ .

As shown in FIG. 3C, in the transmission scene, it is assumed that the virtual scene has only one light source 131 and one transparent thin body 132. A light ray is emitted from the viewpoint E of the camera 133 and is projected to the point O _{4 of the} grid, and then continues to be emitted to a point P _{4 of the} transparent thin body 132, and then is transmitted to the light source L. At this time, the light of the _{point P 4} The intensity and color determine the light intensity and color of _{the pixel O 4.}

However, the reflection scene in Figure 3A, the refraction scene in Figure 3B, and the transmission scene in Figure 3C are the simplest scenes. Figure 3A assumes that there is only one opaque sphere in the virtual scene, and Figure 3B assumes that there is only one opaque sphere in the virtual scene. There is only one transparent sphere. In Figure 3C, it is assumed that there is only one transparent thin body in the virtual scene. In actual applications, the virtual scene is far more complicated than Figures 3A to 3C. For example, there may be multiple opaque objects in the virtual scene at the same time. As well as multiple transparent objects, the light will be reflected, refracted and transmitted multiple times, which will cause the tracing of the light to become very complicated and consume a lot of computing resources.

In the complex virtual scene shown in FIG. 4, it is assumed that the virtual scene includes a light source 140, two

transparent spheres

141 and 142, and an opaque object 143. A light ray is emitted from the viewpoint E of the camera 144, projected to a pixel O ₄ in the rendered image 145, and continues to be _{projected to a point P 1 of the} transparent sphere 141. A shadow test line S1 is made from P ₁ to the light source L, during which For objects that are not obstructed, the local light illumination model can be used to calculate _{the light intensity of the light source to P 1} in the direction of the line of sight E as the local light intensity of the point. At the same time, also the tracking point at which reflected light R1 Tl and refract light, light intensity P which also contributes to _1:00. In the direction of the reflected light R1, if it does not intersect with other objects, the intensity of the light in this direction is set to zero, and the tracing of the light direction is ended. Then, continue to track the direction of the refracted light T1 to calculate the light intensity contribution of the light. The refracted light T1 propagates inside the transparent object 141 and continues to be emitted and intersects the transparent object 142 at the point P _2. Since this point is inside the transparent object 142, it can be assumed that its local light intensity is zero. At the same time, reflected light R2 and refraction are generated. The ray T2, in the direction of the reflected ray R2, can continue to be tracked recursively to calculate its ray intensity, and it will not continue here. Continue to track the refracted light T2. T2 and the opaque object 143 intersect at the point P ₃ , and make _{the shadow test line S3 between P 3} and the light source L. There is no object obscuration, then the local light intensity at that place is calculated. Because the opaque object 143 is non-transparent , then, you may continue to track the direction of the reflected light rays R3 strength, combined with local light intensity to obtain the light intensity at P _3. The tracking of the reflected light R3 is similar to the previous process, and the algorithm can proceed recursively. Repeat the above process until the ray meets the tracing termination condition. In this way, we can get _{the light intensity of pixel O 4} , which is its corresponding color value.

In the above embodiment, the rendering system only includes terminal device 1 and terminal device 2 as an example. In practical applications, the number of terminal devices may be far more than two, and users of different terminal devices often have different perspectives. are different. Therefore, as the number of users increases, the number of rendered images from different perspectives that need to be generated in the same virtual scene also increases, and the amount of calculation will be very huge. Moreover, since these terminal devices perform ray tracing rendering of the same virtual scene to obtain rendered images of different angles, there may be many calculations that are repeated, resulting in unnecessary waste of computing resources.

The rendering system proposed in this application can extract the common calculation part from the ray tracing rendering of the same virtual scene from different angles, and each user only needs to calculate the private part related to the vision separately, thereby effectively saving the rendering required Computing resources to improve rendering efficiency. Refer to FIG. 5, which is a schematic diagram of the comparison between the calculation amount when each user performs the calculation separately and the calculation amount when the common part is extracted for unified calculation. Among them, the left side of FIG. 5 is the calculation amount when each user performs the calculation separately, and the right side of FIG. 5 is the calculation amount when the common part is extracted for unified calculation.

As shown on the left side of Figure 5, when each user performs calculations individually, the total calculation amount is equal to the sum of the calculation amounts of each user calculation separately. That is, the total calculation amount = the individual calculation amount 1 of the user 1 + the individual calculation amount 2 of the user 2 +...+ the individual calculation amount n of the user n.

As shown on the right side of Figure 5, when the public computing part is uniformly calculated, the total calculation amount is equal to the total calculation amount of the public calculation part plus the calculation amount of each user's private part. That is, the total calculation amount = the calculation amount of the common calculation part + the angle calculation amount 1 of the user 1 + the angle calculation amount 2 of the user 2 +... + the angle calculation amount n of the user n.

It can be clearly seen from the comparison in the figure that extracting common parts for unified calculation can save calculations compared to individual calculations for each user, and the more users there are, the more calculations can be saved.

The rendering engine of the rendering system can perform image rendering through the following rendering algorithms:

Assume that there are one or more light sources and one or more three-dimensional models in the virtual scene. The light generated by the light source shines on the three-dimensional model. Among them, the light source can be a point light source, a line light source, a surface light source, and so on. The shape of the three-dimensional model can be various, for example, it can be a sphere, a cone, a curved object, a flat object, an object with an irregular surface, and so on.

Calculations in the public part:

The remote rendering platform divides the surface of the 3D model in the virtual scene into multiple grids. The shapes of the meshes of the three-dimensional models with different shapes may be different. For example, the shapes of the mesh of a sphere and the mesh of a curved object may be completely different. The meshes will be described below in conjunction with specific embodiments.

As shown in Figure 6A, taking the 3D model as a sphere as an example, the grid can be expressed as the center point

And the center point

The points in the neighborhood constitute an approximate square with slightly bulging sides on the surface of the sphere. A three-dimensional orthogonal coordinate system is constructed with the center of the sphere as the origin, where the three-dimensional orthogonal coordinate system includes an x-axis, a y-axis, and a z-axis. In the coordinates of the center point P, r is the length of the line segment OP from the center of the sphere O to the center point P, and θ is the angle between the line segment OP and the positive z axis,

It is expressed as the angle between the projection of the line segment OP on the xoy plane and the x axis. In a particular embodiment, may be uniformly disposed on a sphere center point of the n _{_{P 1, P 2, ...,}} P n, if non-central point from the center point P _i and Q _J shortest, the center point of the non-Q the center point P _i and _j belong to the same grid.

As shown in FIG. 6B, taking the three-dimensional model as a curved object as an example, the grid can be expressed as a square on the curved surface represented by P(u,t). A two-dimensional orthogonal coordinate system is constructed with a set origin of the curved surface, where the coordinate system includes u-axis and t-axis. u represents the offset in one direction of the origin of the curved surface, t represents the offset in the other orthogonal direction, and P(u,t) represents the four vertices in the (u,t) coordinate system as shown in Figure 6B Consists of squares.

It can be understood that the shape of the grid described above is merely a specific example, and in actual applications, the grid may also have other shapes, which is not specifically limited here. In addition, the size of the grid can be set as needed. If the accuracy of the rendered image is higher, the size of the grid can be set to be smaller.

The material of the aforementioned mesh can be smooth or rough. Among them, smooth materials are those with specular reflection or transmission, such as mirrors, metal surfaces, water droplets, and so on. Rough materials are materials that have diffuse reflection, such as natural wood and cloth. When the meshes of the 3D model in the virtual scene are all rough materials, the remote rendering platform can only perform forward ray tracing, or the remote rendering platform can perform forward ray tracing and reverse ray tracing for all meshes; When the mesh of the 3D model in the virtual scene includes smooth materials and rough materials, the remote rendering platform can perform forward ray tracing and reverse ray tracing for smooth mesh materials, or the remote rendering platform can perform Forward ray tracing and reverse ray tracing for all meshes; when the meshes of the 3D model in the virtual scene are all smooth materials, the remote rendering platform can only perform forward ray tracing, or the remote rendering platform can perform Forward ray tracing and reverse ray tracing for all meshes. The concept of forward ray tracing and reverse ray tracing will be introduced in detail below.

Forward ray tracing refers to the forward tracing of the light transmission process in the virtual scene starting from the light source. The remote rendering platform performs forward ray tracing on the light generated by the light source in the virtual scene, so as to obtain the light intensity of each grid in the 3D model in the virtual scene. The forward ray tracing mainly includes four scenes: reflection, refraction, transmission, and direct shooting, which will be described below with reference to FIGS. 7A-7D and specific embodiments respectively.

As shown in FIG. 7A, in the reflection scene, it is assumed that the virtual scene has only one light source 211, an opaque sphere 212, and an opaque sphere 213. A light emitted from the light source 211, is projected to a point 212 on an opaque sphere of P _1, then the opaque sphere is reflected to a center point 213 of the grid point Q ₁ '. Therefore, the local light illumination model can be used to calculate the _{intensity of the light generated by the light source 211 at the point P 1} of the opaque sphere 212, and then continue to trace the light reflected by the opaque sphere 212 at the center point of the opaque sphere 213 as the point Q ₁ The intensity of the light generated on the grid.

As shown in FIG. 7B, in the refraction scene, it is assumed that the virtual scene has only one light source 221, a transparent sphere 222, and an opaque sphere 223. A light ray is emitted from the light source 221, projected onto a point P ₂ of the transparent sphere 222, and then is refracted onto a grid with a center point of the opaque sphere 223 as the point Q _2. Therefore, the light intensity of the light generated by the light source 221 at the point P ₂ of the transparent sphere 222 can be calculated through the local light illumination model, and then continue to trace the light after being refracted by the transparent sphere 222 at the center point of the opaque sphere 223 as the point Q ₂ The intensity of the light generated on the grid.

As shown in FIG. 7C, in the transmission scene, it is assumed that the virtual scene has only one light source 231, a transparent thin body 232, and an opaque sphere 233. A light ray is emitted from the light source 221, is projected to a point P _{3 of the} transparent thin body 232, and then is transmitted to a grid with a center point of the opaque sphere 233 as the point Q _3. Therefore, the local light illumination model can be used to calculate the light intensity of the light generated by the light source 231 at the point P ₃ of the transparent thin body 232, and then continue to trace the light at the center of the opaque sphere 233 after being transmitted by the transparent thin body 232. The intensity of the light generated on the _{Q 3 grid.}

As shown in FIG. 7D, in a direct scene, it is assumed that the virtual scene has only one light source 241 and an opaque sphere 243. A light ray is emitted from the light source 241 and is projected onto a grid with _{a center point of the opaque sphere 243 at the point Q 4.} Therefore, the light intensity of the light generated by the light source 241 on the grid with the center point of the opaque sphere 243 being the point Q _{4 can be calculated by the local light illumination model.}

However, the reflection scene in FIG. 7A, the refraction scene in FIG. 7B, the transmission scene in FIG. 7C, and the direct scene in FIG. 7D is complicated. For example, there may be multiple opaque objects and multiple transparent objects in the virtual scene at the same time. Therefore, the light will be reflected, refracted and transmitted multiple times. In addition, there may be more than one light source, but two or more.

Therefore, during forward ray tracing, the light intensity of each grid of the 3D model in the virtual scene is based on the light intensity generated by all the light reflected on the grid, and the light generated by all the light refracted on the grid. The intensity, the light intensity of all the light rays transmitted to the grid, and the light intensity of all the light rays directly hitting the grid are calculated. For example, it can be the sum of the light intensities.

Taking the example shown in FIG. 8 as an example, there are a first light source 251, a second light source 252, a transparent sphere 253, a transparent thin body 254, a first opaque sphere 255, and a second opaque sphere 256 in the virtual scene. Among them, the first light generated by the first light source 251 is projected to the point P _{1 of the} transparent sphere 253, and then is refracted onto the grid with the center point of the two opaque spheres 256 as the point Q; the second light generated by the first light source 251 Projected on the point P ₂ of the transparent thin body 254, and then is transmitted to the grid with the center point of the second opaque sphere 256 as the point Q; the third light generated by the second light source 252 directly hits the second opaque sphere 256 Q is the center point on the grid points; third light generated by the second light source 252 is projected onto the first opaque sphere point P _3, and then is reflected to the center point 256 of the second opaque sphere grid point Q superior. Therefore, the light intensity of the grid where the point Q is located is based on the light intensity produced by the first light being refracted to the point Q, the light intensity produced by the second light being transmitted to the point Q, and the light intensity produced by the third light being directed to the point Q And the light intensity generated by the fourth light being reflected to the point Q is calculated, for example, it can be the sum of the light intensity.

It is understandable that the above examples are all taking the light emitted by the light source as one light, and the number of light bounces does not exceed 5 as an example. However, in fact, the number of light rays and the number of light bounces can also be other numbers. There is no specific limitation here. Normally, since the number of rays emitted by the light source is unlimited and computing resources are limited, under normal circumstances, it is impossible to perform forward ray tracing on all rays. Therefore, it is necessary to sample the rays emitted by the light source.

When sampling the rays of the forward ray tracing, the parameters involved in the sampling mainly include the number of samples per unit space and the number of ray bounces (bounce) and so on. The following will take the number of samples per unit area (SPUA) and the number of light bounces as examples for detailed introduction.

SPUA defines the number of rays sampled per unit area. Taking the example shown in FIG. 9 as an example, the spherical surface S is constructed with the light source L as the center, and the spherical surface S is divided into multiple unit areas. Then, SPUA is equal to the number of light rays generated by the light source L that pass through the unit area A. Theoretically, the number of rays generated by the light source L passing through a unit area is infinite. However, in the actual tracking process, it is impossible to trace all rays, and only a limited amount of rays can be traced. The greater the number of SPUAs, the greater the number of traced rays and the better the image quality, but the greater the amount of calculation. On the contrary, the smaller the number of SPUAs, the smaller the number of traced rays, and the worse the image quality, but the smaller the amount of calculation.

The number of ray bounces is the sum of the maximum number of reflections and the maximum number of refractions for tracing the ray before the forward tracing of the ray is terminated. Because in a complex scene, light will be reflected and refracted multiple times. In theory, the number of times the light is reflected and refracted can be infinite, but in the actual tracking process, it is impossible to track the light infinitely. Therefore, some tracking termination conditions need to be given. In the application, there can be the following termination conditions: the light will attenuate after many reflections and refractions, and the light contributes little to the light intensity of the viewpoint; or, the number of ray bounces, that is, the tracking depth, is greater than a certain value. Here, the more light bounces, the more effective light that can be tracked, the better the refraction effect between multiple transparent objects, the more realistic, the better the image quality, but the greater the amount of calculation. On the contrary, the fewer the number of light bounces, the less effective light that can be traced, the worse the refraction effect between multiple transparent objects, the more distorted, and the worse the image quality, but the less the amount of calculation.

It can be understood that the above-mentioned sampling parameters are only taken as specific examples. In actual applications, other sampling parameters may also be used, which are not specifically limited here.

In order to reduce the amount of calculation, if the light generated by the light source in the virtual scene is not the light illuminating the direction of the 3D model, it is not necessary to perform forward ray tracing.

Reverse ray tracing rendering is the process of tracing the light rays entering the grid of the 3D model from a preset perspective and passing them to the light source in the virtual scene. Here, the preset angle of view is a certain angle from which the user observes the virtual scene. For example, suppose that when the user views the virtual scene vertically, the preset angle of view is (90, 90), when the user views the virtual scene from the left 45 degrees, the preset angle of view is (45, 0), and the user views 45 degrees from the right In the virtual scene, the preset angle of view is (135, 0) and so on. The above-mentioned light obtained by reverse tracing from the preset angle of view can only be viewed by the human eye or the camera at the preset angle of view. Therefore, in order to realize the observation of the virtual scene from different angles, reverse ray tracing from each preset angle of view is required . Take the example shown in Figure 10 to illustrate each preset viewing angle. Each grid has an open hemispherical space facing its normal direction. The light entering the hemispherical space can be expressed as the end point of the grid center P and the hemisphere Any point O (for example, O ₁ , O ₂ or O ₃ ) on the sphere of the space is the starting point, and reverse ray tracing is performed on the different preset viewing angles of each grid. The preset viewing angle mentioned here refers to the light OP (Θ,π) in the hemispherical coordinate system, where 0<θ<180, 0<π<360. The space is continuous, but we can quantify the preset viewing angle according to the computing power and accuracy requirements. For example, you can set a preset viewing angle every 1 degree, or you can set a preset viewing angle every 2 degrees. It can be understood that the greater the number of preset viewing angles, the smaller the quantization error and the higher the accuracy.

There are mainly two scenes of reflection and refraction in reverse ray tracing rendering, which will be described below in conjunction with specific embodiments.

As shown in FIG. 11A, in the reflection scene, it is assumed that the virtual scene has only one light source 311 and one opaque sphere 312. Light is emitted from a predetermined viewing angle, an opaque sphere projected onto a grid point 312 P _1, then, it is reflected to the light source 311. At this time, the light intensity of the light generated by the light source 311 on the grid of the opaque sphere 312 can be calculated by the local light illumination model.

As shown in FIG. 11B, in the refraction scene, it is assumed that the virtual scene has only one light source 321 and one transparent sphere 322. A light ray is emitted from a preset angle of view, projected to a point P ₂ of a grid of the transparent sphere 322, and then refracted to another point Q _{2 of the} transparent sphere 322, and then refracted to the light source L. At this time, it can pass local illumination model calculation source 321 generates light generated in the light intensity on the _two points Q, and then calculate the light intensity of the light generated from the point ₂ is the point Q ₂ at the center of the refracted P ₂ P grid.

However, the reflection scene in Figure 11A and the refraction scene in Figure 11B are the simplest scenes. Figure 11A assumes that there is only one opaque sphere in the virtual scene, and Figure 11B assumes that there is only one transparent sphere in the virtual scene. In actual applications, the virtual scene is far more complicated than that of Figures 11A to 11B. For example, there may be multiple opaque objects and multiple transparent objects in the virtual scene at the same time. Therefore, the light will be reflected, refracted and transmitted multiple times, resulting in The ray tracing has become very complicated and will not be described here.

In reverse ray tracing, the light intensity of each grid of the 3D model in the virtual scene is based on the light intensity of all the light reflected on the grid, the light intensity of all the light refracted to the grid, The light intensity of all the light rays transmitted to the grid and the light intensity of all the light rays directly hitting the grid are calculated, for example, it can be the sum of the light intensities.

It should be understood that the above examples are all based on reverse ray tracing from a certain preset angle of view. However, in fact, the light gathered by different angles on the same grid is also different, especially for smooth In terms of surface, therefore, reverse ray tracing needs to be performed from each preset viewing angle.

In the case where only forward ray tracing is required, after forward ray tracing is performed on the remote rendering platform, the pre-ray tracing results of the meshes of each 3D model can be obtained. Assuming that there are n grids T ₁ , T ₂ ,..., T _{n in the} virtual scene, after forward ray tracing, n grids T ₁ , T ₂ ,..., T _{n can be obtained after forward ray tracing} The respective forward ray tracing results F ₁ , F ₂ ,..., F _n are used as pre-ray tracing results of _n _{grids T 1} , T ₂ ,..., T n. The remote rendering platform associates the pre-ray tracing results F ₁ , F ₂ ,..., F _{n of n} _{grids T 1} , T ₂ ,..., T _n and n grids T ₁ , T ₂ ,..., T _n Stored in the light intensity table 1. In a specific embodiment, the light intensity table 1 may be the table shown in Table 1:

Table 1 Light intensity table 1

In the case that forward ray tracing and reverse ray tracing are required, after forward ray tracing and reverse ray tracing are performed, the result of forward ray tracing and reverse ray tracing can be obtained The inverse ray tracing results of each 3D model are processed to obtain the pre-ray tracing results of the meshes of each three-dimensional model. specifically,

Assuming that there are n grids T ₁ , T ₂ ,..., T _{n in the} virtual scene, perform forward ray tracing and perform reverse ray ray _{on the n grids T 1} , T ₂ ,..., T _{n from k angles.} track.

After forward ray tracing the light emitted by the light source, the forward ray tracing results F ₁ , F ₂ ,..., F of the _{n grids T 1} , T ₂ ,..., T _{n after the forward ray tracing can be obtained} _n .

After performing reverse ray tracing on the n grids T ₁ , T ₂ ,..., T _n from the first angle, it can be obtained that the n grids T ₁ , T ₂ ,..., T _n are respectively performed from the first angle Reverse ray tracing result obtained by reverse ray tracing

After performing reverse ray tracing on the n grids T ₁ , T ₂ ,..., T _n from the second angle, we can get the n grids T ₁ , T ₂ ,..., T _n from the second angle. Reverse ray tracing result obtained by reverse ray tracing

After performing reverse ray tracing on the n grids T ₁ , T ₂ ,..., T _n from the k-th angle, we can obtain the n grids T ₁ , T ₂ ,..., T _n from the k-th angle. Reverse ray tracing result obtained by reverse ray tracing

_{The forward ray tracing results F 1} , F ₂ ,..., F _n obtained by the forward ray tracing of the _n grids T ₁ , T ₂ ,..., T n respectively and the n grids T from the first angle ₁ ,T ₂ ,...,T _n reverse ray tracing result obtained by reverse ray tracing

Perform linear superposition to obtain the pre-ray tracing results _{of n grids T 1} , T ₂ ,..., T _{n at the first angle}

The forward ray tracing results F ₁ , F ₂ ,..., F _n _{obtained by each of the n} grids T ₁ , T ₂ ,..., T n through forward ray tracing are respectively and from the second angle to the n grids T ₁ ,T ₂ ,...,T _n reverse ray tracing result obtained by reverse ray tracing

Perform linear superposition to obtain the pre-ray tracing results _{of n grids T 1} , T ₂ ,..., T _{n at the second angle}

The forward ray tracing results F ₁ , F ₂ ,..., F _n _{obtained by each of the n} grids T ₁ , T ₂ ,..., T n through the forward ray tracing respectively and the n grid T from the k-th angle ₁ ,T ₂ ,...,T _n reverse ray tracing result obtained by reverse ray tracing

Perform linear superposition to obtain the pre-ray tracing results _{of n grids T 1} , T ₂ ,..., T _{n at the k-th angle}

Here, the pre-ray tracing results of n grids T ₁ , T ₂ ,..., T _{n at the first angle}

Pre-ray tracing results of n grids T ₁ , T ₂ ,..., T _{n at the second angle}

Pre-ray tracing results of n grids T ₁ , T ₂ ,..., T _{n at the kth angle}

It should be understood that in some embodiments, normalization processing may also be performed. In order to reduce the space required for storing the pre-ray tracing results of the grids of each three-dimensional model, the pre-ray tracing results of the grids of each three-dimensional model can be stored in a sparse matrix manner.

The remote rendering platform stores _{the pre-ray tracing results of the n grids T 1} , T ₂ ,..., T _n and the n grids T ₁ , T ₂ ,..., T _n in the light intensity table 2 in association with each other. In a specific embodiment, the light intensity table 2 may be the table shown in Table 2:

Table 2 Light intensity table 2

For the sake of brevity, the above content is described using reverse ray tracing for all n grids in the virtual scene. In practical applications, only part of the n grids (t network ) Perform reverse ray tracing. At this time, you only need to linearize the forward ray tracing result of each grid of t grids and the reverse ray tracing result of k angles of each grid of t grids. The superposition is used as the pre-ray tracing result, and the pre-ray tracing results of the remaining nt grids are recorded as the positive ray tracing result. Here, the mesh that needs to be reversed ray tracing can be the surface of an object that has reflection and refraction phenomena, such as a mirror surface, a transparent object, and the like. The remote rendering platform stores _{the pre-ray tracing results of the n grids T 1} , T ₂ ,..., T _n and the n grids T ₁ , T ₂ ,..., T _n in the light intensity table 3 in association with each other. In a specific embodiment, the light intensity table 3 may be the table shown in Table 3:

Table 3 Light Intensity Table 3

It can be seen that in Table 3, it is assumed that the grid T ₁ and the grid T _n are grids of coarse materials. Therefore, there is no need to perform reverse ray tracing. Naturally, there are only forward ray tracing results, and there is no reverse ray. Tracking Results.

In the above examples, the direct addition of the pre-ray tracing results is taken as an example for description. In practical applications, the pre-ray tracing results may also be weighted addition, etc., which is not specifically limited here.

Calculations in the private part:

When different users observe the virtual scene from different preset angles of view, the remote rendering platform or terminal device adopts the projection-type intersection method, and proposes corresponding observables from the pre-calculated pre-ray tracing results of the grids of each 3D model The rendering result of the obtained grid, and finally generate the rendered image required by the user. How to obtain the rendering result of the observable grid will be described in detail below in conjunction with FIG. 12 and related specific embodiments.

As shown in FIG. 12, it is assumed that the observer 511 observes the virtual scene from the viewpoint E, and the rendered image 512 generated by the observation has m pixels.

First, a light ray is emitted from the viewpoint E and projected onto the first pixel of the rendered image 512. Then, it is assumed that the light continues to be emitted to one of the grids T ₁ of the three-dimensional model in the virtual scene, and the ray enters The first angle of incidence of the grid is the first angle, then the grid can be found from the pre-ray tracing results of the grids of each three-dimensional model as shown in Table 2 according to the first angle of incidence being the first angle Pre-ray tracing results at the same angle

And this pre-ray tracing result

As the rendering result of the first pixel

Then, a light ray is emitted from the viewpoint E and projected onto the second pixel of the rendered image 512. Then, suppose the light continues to be emitted to one of the grids T ₁₀ of the three-dimensional model in the virtual scene, and the ray enters The second angle of incidence of the grid is the fifth angle, then the grid can be found to be the same from the pre-ray tracing results of the grids of each three-dimensional model shown in Table 2 according to the second angle of incidence being the second angle. Pre-ray tracing results for angles

And use the pre-ray tracing result as the rendering result of the second pixel.

…

Finally, a light ray is emitted from the viewpoint E and projected onto the m-th pixel of the rendered image 512. Then, it is assumed that the light continues to be emitted to one of the grids T _n-9 of the three-dimensional model in the virtual scene, and the ray The m-th angle of incidence into the grid is the k-th angle, then the mesh can be found from the pre-ray tracing results of the grids of each three-dimensional model as shown in Table 2 according to the m-th angle of incidence as the k-th angle. Pre-ray tracing results of the same angle of the grid

And use the pre-ray tracing result as the rendering result of the m-th pixel.

So far, the rendering results of the m pixels have been determined, and the rendered image 512 can be determined.

In the above examples, it is assumed that the first angle of incidence is exactly equal to the first angle, the second angle of incidence is exactly equal to the second angle, and the m-th angle of incidence is exactly equal to the k-th angle. However, in practical applications, the preset viewing angle Usually quantified, there may be cases where the angle of incidence is not exactly equal to the preset angle of view. For example, the first angle of incidence can be between the first angle and the second angle. In this case, it can be rounded up or down. The processing is performed in a whole and other manner, and there is no specific limitation here.

For the sake of simplicity, the embodiment corresponding to FIG. 12 is described by taking the sample per pixel (SPP) equal to 1 as an example, that is, each pixel only passes through one light, where SPP can be defined as each The number of rays obtained by pixel sampling. However, in practical applications, in order to improve the quality of the rendered image, the value of SPP is usually larger.

Assuming that SPP is equal to 2, the following will take a pixel of the rendered image as an example to illustrate the process of obtaining the light intensity of the rendered image.

A ray is emitted from the viewpoint E and projected onto the i-th pixel of the rendered image 512. Then, suppose that the ray continues to be emitted to one of the grids T ₃ of the three-dimensional model in the virtual scene, and the ray enters the grid The first angle of incidence of the grid is the first angle, then the same angle of the grid can be found from the pre-ray tracing results of the grid of each three-dimensional model as shown in Table 2 according to the first angle of incidence as the first angle Pre-ray tracing results

Another ray is emitted from the viewpoint E and projected onto the i-th pixel of the rendered image 512. Then, suppose that the ray continues to be emitted onto one of the grids T ₄ of the three-dimensional model in the virtual scene, and the ray enters the The first incident angle of the grid is the third angle, then the grid can be found from the pre-ray tracing results of the three-dimensional model grids shown in Table 2 based on the first incident angle being the third angle. Pre-ray tracing results for angles

Then, the pre-ray tracing results can be

And pre-ray tracing results

The average value of is used as the rendering result of the i-th pixel.

It can be understood that when the value of SPP is higher, it can be deduced by analogy. For the sake of brevity, it will not be repeated here.

The reason why the number of SPPs can affect the image quality of the rendered image is: Take Figure 13A as an example, if Spp is 1 (that is, only one light passes through each pixel), then even if the light shifts slightly, the pixel rendering The results may also vary greatly. Take the example shown in Figure 13A, if the light passes through pixel A, then the light will be projected on the opaque object 1 with lower light intensity. At this time, the rendering result of pixel A is projected on the opaque object 1. It is determined by the grid where the point is located, that is, the light intensity of the pixel point A is low. If the light passes through the pixel point B, the light will be projected onto the opaque object 2 with higher light intensity. At this time, the rendering result of the pixel point B is determined by the grid where the projection point on the opaque object 2 is located. That is, the light intensity of the pixel point B is relatively high. Therefore, although the pixel point A and the pixel point B are adjacent pixels, the rendering results of the pixel point A and the pixel point B will be very different, resulting in a sawtooth effect. In order to solve the above problem, take the example shown in Figure 13B, if the SPP is n (that is, n rays are emitted from the viewpoint to the same pixel on the rendered image, and n is an integer greater than 1), then these n rays of light pass through Each pixel is projected on the grid where the n projection points of the opaque object 1 or the opaque object 2 are located, so that the n light intensity of the pixel can be determined according to the light intensity of the grid where the n projection points are located, and finally, The n light intensity is averaged to obtain the rendering result of the pixel. If the pixel rendering result matches the picture reference frame (mathematical expectation value), the sampling noise is lower. Therefore, the greater the number of SPPs, the better the anti-aliasing effect of the rendered image, the lower the noise index, and the better the quality of the rendered image.

The whole process can be divided into at least two parts: The first part: the user uploads the virtual scene to the remote rendering platform in advance, and the remote rendering platform executes the calculation of the above-mentioned common part to obtain the light intensity of multiple grids and store them for future use. After receiving the rendering request sent by the terminal device, the second part of the remote rendering platform performs the calculation of the private part to obtain the rendered image. The two parts of the process will be described in detail below in conjunction with specific examples. The example shown in FIG. 14 mainly illustrates the process of the first part, and the example shown in FIG. 15 mainly illustrates the process of the second part.

Refer to FIG. 14, which is a schematic flowchart of a method for generating a pre-ray tracing result proposed by the present application. As shown in Figure 14, this method is implemented on the basis of the rendering system shown in Figure 1A or Figure 1B, and includes the following steps:

S101: The remote rendering platform acquires a virtual scene.

In a specific embodiment, the virtual scene may be sent by the terminal device to the remote rendering platform, or sent by the management device to the remote rendering platform.

In a specific implementation, the virtual scene may have a unique identifier, that is, the identifier of the virtual scene. Here, different virtual scenes have different identifiers of the virtual scenes. For example, the identifier of the virtual scene of virtual scene 1 is S ₁ , and the identifier of the virtual scene of virtual scene 2 is S ₂ ,,...

In a specific embodiment, the definition of the virtual scene, the light source in the virtual scene, the three-dimensional model in the virtual scene, the grid in each three-dimensional model, etc., please refer to the above, and the details will not be repeated here.

S102: The remote rendering platform performs forward ray tracing on the grid of each three-dimensional model of the virtual scene according to the light source of the virtual scene, to obtain a positive ray tracing result of the grid of each three-dimensional model.

In a specific implementation, before step S102 is performed, the method further includes: the provider of the virtual scene and the user who issued the rendering request setting forward ray tracing parameters. At this time, step S102 may include: the remote rendering platform obtains the forward ray tracing parameters set by the provider of the virtual scene or the user who issued the rendering request, and analyzes the three-dimensional models of the virtual scene according to the light source and forward ray tracing parameters of the virtual scene. The grid performs forward ray tracing, and the result of forward ray tracing is obtained. Wherein, the forward ray tracing parameter includes at least one of the following: the number of samples per unit area, the number of ray bounces, and so on.

In a specific implementation, the introduction of forward ray tracing can be added to the relevant content above, and will not be repeated here.

S103: According to the light source of the virtual scene, the remote rendering platform performs reverse ray tracing on part or all of the meshes of the three-dimensional models of the virtual scene with multiple preset angles of view to obtain the reflections of the meshes of the three-dimensional models. Trace the result to the ray.

In a specific implementation, before step S103 is performed, the method further includes: the provider of the virtual scene and the user who issued the rendering request setting the reverse ray tracing parameters. At this time, step S103 may include: the remote rendering platform obtains the reverse ray tracing parameters set by the provider of the virtual scene or the user who issued the rendering request, and performs a calculation of each three-dimensional model of the virtual scene according to the light source and reverse ray tracing parameters of the virtual scene. Part or all of the meshes are reversed ray tracing, and the reverse ray tracing results of the meshes of each three-dimensional model are obtained. Wherein, the reverse ray tracing parameter includes at least one of the following: a preset viewing angle parameter, the number of light bounces, and so on. The preset viewing angle parameter may be the number of preset viewing angles, or may be multiple preset viewing angles.

S104: The remote rendering platform determines the pre-ray tracing result of the mesh of each three-dimensional model according to the forward ray tracing result and the reverse ray tracing result.

In a specific embodiment, the remote rendering platform determines the first observable grid according to the reverse ray tracing result of the first observable grid and the forward ray tracing result of the first observable grid. For a process of observable grid rendering results, please refer to the process of determining the pre-ray tracing results of the grids of each three-dimensional model above, which will not be repeated here.

It can be understood that the above example is described by taking the presence of a grid that needs reverse ray tracing in the virtual scene as an example. If there is no grid that needs reverse ray tracing in the virtual scene, there is no need to perform step S103. And, in step S104, the pre-ray tracing result of the grid of each three-dimensional model can be directly determined according to the forward ray tracing result, and the description is not repeated here.

Refer to FIG. 15, which is a schematic flowchart of a rendering method proposed by the present application. As shown in FIG. 15, the rendering method is implemented on the basis of the rendering system shown in FIG. 1A or FIG. 1B, and includes the following steps:

S201: The first terminal device sends a first rendering request to the remote rendering platform through a network device. Correspondingly, the remote rendering platform receives the first rendering request sent by the first terminal device through the network device.

In a specific embodiment, the first rendering request includes the identifier of the virtual scene and the perspective of the first user, wherein the identifier of the virtual scene is a unique identifier of the virtual scene, and the first user’s The angle of view is the angle at which the first user observes the virtual scene.

S202: The second terminal device sends a second rendering request to the remote rendering platform through the network device. Correspondingly, the remote rendering platform receives the second rendering request sent by the second terminal device through the network device.

In a specific embodiment, the second rendering request includes an identifier of the virtual scene and a perspective of a second user, wherein the perspective of the second user is an angle at which the second user observes the virtual scene.

S203: The remote rendering platform receives a first rendering request, determines that the first rendering request is in the first observable grid of each three-dimensional model of the virtual scene, from the stored three-dimensional models of the virtual scene In the pre-ray tracing result of the grid, the rendering result of the first observable grid is determined, so as to generate the first rendered image.

In a specific embodiment, the remote rendering platform determines that the first rendering request is in the first observable grid of each three-dimensional model of the virtual scene, from the stored network of each three-dimensional model of the virtual scene In the pre-ray tracing result of the grid, the method of determining the rendering result of the first observable grid can refer to the calculation of the private part above, which will not be repeated here.

S204: The remote rendering platform receives a second rendering request, determines that the second rendering request is in the second observable grid of each three-dimensional model of the virtual scene, from the stored three-dimensional models of the virtual scene In the pre-ray tracing result of the grid, the rendering result of the second observable grid is determined, so as to generate a second rendered image.

In a specific embodiment, the remote rendering platform determines that the second rendering request is in the second observable grid of each three-dimensional model of the virtual scene, from the stored network of each three-dimensional model of the virtual scene In the pre-ray tracing result of the grid, the method of determining the rendering result of the second observable grid can refer to the calculation of the private part above, which will not be repeated here.

S205: The remote rendering platform sends the first rendered image to the first terminal device through the network device. Correspondingly, the first terminal device receives the first rendered image sent by the remote rendering platform through the network device.

S206: The remote rendering platform sends the second rendered image to the second terminal device through the network device. Correspondingly, the second terminal device receives the second rendered image sent by the remote rendering platform through the network device.

It can be understood that the above step sequence is only taken as a specific example. In other examples, the execution sequence can also be step S201 -> step S203 -> step S205 -> step S202 -> step S204 -> step S206, etc. , There is no specific limitation here.

It can be understood that in the above example, the first rendered image and the second rendered image are generated by the remote rendering platform as an example for description. In other embodiments, the remote rendering platform can compare the pre-ray tracing results of the meshes of each 3D model. Respectively sent to the first terminal device and the second terminal device, and the first terminal device generates the first rendered image according to the pre-ray tracing results of the grid of each three-dimensional model, and the second terminal device generates the first rendered image according to the grid of each three-dimensional model. The pre-ray tracing result of the grid generates the second rendered image, which is not specifically limited here.

Refer to FIG. 16, which is a schematic structural diagram of a rendering node proposed in this application. As shown in FIG. 16, the rendering node includes: a rendering application server 610 and a rendering engine 620.

The rendering application server 610 is configured to obtain a virtual scene, the virtual scene including a light source and at least one three-dimensional model;

The rendering engine 620 is configured to perform forward ray tracing on the grid of each three-dimensional model of the virtual scene according to the light source of the virtual scene, wherein the surface of the three-dimensional model of the virtual scene is segmented to obtain the virtual scene The grids of the three-dimensional models of the virtual scene; generate the pre-ray tracing results of the grids of the three-dimensional models of the virtual scene according to the forward ray tracing results of the grids of the three-dimensional models of the virtual scene; store each of the virtual scenes Pre-ray tracing results of the mesh of the 3D model;

The rendering application server 610 is configured to receive a first rendering request, and determine the first observable grid of each three-dimensional model of the virtual scene in the first rendering request;

The rendering engine 620 is configured to determine the rendering result of the first observable grid from the stored pre-ray tracing results of the grids of the three-dimensional models of the virtual scene.

For simplicity, there is no definition of the virtual scene in this embodiment. The light source in the virtual scene, the three-dimensional model in the virtual scene, and the grids in each three-dimensional model, forward ray tracing, and the prediction of the grid of each three-dimensional model are not included in this embodiment. The ray tracing result, the first observable grid, and the rendering result of the first observable grid, etc. are introduced. For details, please refer to the relevant content above, which is not specifically limited here. In addition, the rendering application server 610 and the rendering engine 620 in this embodiment can be set in the rendering nodes of FIG. 1A and FIG. 1B. For details, please refer to FIG. 1A and FIG. The rendering node in this embodiment can also perform the steps performed by the rendering node in FIG. 14 and the steps performed by the rendering node in FIG. 15.

Figure 17 is a schematic structural diagram of a rendering node. As shown in FIG. 17, the rendering node includes: a processing system 910, a first memory 920, a smart network card 930, and a bus 940.

The processor system 910 may adopt a heterogeneous structure, that is, include one or more general-purpose processors, and one or more special processors, such as GPUs or AI chips, etc., where the general-purpose processors may be capable of processing Any type of electronic instruction equipment, including central processing unit (CPU), microprocessor, microcontroller, main processor, controller, application specific integrated circuit (ASIC) and so on. The general-purpose processor executes various types of digital storage instructions, such as software or firmware programs stored in the first memory 920. In a specific embodiment, the general-purpose processor may be an x86 processor or the like. The general-purpose processor sends commands to the first memory 920 through the physical interface to complete storage-related tasks. For example, the commands that the general-purpose processor can provide include read commands, write commands, copy commands, and erase commands. The command may specify an operation related to a specific page and block of the first memory 920. Special processors are used to complete complex operations of image rendering and so on.

The first memory 920 may include random access memory (RAM), flash memory (Flash Memory), etc., and may also be RAM, read-only memory (Read-Only Memory, ROM), or hard disk (Hard Disk Drive). , HDD) or Solid-State Drive (SSD). The first memory 920 stores program codes for implementing the rendering engine and rendering application server.

The smart network card 930 is also called a network interface controller, a network interface card, or a local area network (LAN) adapter. Each smart network card 930 has a unique MAC address, which is burned into the read-only memory chip by the manufacturer of the smart network card 930 during production. The smart network card 930 includes a processor 931, a second memory 932, and a transceiver 933. The processor 931 is similar to a general-purpose processor, but the performance requirement of the processor 931 may be lower than that of a general-purpose processor. In a specific embodiment, the processor 931 may be an ARM processor or the like. The second memory 932 may also be a flash memory, HDD or SDD, and the storage capacity of the second memory 932 may be smaller than the storage capacity of the first memory 920. The transceiver 933 may be used to receive and send messages, and upload the received messages to the processor 931 for processing. The smart network card 930 may also include a plurality of ports, and the ports may be any one or more of the three interface types of a thick cable interface, a thin cable interface, and a twisted pair interface.

For simplicity, there is no definition of the virtual scene in this embodiment. The light source in the virtual scene, the three-dimensional model in the virtual scene, and the grids in each three-dimensional model, forward ray tracing, and the prediction of the grid of each three-dimensional model are not included in this embodiment. The ray tracing result, the first observable grid, and the rendering result of the first observable grid, etc. are introduced. For details, please refer to the relevant content above, which is not specifically limited here. In addition, the program codes of the rendering application server 610 and the rendering engine 620 in FIG. 16 may be set in the first memory 920 in FIG. 17. The rendering node in this embodiment can also perform the steps performed by the rendering node in FIG. 14 and the steps performed by the rendering node in FIG. 15.

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website, computer, server or data center via wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media. The usable medium may be a magnetic medium, (for example, a floppy disk, a storage disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

Claims

A rendering method is characterized in that it includes:

Acquiring a virtual scene by a remote rendering platform, the virtual scene including a light source and at least one three-dimensional model;

The remote rendering platform performs forward ray tracing on the grids of the three-dimensional models of the virtual scene according to the light source of the virtual scene, wherein the surface of the three-dimensional model of the virtual scene is segmented to obtain the three-dimensional model of the virtual scene Grid

The remote rendering platform generates the pre-ray tracing results of the grids of the three-dimensional models of the virtual scene according to the forward ray tracing results of the grids of the three-dimensional models of the virtual scene;

The remote rendering platform stores the pre-ray tracing results of the grids of the three-dimensional models of the virtual scene;

The remote rendering platform receives a first rendering request, and determines the first observable grid of each three-dimensional model of the virtual scene in the first rendering request;

The remote rendering platform determines the rendering result of the first observable grid from the stored pre-ray tracing results of the grids of the three-dimensional models of the virtual scene.
The method according to claim 1, wherein the first rendering request is issued by the first terminal according to the operation of the first user, and the first rendering request carries the information of the first user in the virtual scene. Perspective.
The method according to claim 1 or 2, characterized in that, before the remote rendering platform performs forward ray tracing on the grids of the three-dimensional models of the virtual scene according to the light source of the virtual scene, the method further include:

The remote rendering platform obtains the forward ray tracing parameters set by the provider of the virtual scene or the user who issued the first rendering request, and the forward ray tracing parameters include at least one of the following: the number of samples per unit area; and The number of light bounces;

The remote rendering platform performs forward ray tracing on the grids of the three-dimensional models of the virtual scene according to the light source of the virtual scene, including:

The remote rendering platform performs forward ray tracing on the grid of each three-dimensional model of the virtual scene according to the light source of the virtual scene and the forward ray tracing parameter.
The method according to any one of claims 1 to 3, wherein before the remote rendering platform stores the pre-ray tracing results of the meshes of the three-dimensional models of the virtual scene, the method further comprises:

The remote rendering platform performs reverse ray tracing of multiple preset viewing angles on part or all of the grids of each three-dimensional model of the virtual scene according to the light source of the virtual scene;

The remote rendering platform generates the pre-ray tracing results of the grids of the three-dimensional models of the virtual scene according to the forward ray tracing results of the grids of the three-dimensional models of the virtual scene, including:

The remote rendering platform according to the forward ray tracing results of the grids of the three-dimensional models of the virtual scene and the backward ray tracing results of multiple preset viewing angles of part or all of the grids of the three-dimensional models of the virtual scene A pre-ray tracing result of the grid of each three-dimensional model of the virtual scene is generated.
The method according to claim 4, wherein the remote rendering platform obtains preset viewing angle parameters set by the provider of the virtual scene or the user.
The method according to claim 4 or 5, wherein the remote rendering platform performs multiple preset perspective reflections on part or all of the meshes of each three-dimensional model of the virtual scene according to the light source of the virtual scene. To ray tracing, including:

According to the light source of the virtual scene, the remote rendering platform performs reverse ray tracing of a plurality of preset viewing angles on a smooth grid of materials of each three-dimensional model of the virtual scene.
A rendering node, which is characterized by comprising: a rendering application server and a rendering engine,

The rendering application server is configured to obtain a virtual scene, the virtual scene including a light source and at least one three-dimensional model;

The rendering engine is configured to perform forward ray tracing on the grid of each three-dimensional model of the virtual scene according to the light source of the virtual scene, wherein the surface segmentation of the three-dimensional model of the virtual scene obtains the The grid of the three-dimensional model; generating the pre-ray tracing results of the grids of the three-dimensional models of the virtual scene according to the forward ray tracing results of the grids of the three-dimensional models of the virtual scene; storing the three-dimensional models of the virtual scene Pre-ray tracing results of the mesh of the model;

The rendering application server is configured to receive a first rendering request, and determine the first observable grid of each three-dimensional model of the virtual scene in the first rendering request;

The rendering engine is configured to determine the rendering result of the first observable grid from the stored pre-ray tracing results of the grids of the three-dimensional models of the virtual scene.
The rendering node according to claim 7, wherein the first rendering request is issued by a first terminal according to an operation of a first user, and the first rendering request carries the first user in the virtual scene Perspective.
The rendering node according to claim 7 or 8, wherein:

The rendering application server is further configured to obtain forward ray tracing parameters set by the provider of the virtual scene or the user who issued the first rendering request, and the forward ray tracing parameters include at least one of the following: per unit The number of samples of the area and the number of light bounces;

The rendering engine is configured to perform forward ray tracing on the grid of each three-dimensional model of the virtual scene according to the light source of the virtual scene and the forward ray tracing parameters.
The rendering node according to any one of claims 7 to 9, wherein:

The rendering engine is further configured to perform reverse ray tracing of a plurality of preset viewing angles on part or all of the grids of each three-dimensional model of the virtual scene according to the light source of the virtual scene; The forward ray tracing result of the mesh of the model and the reverse ray tracing result of a plurality of preset viewing angles of part or all of the meshes of each three-dimensional model of the virtual scene are generated. Pre-ray tracing results.
The rendering node according to claim 10, wherein:

The rendering application server is also used to obtain preset viewing angle parameters set by the provider of the virtual scene or the user.
The rendering node according to claim 10 or 11, wherein:

The rendering engine is configured to perform reverse ray tracing of a plurality of preset viewing angles on a smooth grid of materials of each three-dimensional model of the virtual scene according to the light source of the virtual scene.
A rendering node, characterized by comprising a memory and a processor, and the processor executes the program in the memory to execute the method according to any one of claims 1 to 6.
A computer-readable storage medium, characterized by comprising instructions, which when run on a computing node, cause the computing node to execute the method according to any one of claims 1 to 6.
A rendering system, characterized by comprising: a terminal device, a network device, and a remote rendering platform, the terminal device communicates with the remote rendering platform through the network device, wherein the remote rendering platform is used to execute rights The method of any one of claim 1 to claim 6.
A computer program product, characterized by comprising instructions, which when the instructions are run on a rendering node, cause the rendering node to execute the method according to any one of claims 1 to 6.