CN116740245A - Training method and rendering method of image rendering model - Google Patents

Abstract

The embodiment of the application provides a training method and a rendering method of an image rendering model, wherein in the training method of the image rendering model, the rendering model to be trained is trained according to differences among a rendering image, a rendering segmentation image, an initial image and an initial segmentation image; wherein the training the rendering model to be trained includes: and adjusting the rendering parameters of the rendering model to be trained. According to the method, the rendering parameters of the rendering model to be trained are adjusted, so that differences between the rendering image and the rendering segmentation image obtained by the adjusted rendering model and the initial image and the initial segmentation image are reduced, and the object reduction degree in the rendering image is improved.

Description

An image rendering model training method and rendering method

Technical field

This application relates to the field of computer technology, specifically to a training method, device, electronic equipment and computer storage medium for an image rendering model. This application also relates to a rendering method, device, electronic equipment and computer storage medium.

Background technique

3D reconstruction is the opposite process to rendering, which is the 3D information of a given object or scene, simulating a camera taking a 2D image. Three-dimensional reconstruction is to establish a mathematical model of a three-dimensional object that is suitable for computer representation and processing. Specifically, it is to reconstruct three-dimensional information based on single-view or multi-view images. In other words, input a two-dimensional image and infer the three-dimensional structure. For example, the three-dimensional modeling information of an object displayed on the online service platform is a three-dimensional geometric diagram of the object obtained by reconstructing the three-dimensional information from the two-dimensional image of the actual object. The process of 3D reconstruction of objects includes object segmentation, camera pose estimation, mesh reconstruction and texture recovery.

The object restoration degree in the rendered image obtained during the actual three-dimensional modeling process is low. Therefore, how to improve the object restoration degree in the rendered image obtained by three-dimensional modeling is a problem that needs to be solved.

Contents of the invention

Embodiments of the present application provide a training method for an image rendering model to improve the object restoration degree in the rendered image obtained by the image rendering model. Embodiments of the present application also relate to an image rendering model training device, electronic equipment, and computer storage media. The embodiments of the present application also relate to a rendering method, device and electronic equipment.

Embodiments of the present application provide a training method for an image rendering model, which includes: obtaining an initial image and an initial segmented image of a target object; using a rendering model to be trained to render the initial image and the initial segmented image to obtain the initial The rendered image and the rendered segmented image of the image; based on the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image, the rendering model to be trained is trained; wherein, Training the rendering model to be trained includes: adjusting rendering parameters of the rendering model to be trained.

Optionally, adjusting the rendering parameters of the rendering model to be trained includes: adjusting the texture parameters of the rendering model to be trained.

Optionally, it also includes: updating the adjusted texture parameters to the rendering model to be trained, obtaining a first rendering model, and continuing to render the initial image and the initial segmented image using the rendering model to be trained. , the steps of obtaining the rendered image of the initial image and rendering the segmented image.

Optionally, adjusting the rendering parameters of the rendering model to be trained includes: jointly adjusting the texture parameters and pose parameters of the rendering model to be trained.

Optionally, the method further includes: updating the adjusted texture parameters and pose parameters to the rendering model to be trained, obtaining a second rendering model, and continuing to perform the rendering process of using the rendering model to be trained to compare the initial image and the initial image. The steps of segmenting the image for rendering, obtaining a rendered image of the initial image and rendering the segmented image.

Optionally, adjusting the rendering parameters of the rendering model to be trained includes: jointly adjusting the texture parameters, pose parameters and grid parameters of the rendering model to be trained.

Optionally, it also includes: updating the adjusted texture parameters, pose parameters and grid parameters to the rendering model to be trained, obtaining a third rendering model, and continuing to perform the rendering process of using the rendering model to be trained to the initial image. Rendering with the initial segmented image, obtaining a rendered image of the initial image and rendering the segmented image.

Optionally, adjusting the rendering parameters of the rendering model to be trained includes: jointly adjusting the texture parameters and lighting parameters of the rendering model to be trained.

Optionally, it also includes: updating the adjusted texture parameters and lighting parameters to the rendering model to be trained, obtaining a fourth rendering model, and continuing to perform the process of using the rendering model to be trained to perform the segmentation of the initial image and the initial segmentation The steps of rendering the image, obtaining a rendered image of the initial image and rendering the segmented image.

Optionally, it also includes: obtaining the texture data difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image; , and the difference between the initial image and the initial segmented image, training the rendering model to be trained, including: based on the texture data difference, determining the loss value of the rendering model to be trained; based on the The loss value of the rendering model to be trained is used to adjust the texture parameters of the rendering model to be trained.

Optionally, it also includes: obtaining the texture data difference and pose data difference between the rendered image and the rendered segmented image, and the initial image and the initial segmented image; said based on the rendered image and The difference between the rendered segmented image and the initial image and the initial segmented image is used to train the rendering model to be trained, including: based on the texture data difference and the pose data difference, determining the Describe the loss value of the rendering model to be trained; based on the loss value of the rendering model to be trained, jointly adjust the texture parameters and pose parameters of the rendering model to be trained.

Optionally, it also includes: obtaining the rendered image and the rendered segmented image, the texture data difference and the pose data difference between the rendered image and the initial segmented image, and the rendered image and the The grid data difference between the initial images; the said rendering model to be trained is trained based on the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image, The method includes: determining the loss value of the rendering model to be trained based on the texture data difference, pose data difference, and grid data difference; based on the loss value of the rendering model to be trained, determining the loss value of the rendering model to be trained. Texture parameters, pose parameters and mesh parameters are jointly adjusted.

Optionally, it also includes: obtaining the texture data difference between the rendered image and the rendered segmented image, and the initial image and the initial segmented image, and the texture data difference between the rendered image and the initial image. Illumination data difference; training the rendering model to be trained based on the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image includes: based on the texture The data difference and the lighting data difference are used to determine the loss value of the rendering model to be trained; based on the loss value of the rendering model to be trained, the texture parameters and lighting parameters of the rendering model to be trained are jointly adjusted.

Optionally, determining the loss value of the rendering model to be trained includes: determining a first loss value used to characterize the difference in object geometry between the rendering result of the rendering model to be trained and the target object.

Optionally, determining a loss value of the rendering model to be trained based on the texture data difference and the lighting data difference includes: determining a loss value used to characterize the rendering model to be trained based on the texture data difference. The first loss value for the object geometry difference between the rendering result and the target object; based on the lighting data difference, determine the lighting data between the rendering result used to characterize the rendering model to be trained and the target object The second loss value of the difference.

Optionally, it also includes: obtaining the grid data of the target object; the grid data of the target object is obtained in the following manner: obtaining a multi-view initial image of the target object, and the multi-view initial images respectively correspond to The image pose data; construct the grid data of the target object according to the multi-view initial image, the image pose data and the initial segmented image.

Optionally, it also includes: obtaining the lighting data of the rendering result obtained by the rendering model to be trained; the obtaining the lighting data of the rendering result obtained by the rendering model to be trained includes: initializing and setting the differentiable texture data of the target object. , differentiable pose data, differentiable mesh data; input the differentiable texture data, differentiable pose data, and differentiable mesh data of the target object into the lighting model to obtain the lighting data of the rendering result.

Embodiments of the present application also provide a rendering method, which includes: providing a multi-view initial image of a target object to a target rendering model, obtaining an initial segmented image of the target object, texture data, and corresponding bits of the multi-view initial image. pose data and illumination data of the target object; determine the target object according to the multi-view initial image, the initial segmentation image of the target object, texture data, and the pose data corresponding to the multi-view initial image. the rendering grid; according to the texture data, perform a texture mapping operation on the rendering grid of the target object; according to the lighting data, perform lighting processing on the rendering grid that has completed texture mapping to obtain a rendering image.

An embodiment of the present application also provides an electronic device. The electronic device includes a processor and a memory. A computer program is stored in the memory. After the processor runs the computer program, it executes the above method.

Embodiments of the present application also provide a computer storage medium. The computer storage medium stores a computer program. After the computer program is run by a processor, the above method is executed.

Compared with the existing technology, the embodiments of the present application have the following advantages:

Embodiments of the present application provide a training method for an image rendering model, which includes: obtaining an initial image and an initial segmented image of a target object; using a rendering model to be trained to render the initial image and the initial segmented image to obtain the initial The rendering image and the rendered segmented image of the image; the rendering model to be trained is trained based on the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image; wherein, the Training the rendering model to be trained includes: adjusting rendering parameters of the rendering model to be trained.

In the above method, the rendering model to be trained is trained according to the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image; wherein the training of the rendering model to be trained includes: The rendering parameters of the rendering model to be trained are adjusted. This method adjusts the rendering parameters of the rendering model to be trained, so that the difference between the rendered image and the rendered segmented image obtained by the adjusted rendering model and the initial image and the initial segmented image is reduced, and the object restoration in the rendered image is improved. Spend.

Description of drawings

Figure 1 is a schematic diagram of the parameter optimization strategy of the training method of the image rendering model provided by the embodiment of the present application.

Figure 2 is a schematic scene diagram of the training method of the image rendering model provided by the embodiment of the present application.

FIG. 3 is a flow chart of an image rendering model training method provided by the first embodiment of the present application.

Figure 4 is a flow chart of a rendering method provided by the second embodiment of the present application.

FIG. 5 is a schematic diagram of a training device for an image rendering model provided by the third embodiment of the present application.

Figure 6 is a schematic diagram of a rendering device provided by the fourth embodiment of the present application.

FIG. 7 is a schematic diagram of an electronic device provided in the fifth embodiment of the present application.

Detailed ways

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described here. Those skilled in the art can make similar extensions without violating the connotation of the present application. Therefore, the present application is not limited by the specific implementation disclosed below.

The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The descriptions used in this application and the appended claims, such as "a", "first", and "second", etc., are not intended to limit the quantity or sequence, but rather Used to distinguish information of the same type from each other.

First, let’s explain the concepts involved in this application:

Differentiable rendering: The rendering process of traditional graphics models is non-differentiable. Compared with traditional graphics models, differentiable rendering constructs a rendering pipeline so that all aspects of the rendering process can be optimized.

Texture mapping: The process of mapping an image (texture) to the surface of a 3D (three-dimensional, three-dimensional) rendering object is called texture mapping. Texture maps provide rich details to objects and simulate the complex appearance of objects.

Texture reconstruction: Map the geometry through images, and restore the texture map data of the mesh during the three-dimensional model reconstruction process. Texture mapping is a method of texture reconstruction.

Rendering parameters: Rendering parameters refer to the relevant data involved in the entire rendering link. These parameters control the final rendering result. The parameters here include model attributes (such as mesh vertices, patches, UV coordinates, etc.), texture attributes (such as color maps, normal maps, etc.), light source configuration (light source type, light map, etc.), camera configuration (camera position, screen size, etc.), material rendering parameters, etc.

Mesh optimization: The mesh is a geometry composed of triangular patches. Mesh optimization involves fine-tuning each vertex of the triangle of the geometry to improve the accuracy of the mesh vertices.

An embodiment of the present application provides a training method for an image rendering model. Embodiments of the present application also provide an image rendering model training device, electronic equipment, and computer storage media. Embodiments of the present application also provide a rendering method, device, electronic equipment and computer storage medium. Detailed descriptions will be given one by one in the following embodiments.

In order to facilitate understanding of the methods and devices provided by the embodiments of the present application, the background of the embodiments of the present application is first introduced before introducing the embodiments of the present application.

3D reconstruction is the opposite process to rendering, which is the 3D information of a given object or scene, simulating a camera taking a 2D image. Three-dimensional reconstruction is to establish a mathematical model of a three-dimensional object that is suitable for computer representation and processing. Specifically, it is to reconstruct three-dimensional information based on single-view or multi-view images. In other words, input a two-dimensional image and infer the three-dimensional structure. For example, the three-dimensional modeling information of an object displayed on the online service platform is a three-dimensional geometric diagram of the object obtained by reconstructing the three-dimensional information from the two-dimensional image of the actual object. The process of 3D reconstruction of objects includes object segmentation, camera pose estimation, mesh reconstruction and texture recovery.

The object restoration degree in the rendered image obtained during the actual three-dimensional modeling process is low. Therefore, how to improve the object restoration degree in the rendered image obtained by three-dimensional modeling is a problem that needs to be solved.

Through the background introduction of the above content, those skilled in the art can understand the problems existing in the existing technology. Next, the application scenarios of the image rendering model training method of the present application will be described in detail. The training method of the image rendering model provided by this application can be applied to various online service platforms to perform three-dimensional reconstruction of the target object and obtain the application scenario of the three-dimensional reconstructed geometry of the target object. For example, in an online shopping platform, in order to increase users' understanding of all-round information about products, the image rendering model training method provided by this application is used to build a target rendering model. The target rendering model infers and obtains the three-dimensional geometry of the product based on two-dimensional images from multiple perspectives of the product. With this, users can intuitively experience the three-dimensional visual renderings of the product in the store on the online shopping platform, and increase their awareness and understanding of the product. , to increase the user’s ordering rate for products.

The training method of the image rendering model provided by this application trains the rendering model to be trained to obtain the target rendering model. During the training process, the initial image and the initial segmented image of the target object are rendered through the rendering model to be trained, and the rendered image and the rendered segmented image of the initial image are obtained. Compare the rendered image and the initial image, and compare the initial segmented image and the rendered segmented image, and obtain the difference between the rendered image, the rendered segmented image, and the initial image and the initial segmented image. According to the difference, the rendering model to be trained is trained; wherein the training of the rendering model to be trained includes: adjusting the rendering parameters of the rendering model to be trained.

In the above process of training the rendering model to be trained, a loss value used to characterize the object difference between the rendered image and the target object is obtained based on the difference between the rendered image and the initial image. Based on the loss value, the rendering parameters of the rendering model to be trained are adjusted. Rendering parameters include at least one of the following parameters: texture parameters, pose parameters, mesh parameters, lighting parameters, and material rendering parameters. Therefore, adjusting the rendering parameters of the rendering model to be trained may be to adjust at least one of the above rendering parameters in the rendering model to be trained to improve the restoration degree of the target object in the rendering results obtained by the rendering model to be trained.

Among them, the process of adjusting the rendering parameters based on the loss value can be an iterative process. For example, in the first model training process, based on the difference between the rendered image and the initial image, a parameter used to characterize the relationship between the rendered image and the target object is obtained. According to the first loss value of the object geometry difference, the rendering parameters of the rendering model to be trained are adjusted to obtain a rendering model after adjusting the rendering parameters.

After that, the rendering model after adjusting the rendering parameters is used to analyze and process the initial image, and obtain the second difference between the rendered image and the initial image. Based on the second difference, determine the object used to characterize the relationship between the rendered image and the target object. The first loss value of geometry dissimilarity. According to the first loss value, the rendering parameters of the rendering model are adjusted again.

Based on the above process, the rendering parameters of the rendering model to be trained are adjusted a preset number of times to obtain a target rendering model after adjusting the rendering parameters a preset number of times. The target rendering model reduces the difference between the rendered image and the rendered segmented image obtained by the initial image analysis and the initial image and the initial segmented image, which improves the object restoration degree of the target object in the rendering result obtained by the trained rendering model. .

Please refer to FIG. 1 , which is a schematic diagram of the parameter optimization strategy of the training method of the image rendering model provided by the embodiment of the present application.

In Figure 1, through the preset parameter optimization sequence, the rendering parameters of the rendering model to be trained are adjusted so that the difference between the rendering result and the initial image of the target object is reduced. For example, a first loss value characterizing the difference between the rendered image and the initial image for the object geometry is reduced, and a second loss value characterizing the difference between the rendered image and the initial image for the lighting data is reduced. details as follows:

First, the texture is optimized. In other words, by adjusting the texture parameters, the first loss value for the object geometry between the rendering result obtained by the rendering model after adjusting the texture parameters and the initial image of the target object is reduced. By adjusting the texture parameters of the rendering model to be trained a preset number of times, the first loss value is reduced, thereby improving the restoration degree of the target object in the rendering result obtained by the rendering model after adjusting the texture parameters.

The above process is the process of texture optimization for the rendering model to be trained. By adjusting the texture parameters of the rendering model to be trained, the restoration degree of the target object in the rendering results obtained by the rendering model is improved. Secondly, it can also be texture and pose optimization. In other words, the texture parameters and pose parameters of the rendering model to be trained are jointly adjusted to improve the restoration degree of the target object in the rendering results.

Specifically, by jointly adjusting the texture parameters and pose parameters, the first loss value for the object geometry between the rendering result obtained by the rendering model after adjusting the texture parameters and pose parameters and the initial image of the target object is reduced. Based on this, the restoration degree of the target object in the rendering result obtained by adjusting the texture parameters and pose parameters is improved.

The above process is the process of optimizing the texture and pose of the rendering model to be trained. By adjusting the texture parameters and pose parameters of the rendering model to be trained, the restoration degree of the target object in the rendering results obtained by the rendering model is improved. In addition, the texture, pose, and mesh can also be jointly optimized. In other words, the texture parameters, pose parameters, and mesh parameters of the rendering model to be trained are jointly adjusted to improve the restoration degree of the target object in the rendering result.

Specifically, the texture parameters, pose parameters and mesh parameters are adjusted simultaneously to reduce the first difference between the rendering result obtained by the rendering model after adjusting the texture parameters, pose parameters and mesh parameters and the initial image of the target object for the object geometry. loss value. Based on this, the restoration degree of the target object in the rendering result obtained by adjusting the texture parameters, pose parameters and mesh parameters of the rendering model is improved.

The three processes of adjusting model parameters of the rendering model to be trained described above are used to reduce the difference value between the rendering result obtained by the rendering model and the target object with respect to the object geometry. On this basis, texture and lighting can also be jointly optimized. In other words, by adjusting texture parameters and lighting parameters at the same time, the rendering results obtained by the target rendering model obtained through training can have better texture, grid and lighting effects than the target object. Meet the predetermined requirements and ensure the consistency of global texture lighting in the rendering results.

Please refer to FIG. 2 , which is a schematic scene diagram of the training method of the image rendering model provided by the embodiment of the present application. In Figure 2, shoes are taken as an example to introduce the process of three-dimensional geometry reconstruction of shoes.

First, collect the original image of the shoe (the object image of the shoe in Figure 2). The original image collected here can be the original image of the shoe from multiple perspectives, centered on the shoe, and obtained through image acquisition tools located at multiple perspectives of the shoe. Pictures corresponding to each perspective. Secondly, perform segmentation processing on the original image of the shoes, separate the foreground image and background image corresponding to the original image of the shoes, and obtain the segmented image of the original image (sole object segmentation in Figure 2). Then, based on the original image of the shoe, the segmented image of the shoe, and the original image of the shoe from each perspective, the mesh information of the shoe is constructed based on the camera pose information of the object (the object mesh of the shoe in Figure 2).

Provide the original image of the shoe to the initial rendering model to obtain the rendering result of the shoe and the rendered segmentation image corresponding to the rendering result. According to the first comparison result between the original image of the shoe and the rendering result, and the second comparison result between the original segmented image corresponding to the original image of the shoe and the rendered segmented image of the rendering result, determine the distance between the original image of the shoe and the rendering result. First loss value for differences in shoe geometry. According to the first loss value, the rendering parameters of the initial rendering model are adjusted. By adjusting the rendering parameters of the initial rendering model, the difference in the shoe geometry between the rendering results obtained by the rendering model after adjusting the rendering parameters and the target object is reduced, and the restoration degree of the target object in the rendering result obtained by the rendering model after adjusting the rendering parameters is improved.

The first loss value is reduced by adjusting rendering parameters as described above. The rendering parameters include at least one of the following parameters: texture parameters, pose parameters, mesh parameters, and lighting parameters. Therefore, this application can reduce the first loss value by adjusting texture parameters (texture optimization in Figure 2). The calculation method of the first loss value may be to determine the first loss value based on the similarity between the texture data of the rendering result and the texture data of the target object.

This application can also reduce the first loss value by jointly adjusting the texture parameters and pose parameters (such as the joint process of texture optimization and camera pose optimization in Figure 2). The calculation of the first loss value here mainly involves weighted calculation based on the loss value of the texture parameter and the loss value of the pose parameter to obtain the similarity between the rendering result and the original image, and determine the first loss value.

In addition, the relationship between the rendering results and the original image can be simultaneously improved by jointly adjusting the texture parameters, pose parameters and mesh parameters (such as the joint process of texture optimization, camera pose optimization and mesh optimization in Figure 2). similarity, reducing the first loss value. The first loss value here is mainly obtained by weighting the loss value of the texture parameter, the loss value of the pose parameter, and the loss value of the mesh parameter.

The above three methods of adjusting the model parameters of the rendering model to be trained and improving the restoration degree of the target object in the rendering results are mainly to improve the similarity of the object geometry between the rendering results obtained by the rendering model after adjusting the model parameters and the original image. . On this basis, texture optimization and lighting optimization as shown in Figure 2 are performed simultaneously. In other words, on the premise that the pose parameters and grid parameters of the initial rendering model have been optimized to the target state, by adjusting the texture parameters and Lighting parameters so that the rendering results have consistent texture and lighting effects compared to the original image of the shoe.

Embodiments of the present application provide a training method for an image rendering model, which includes: obtaining an initial image and an initial segmented image of a target object; using a rendering model to be trained to render the initial image and the initial segmented image to obtain the initial The rendered image and the rendered segmented image of the image; based on the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image, the rendering model to be trained is trained; wherein, Training the rendering model to be trained includes: adjusting rendering parameters of the rendering model to be trained.

In the above method, the rendering model to be trained is trained according to the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image; wherein the training of the rendering model to be trained includes: The rendering parameters of the rendering model to be trained are adjusted. This method adjusts the rendering parameters of the rendering model to be trained, so that the difference between the rendered image and the rendered segmented image obtained by the adjusted rendering model and the initial image and the initial segmented image is reduced, and the object restoration in the rendered image is improved. Spend.

First embodiment

FIG. 3 is a flow chart of a training method for an image rendering model provided by the first embodiment of the present application. The training method of the image rendering model provided by the first embodiment of the present application will be described in detail below with reference to FIG. 3 .

As shown in Figure 3, in step S301, an initial image and an initial segmented image of the target object are obtained.

This step is used to obtain the initial image and initial segmented image of the target object, and the initial image and initial segmented image are used as the sample image and sample segmented image of the target object. The initial image and the initial segmented image are used to train the rendering model to be trained, thereby improving the similarity between the target object and the actual target object in the rendering result obtained by rendering the trained rendering model based on the initial image.

Here, obtaining the initial image of the target object may include obtaining the multi-view initial image of the target object. For example, taking the target object as the center, collecting the original image of the target object through image acquisition tools in multiple directions of the target object as the initial image of the target object. image.

After obtaining the initial image of the target object, segment the foreground content and background content in the image on the initial image to obtain an initial segmented image of the initial image.

As shown in Figure 3, in step S302, the initial image and the initial segmented image are rendered using a rendering model to be trained, and a rendered image and a rendered segmented image of the initial image are obtained.

This step is used to render the initial image and the initial segmented image through the rendering model to be trained, and obtain the rendered image and the rendered segmented image of the initial image. Therefore, in subsequent steps, the rendered image is compared with the initial image, and the rendered segmented image is compared with the initial segmented image. Based on the comparison results between the two, the rendering parameters of the rendering model to be trained are adjusted.

As shown in Figure 3, in step S303, the rendering model to be trained is trained based on the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image; wherein , the training of the rendering model to be trained includes: adjusting the rendering parameters of the rendering model to be trained.

This step is used to adjust the model parameters of the rendering model to be trained based on the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image, including: adjusting the rendering parameters of the rendering model to be trained. As a result, the difference between the rendering result obtained by the rendering model after adjusting the rendering parameters and the initial image is reduced, and the similarity between the rendering result and the target object is improved.

Among them, the rendering parameters of the rendering model to be trained are adjusted, including the first adjustment method:

The adjusting the rendering parameters of the rendering model to be trained includes: adjusting the texture parameters of the rendering model to be trained.

According to the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image, the texture parameters of the rendering model to be trained are adjusted. This improves the similarity between the rendering results obtained by the rendering model after adjusting the texture parameters and the target object with respect to the object geometry.

Among them, the adjustment of the texture parameters of the rendering model to be trained is determined based on the texture data difference between the rendered image obtained by the rendering model to be trained and the initial image. Therefore, the method further includes: obtaining the texture data difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image; and based on the rendered image and the rendered segmented image, and Using the difference between the initial image and the initial segmented image to train the rendering model to be trained can be achieved in the following ways:

Based on the texture data difference, the loss value of the rendering model to be trained is determined; based on the loss value of the rendering model to be trained, the texture parameters of the rendering model to be trained are adjusted.

Use the rendering model to be trained to obtain the rendered image of the initial image and the rendered segmented image. Get the texture data of the initial image, and the texture data of the rendered image. According to the texture data difference between the texture data of the initial image and the texture data of the rendered image, the loss value between the initial image and the rendered image is obtained.

Wherein, determining the loss value of the rendering model to be trained includes: determining a first loss value used to characterize the difference in object geometry between the rendering result of the rendering model to be trained and the target object.

Therefore, based on the texture data difference between the texture data of the initial image and the texture data of the rendered image, the loss value for the object geometry between the initial image and the rendered image is determined. Based on the loss value for the object geometry, the texture of the rendering model to be trained is Parameters are adjusted. Based on this, the similarity between the object geometry in the rendering result obtained by the rendering model after adjusting the texture parameters and the object geometry of the actual object is improved.

The above-mentioned adjustment process of the texture parameters of the rendering model to be trained may be an iterative process. Therefore, the method provided by the embodiment of this application also includes:

Update the adjusted texture parameters to the rendering model to be trained to obtain a first rendering model, continue to render the initial image and the initial segmented image using the rendering model to be trained, and obtain the initial image Steps to render an image and render a segmented image.

For example, after the first texture parameter adjustment of the rendering model to be trained is performed, the first rendering model after the first adjustment of the texture parameters is obtained, the first rendering model is used to render the initial image and the initial segmented image, and the initial image is obtained. Render image and render split image.

Then, based on the texture data difference between the rendered image, the rendered segmented image, and the initial image and the initial segmented image, a loss value of the first rendering model is determined, and based on the loss value, the texture parameters of the first rendering model are adjusted.

After adjusting the texture parameters for a preset number of times on the first rendering model according to the above process, the target rendering model is obtained. Perform a preset number of texture parameter adjustment processes on the first rendering model to gradually reduce the difference in object geometry between the rendering result obtained by the first rendering model and the target object, and improve the restoration degree of the target object in the rendering result.

The above description is the first adjustment method to improve the restoration degree of the target object in the rendering result by adjusting the texture parameters of the rendering model to be trained.

In addition, adjusting the rendering parameters of the rendering model to be trained may also include a second adjustment method: adjusting the rendering parameters of the rendering model to be trained includes: adjusting the texture parameters and bits of the rendering model to be trained. The posture parameters are jointly adjusted.

By jointly adjusting the texture parameters and pose parameters of the rendering model to be trained, the difference in object geometry between the rendering results obtained by the rendering model after adjusting the texture parameters and pose parameters and the initial image is reduced, and the object geometry is improved. Geometry similarity.

Among them, the joint adjustment of texture parameters and pose parameters of the rendering model to be trained is based on the texture data difference and pose data difference between the rendering results obtained by the rendering model to be trained and the target object. Therefore, it also includes: obtaining the The texture data difference and pose data difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image; the difference between the rendered image and the rendered segmented image and the initial Using the difference between the image and the initial segmented image to train the rendering model to be trained can be achieved in the following ways:

Based on the texture data difference and pose data difference, determine the loss value of the rendering model to be trained; based on the loss value of the rendering model to be trained, combine the texture parameters and pose parameters of the rendering model to be trained Adjustment.

The initial image and the initial segmented image of the target object are provided to the rendering model to be trained, and the rendered image and the rendered segmented image of the initial image are obtained. The texture data difference and pose data difference between the rendered image and the initial image are obtained, as well as the texture data difference and pose data difference between the rendered segmented image and the initial segmented image.

Wherein, determining the loss value of the rendering model to be trained includes: determining a first loss value used to characterize the difference in object geometry between the rendering result of the rendering model to be trained and the target object.

Based on the above differences in texture data and pose data, determine the loss value for the object geometry between the rendering result obtained by the rendering model to be trained and the target object, and adjust the texture parameters and pose parameters based on the loss value. Based on this, the restoration degree of the target object in the rendering results obtained by the rendering model to be trained is improved.

Among them, the process of jointly adjusting the texture parameters and pose parameters of the rendering model to be trained may be an iterative process. details as follows:

After jointly adjusting the texture parameters and pose parameters of the rendering model to be trained, the method further includes: updating the adjusted texture parameters and pose parameters to the rendering model to be trained, obtaining a second rendering model, and continuing to execute the use of the rendering model to be trained. The steps of training a rendering model to render the initial image and the initial segmented image, and obtaining a rendered image of the initial image and rendering the segmented image.

For example, after the first two-parameter joint adjustment is performed on the texture parameters and pose parameters of the rendering model to be trained, a second rendering model after the first two-parameter joint adjustment is obtained. The second rendering model is used to render the initial image and the initial segmented image to obtain the rendered image and the rendered segmented image of the initial image.

Then, based on the texture data difference and pose data difference between the rendered image and the rendered segmented image, and the initial image and the initial segmented image, determine a first loss value of the second rendering model for the object geometry difference, based on the first loss value, continue to adjust the texture parameters and pose parameters of the second rendering model.

After jointly adjusting the texture parameters and pose parameters for a preset number of times on the second rendering model according to the above process, the target rendering model is obtained. Perform a joint adjustment process of texture parameters and pose parameters for a preset number of times on the second rendering model, gradually reducing the difference in object geometry between the rendering results obtained by the second rendering model and the target object, and improving the restoration of the target object in the rendering results. Spend.

What is described above is the second adjustment method to improve the restoration degree of the target object in the rendering result by jointly adjusting the texture parameters and pose parameters of the rendering model to be trained.

In addition, adjusting the rendering parameters of the rendering model to be trained may also include a third adjustment method: adjusting the rendering parameters of the rendering model to be trained includes: adjusting the texture parameters and bits of the rendering model to be trained. The pose parameters and mesh parameters are jointly adjusted.

By jointly adjusting the texture parameters, pose parameters and mesh parameters of the rendering model to be trained, the rendering result obtained by the rendering model after adjusting the texture parameters, pose parameters and mesh parameters is compared with the initial image for the object geometry. Differences are reduced, improving similarity for object geometries.

Among them, the joint adjustment of texture parameters, pose parameters and mesh parameters of the rendering model to be trained is based on the difference in texture data, pose data and mesh data between the rendering results obtained by the rendering model to be trained and the target object. . Therefore, the method further includes: obtaining the rendered image and the rendered segmented image, the texture data difference and the pose data difference between the initial image and the initial segmented image, and the rendered image and the initial image. The mesh data difference between them; the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image is used to train the rendering model to be trained by This is achieved as follows:

Based on the texture data difference, pose data difference, and mesh data difference, determine the loss value of the rendering model to be trained; based on the loss value of the rendering model to be trained, texture parameters of the rendering model to be trained , pose parameters and grid parameters are jointly adjusted.

The initial image and the initial segmented image of the target object are provided to the rendering model to be trained, and the rendered image and the rendered segmented image of the initial image are obtained. Obtain the texture data difference, pose data difference and mesh data difference between the rendered image and the initial image, as well as the texture data difference, pose data difference and mesh data difference between the rendered segmented image and the initial segmented image.

Wherein, determining the loss value of the rendering model to be trained includes: determining a first loss value used to characterize the difference in object geometry between the rendering result of the rendering model to be trained and the target object.

Based on the above differences in texture data, pose data and mesh data, determine the first loss value for the object geometry between the rendering result obtained by the rendering model to be trained and the target object. Based on the first loss value, the rendering result of the rendering model to be trained is determined. Texture parameters, pose parameters and mesh parameters are jointly adjusted. Based on this, the restoration degree of the target object in the rendering results obtained by the rendering model to be trained is improved.

The mesh data difference obtained in the above process is obtained based on the mesh data difference between the mesh data of the rendered image and the mesh data of the target object. Therefore, the method also includes: obtaining the grid data of the target object; the grid data of the target object is obtained in the following manner:

Obtain the multi-view initial image of the target object and the image pose data corresponding to the multi-view initial image; construct the multi-view initial image according to the multi-view initial image, the image pose data and the initial segmented image. The mesh data of the target object.

Among them, the process of jointly adjusting the texture parameters, pose parameters and mesh parameters of the rendering model to be trained can be an iterative process. details as follows:

After jointly adjusting the texture parameters, pose parameters and grid parameters of the rendering model to be trained, it also includes: updating the adjusted texture parameters, pose parameters and grid parameters to the rendering model to be trained to obtain a third rendering model, continue to perform the step of rendering the initial image and the initial segmented image using the rendering model to be trained, and obtaining the rendered image and the rendered segmented image of the initial image.

For example, after the first three-parameter joint adjustment is performed on the texture parameters, pose parameters and mesh parameters of the rendering model to be trained, a third rendering model after the first three-parameter joint adjustment is obtained. The third rendering model is used to render the initial image and the initial segmented image to obtain the rendered image and the rendered segmented image of the initial image.

Then, based on the texture data difference, pose data difference and mesh data difference between the rendered image and the rendered segmented image, and the initial image and the initial segmented image, determine the first loss value of the third rendering model for the object geometry difference. . Based on the first loss value, continue to adjust the texture parameters, pose parameters and mesh parameters of the third rendering model.

According to the above process, the third rendering model is jointly adjusted for a preset number of texture parameters, pose parameters and grid parameters to obtain the target rendering model. The third rendering model performs a joint adjustment process of texture parameters, pose parameters and mesh parameters for a preset number of times, gradually reducing the difference in object geometry between the rendering results obtained by the third rendering model and the target object, and improving the accuracy of the rendering results. Degree of reduction for object geometry.

The above three processes of adjusting the rendering parameters of the rendering model to be trained are to improve the similarity between the rendering results obtained by the adjusted parameters of the rendering model and the target object with respect to the object geometry. On this basis, you can continue to adjust the lighting parameters of the rendering model to be trained to improve the difference in lighting data between the rendering results obtained by the rendering model and the target object.

Therefore, the adjustment of the rendering parameters of the rendering model to be trained in this application also includes a fourth adjustment method: the adjustment of the rendering parameters of the rendering model to be trained includes: adjusting the texture parameters of the rendering model to be trained. and lighting parameters are jointly adjusted.

By jointly adjusting the texture parameters and lighting parameters of the rendering model to be trained, the differences in object geometry and lighting data between the rendering results obtained by the rendering model after adjusting the texture parameters and lighting parameters and the initial image are reduced and improved. The similarity of object geometry and lighting data between the rendering result and the target object.

Among them, the joint adjustment of texture parameters and lighting parameters of the rendering model to be trained can be based on the adjustment of the pose parameters and grid parameters of the rendering model to be trained to reach the target state.

The joint adjustment of texture parameters and lighting parameters of the rendering model to be trained is based on the difference in texture data and lighting data between the rendering results obtained by the rendering model to be trained and the target object. Therefore, it also includes: obtaining the rendering image and the The texture data difference between the rendered segmented image, the initial image and the initial segmented image, and the lighting data difference between the rendered image and the initial image.

The training of the rendering model to be trained based on the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image can be implemented in the following manner:

Based on the texture data difference and the lighting data difference, determine the loss value of the rendering model to be trained; based on the loss value of the rendering model to be trained, combine the texture parameters and lighting parameters of the rendering model to be trained Adjustment.

The initial image and the initial segmented image of the target object are provided to the rendering model to be trained, and the rendered image and the rendered segmented image of the initial image are obtained. Obtain the texture data difference and lighting data difference between the rendered image and the original image.

Wherein, determining the loss value of the rendering model to be trained based on the texture data difference and the lighting data difference includes: determining a rendering result used to characterize the rendering model to be trained based on the texture data difference. A first loss value for the object geometry difference between the target object and the target object; based on the lighting data difference, determine the lighting data difference between the rendering result used to characterize the rendering model to be trained and the target object. the second loss value.

Based on the above difference in texture data, determine the first loss value for the object geometry between the rendering result obtained by the rendering model to be trained and the target object, and based on the above difference in lighting data, determine the difference between the rendering result used to characterize the rendering model to be trained and the target object. The second loss value for the difference in lighting data between the target objects. According to the first loss value, the texture parameters are adjusted, and according to the second loss value, the lighting parameters are adjusted. Based on this, the restoration degree of the target object in the rendering results obtained by the rendering model to be trained is improved.

Among them, the process of jointly adjusting the texture parameters and lighting parameters of the rendering model to be trained may be an iterative process. details as follows:

After jointly adjusting the texture parameters and lighting parameters of the rendering model to be trained, the method further includes: updating the adjusted texture parameters and lighting parameters to the rendering model to be trained, obtaining a fourth rendering model, and continuing to perform the rendering using the rendering model to be trained. The model renders the initial image and the initial segmented image, and obtains a rendered image of the initial image and a step of rendering the segmented image.

For example, after the first two-parameter joint adjustment is performed on the texture parameters and lighting parameters of the rendering model to be trained, a fourth rendering model after the first two-parameter joint adjustment is obtained. The fourth rendering model is used to render the initial image and the initial segmented image to obtain the rendered image and the rendered segmented image of the initial image.

Then, based on the texture data difference between the rendered image and the rendered segmented image, and the initial image and the initial segmented image, a first loss value for the object geometry difference degree of the fourth rendering model is determined. Based on the lighting data difference between the rendered image and the initial image, a second loss value for the lighting data difference of the fourth rendering model is determined. According to the first loss value and the second loss value, continue to adjust the texture parameters and lighting parameters of the fourth rendering model.

After jointly adjusting the texture parameters and lighting parameters for a preset number of times on the fourth rendering model according to the above process, the target rendering model is obtained. The process of jointly adjusting texture parameters and lighting parameters for a preset number of times on the fourth rendering model gradually reduces the difference between the rendering result obtained by the fourth rendering model and the target object, and improves the restoration degree of the target object in the rendering result.

What is described above is the fourth adjustment method for improving the restoration degree of the target object in the rendering result by jointly adjusting the texture parameters and lighting parameters of the rendering model to be trained.

The difference in lighting data obtained between the rendered image and the initial image in the above description is the lighting data difference obtained by comparing the lighting data of the rendered image and the lighting data of the initial image. Therefore, the method also includes: obtaining lighting data of the rendering result obtained by the rendering model to be trained.

The acquisition of lighting data of the rendering results obtained by the rendering model to be trained can be achieved in the following manner:

Initialize and set the differentiable texture data, differentiable pose data, and differentiable mesh data of the target object; input the differentiable texture data, differentiable pose data, and differentiable mesh data of the target object into the lighting model, Obtain the lighting data of the rendering result.

Embodiments of the present application provide a training method for an image rendering model, which includes: obtaining an initial image and an initial segmented image of a target object; using a rendering model to be trained to render the initial image and the initial segmented image to obtain the initial The rendering image and the rendered segmented image of the image; the rendering model to be trained is trained based on the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image; wherein, the Training the rendering model to be trained includes: adjusting rendering parameters of the rendering model to be trained.

In the above method, the rendering model to be trained is trained according to the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image; wherein the training of the rendering model to be trained includes: at least Adjust the rendering parameters of the rendering model to be trained. This method adjusts at least the rendering parameters of the rendering model to be trained, so that the difference between the rendered image and the rendered segmented image obtained by the adjusted rendering model and the initial image and the initial segmented image is reduced, thereby improving the objects in the rendered image. Reduction degree.

Second embodiment

Figure 4 is a flow chart of a rendering method provided by the second embodiment of the present application. The rendering method provided by the second embodiment of the present application will be described in detail below with reference to FIG. 4 . Among them, the rendering method provided by the second embodiment is based on the target rendering model obtained by training in the first embodiment, and the rendering result of the target object is obtained according to the initial image rendering of the target object. For the specific description process, reference may be made to the above scenario embodiment and the description of the first embodiment, and will not be described again here.

As shown in Figure 4, in step S401, the multi-view initial image of the target object is provided to the target rendering model, and the initial segmented image, texture data, and pose data corresponding to the multi-view initial image of the target object are obtained. and lighting data of the target object.

This step is used by the target rendering model to determine the initial segmentation image, texture parameters, pose parameters and lighting parameters of the target object corresponding to the initial image of the target object based on the multi-view initial image of the target object. The data obtained above are used to construct the basic data of the three-dimensional geometry of the target object.

As shown in Figure 4, in step S402, the target object is determined based on the multi-view initial image, the initial segmented image of the target object, texture data, and the pose data corresponding to the multi-view initial image. The rendering grid.

This step is used to construct the rendering grid information of the target object, so as to perform texture mapping processing in the rendering grid of the target object in subsequent steps.

As shown in Figure 4, in step S403, a texture mapping operation is performed on the rendering grid of the target object according to the texture data.

This step is used to perform texture mapping operations in the rendering grid of the target object based on texture parameters.

As shown in Figure 4, in step S404, according to the lighting data, lighting processing is performed on the rendering grid that has completed texture mapping to obtain a rendering image.

This step is used to perform texture mapping processing on the rendering mesh of the target object, and then further perform light mapping processing, so that the finally obtained first rendered image has the texture effect and lighting effect of the object geometry compared with the initial image of the target object. All reached consistency.

The rendering method in this embodiment uses the target rendering model obtained by using the image rendering model training method in the first embodiment, and renders the initial image of the target object to obtain the rendering result. The target rendering model obtained in the first embodiment is a model obtained by performing joint optimization processing of multi-level rendering parameters on the initial rendering model. The model that has been jointly optimized can achieve the loss function value between the obtained rendering result and the initial image to meet the predetermined requirements, improving the degree of object restoration in the rendering result obtained by the target rendering model.

Third embodiment

On the basis of the first embodiment, the third embodiment of the present application provides a training device for an image rendering model. Please refer to Figure 5 , which is a diagram of a training device for an image rendering model provided by the third embodiment of the present application. Schematic diagram. The training device of the image rendering model shown in Figure 5 includes:

The first obtaining unit 501 obtains the initial image and the initial segmented image of the target object;

Rendering unit 502, configured to render the initial image and the initial segmented image using a rendering model to be trained, and obtain a rendered image and a rendered segmented image of the initial image;

The training unit 503 is configured to train the rendering model to be trained based on the difference between the rendered image and the rendered segmented image and the initial image and the initial segmented image; wherein, the Training the rendering model to be trained includes: adjusting rendering parameters of the rendering model to be trained.

Fourth embodiment

Based on the second embodiment, a fourth embodiment of the present application provides a rendering device. Please refer to FIG. 6 , which is a schematic diagram of a rendering device provided by the fourth embodiment of the present application. The rendering device shown in Figure 6 includes:

The second obtaining unit 601 is used to provide the multi-view initial image of the target object to the target rendering model, and obtain the initial segmentation image of the target object, texture data, pose data corresponding to the multi-view initial image and the Lighting data of the target object;

The second determination unit 602 is configured to determine the rendering network of the target object based on the multi-view initial image, the initial segmentation image of the target object, texture data, and the pose data corresponding to the multi-view initial image. grid;

Execution unit 603, configured to perform a texture mapping operation on the rendering grid of the target object according to the texture data;

The third obtaining unit 604 is configured to perform illumination processing on the rendering grid with completed texture mapping according to the illumination data to obtain a rendering image.

Fifth embodiment

Corresponding to the methods of the first and second embodiments of the present application, a fifth embodiment of the present application further provides an electronic device. As shown in Figure 7, Figure 7 is a schematic diagram of an electronic device provided in the fifth embodiment of the present application. The electronic device includes: at least one processor 701, at least one communication interface 702, at least one memory 703 and at least one communication bus 704; optionally, the communication interface 702 can be an interface of a communication module, such as an interface of a GSM module; processing The processor 701 may be a processor CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The memory 703 may include high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 703 stores programs, and the processor 701 calls the programs stored in the memory 703 to execute the methods of the first and second embodiments of the present invention.

Sixth embodiment

Corresponding to the methods of the first and second embodiments of the present application, the sixth embodiment of the present application also provides a computer storage medium. The computer storage medium stores a computer program, and the computer program is run by the processor to perform the methods of the first and second embodiments.

Although the present application is disclosed as above with preferred embodiments, it is not intended to limit the present application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the present application The scope of protection shall be subject to the scope defined by the claims of this application.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. Memory may include non-permanent storage in computer-readable media, random access memory (RAM), and/or non-volatile memory in the form of read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information may be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), and read-only memory. (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cassettes, tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include non-transitory computer-readable media (Transitory Media), such as modulated data signals and carrier waves.

2. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

It should be noted that the embodiments of this application may involve the use of user data. In actual applications, this can be done in compliance with the applicable laws and regulations of the country where the user is located (for example, the user explicitly agrees, the user is effectively notified, etc.), use user-specific personal data in the scenarios described herein to the extent permitted by applicable laws and regulations.

It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all It is information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or reject.

Claims (20)

1. A method of training an image rendering model, comprising:

obtaining an initial image and an initial segmentation image of a target object;

rendering the initial image and the initial segmentation image by utilizing a rendering model to be trained to obtain a rendering image and a rendering segmentation image of the initial image;

training the rendering model to be trained based on differences between the rendering image and the rendering segmentation image, and the initial image and the initial segmentation image;

wherein the training the rendering model to be trained includes: and adjusting the rendering parameters of the rendering model to be trained.

2. The method of claim 1, wherein the adjusting the rendering parameters of the rendering model to be trained comprises: and adjusting texture parameters of the rendering model to be trained.

3. The method as recited in claim 2, further comprising:

updating the adjusted texture parameters to the rendering model to be trained to obtain a first rendering model, and continuing to execute the steps of rendering the initial image and the initial segmentation image by using the rendering model to be trained to obtain a rendering image and a rendering segmentation image of the initial image.

4. The method of claim 1, wherein the adjusting the rendering parameters of the rendering model to be trained comprises: and carrying out joint adjustment on texture parameters and pose parameters of the rendering model to be trained.

5. The method as recited in claim 4, further comprising:

updating the adjusted texture parameters and pose parameters to the rendering model to be trained to obtain a second rendering model, and continuing to execute the steps of rendering the initial image and the initial segmentation image by using the rendering model to be trained to obtain a rendering image and a rendering segmentation image of the initial image.

6. The method of claim 1, wherein the adjusting the rendering parameters of the rendering model to be trained comprises: and carrying out joint adjustment on texture parameters, pose parameters and grid parameters of the rendering model to be trained.

7. The method as recited in claim 6, further comprising:

updating the adjusted texture parameters, pose parameters and grid parameters to the rendering model to be trained to obtain a third rendering model, and continuing to execute the steps of rendering the initial image and the initial segmentation image by using the rendering model to be trained to obtain a rendering image and a rendering segmentation image of the initial image.

8. The method of claim 1, wherein the adjusting the rendering parameters of the rendering model to be trained comprises: and carrying out joint adjustment on the texture parameters and the illumination parameters of the rendering model to be trained.

9. The method as recited in claim 8, further comprising:

updating the adjusted texture parameters and illumination parameters to the rendering model to be trained to obtain a fourth rendering model, and continuing to execute the steps of rendering the initial image and the initial segmentation image by using the rendering model to be trained to obtain a rendering image and a rendering segmentation image of the initial image.

10. The method as recited in claim 2, further comprising: obtaining texture data differences between the rendered image and the rendered segmented image and the initial segmented image;

the training the rendering model to be trained based on the differences between the rendering image and the rendering segmentation image, and the initial image and the initial segmentation image, includes:

determining a loss value of the rendering model to be trained based on the texture data differences;

And adjusting texture parameters of the rendering model to be trained based on the loss value of the rendering model to be trained.

11. The method as recited in claim 4, further comprising: obtaining texture data differences and pose data differences between the rendered image and the rendered segmentation image and the initial segmentation image;

the training the rendering model to be trained based on the differences between the rendering image and the rendering segmentation image, and the initial image and the initial segmentation image, includes:

determining a loss value of the rendering model to be trained based on the texture data difference and the pose data difference;

and based on the loss value of the rendering model to be trained, carrying out joint adjustment on texture parameters and pose parameters of the rendering model to be trained.

12. The method as recited in claim 6, further comprising: obtaining texture data differences and pose data differences between the rendered image and the rendered segmented image and the initial segmented image, and grid data differences between the rendered image and the initial image;

The training the rendering model to be trained based on the differences between the rendering image and the rendering segmentation image, and the initial image and the initial segmentation image, includes:

determining a loss value of the rendering model to be trained based on the texture data differences, the pose data differences and the grid data differences;

and based on the loss value of the rendering model to be trained, carrying out joint adjustment on texture parameters, pose parameters and grid parameters of the rendering model to be trained.

13. The method as recited in claim 8, further comprising: obtaining texture data differences between the rendered image and the rendered segmented image and the initial segmented image, and illumination data differences between the rendered image and the initial image;

the training the rendering model to be trained based on the differences between the rendering image and the rendering segmentation image, and the initial image and the initial segmentation image, includes:

determining a loss value of the rendering model to be trained based on the texture data differences and the illumination data differences;

And based on the loss value of the rendering model to be trained, carrying out joint adjustment on the texture parameter and the illumination parameter of the rendering model to be trained.

14. The method according to any of claims 10-12, wherein said determining a loss value of the rendering model to be trained comprises:

a first penalty value for object geometry differences between rendering results characterizing the rendering model to be trained and the target object is determined.

15. The method of claim 13, wherein the determining a loss value for the rendering model to be trained based on the texture data differences and the illumination data differences comprises:

determining a first loss value for object geometry degree difference between a rendering result used for representing the rendering model to be trained and the target object based on the texture data difference;

based on the illumination data differences, a second loss value for the illumination data difference degree between the rendering result used for representing the rendering model to be trained and the target object is determined.

16. The method according to claim 1 or 12, further comprising: acquiring grid data of the target object;

The grid data of the target object is acquired by the following method:

acquiring a multi-view initial image of the target object and image pose data corresponding to the multi-view initial image respectively;

and constructing grid data of the target object according to the multi-view initial image, the image pose data and the initial segmentation image.

17. The method as recited in claim 13, further comprising: obtaining illumination data of a rendering result obtained by the rendering model to be trained;

the obtaining the illumination data of the rendering result obtained by the rendering model to be trained comprises the following steps:

setting micro texture data, micro pose data and micro grid data of a target object in an initialized mode;

and inputting the micro-texture data, the micro-pose data and the micro-grid data of the target object into a lighting model to obtain lighting data of the rendering result.

18. A rendering method, comprising:

providing a multi-view initial image of a target object to a target rendering model to obtain an initial segmentation image and texture data of the target object, wherein the multi-view initial image corresponds to pose data and illumination data of the target object respectively;

Determining a rendering grid of the target object according to the multi-view initial image, the initial segmentation image of the target object, texture data and pose data respectively corresponding to the multi-view initial image;

according to the texture data, performing texture mapping operation on the rendering grid of the target object;

and carrying out illumination treatment on the rendering grid with the texture map according to the illumination data to obtain a rendering image.

19. An electronic device comprising a processor and a memory;

the memory having stored therein a computer program which, when executed by the processor, performs the method of any of claims 1-18.

20. A computer storage medium, characterized in that the computer storage medium stores a computer program which, when executed by a processor, performs the method of any of claims 1-18.