CN110992459A

CN110992459A - Indoor scene rendering and optimizing method based on partitions

Info

Publication number: CN110992459A
Application number: CN201911066240.0A
Authority: CN
Inventors: 陈旋; 周海; 李芳芳; 席璐
Original assignee: Jiangsu Aijia Household Products Co Ltd
Current assignee: Nanjing Aixiaobao Intelligent Technology Co ltd
Priority date: 2019-11-04
Filing date: 2019-11-04
Publication date: 2020-04-10
Anticipated expiration: 2039-11-04
Also published as: CN110992459B

Abstract

The invention relates to a partition-based indoor scene rendering and optimizing method, and belongs to the technical field of computer graphic processing. The indoor space to be rendered is divided into different functional spaces, each divided functional space contains different placing objects, and each kind of object contains a default space position in the space, so that a user can design an indoor scene conveniently, and the design efficiency is improved. Based on the indoor scene data structure formed by the semi-automatic partition method, a partition object-based rendering method is adopted during scene rendering, and a convolution-based deep learning neural network algorithm is added to perform noise reduction optimization processing on the rendered image. The invention mainly provides a rendering method for an indoor scene aiming at the regional characteristics of an indoor space and the regularity of placing indoor articles, and improves the rendering efficiency and the image rendering quality of the indoor scene.

Description

Indoor scene rendering and optimizing method based on partitions

Technical Field

The invention relates to a partition-based indoor scene rendering and optimizing method, and belongs to the technical field of computer graphic processing.

Background

At present, no special rendering method is available for rendering indoor scenes, and the rendering method is based on a common three-dimensional scene rendering method, and the methods are usually long in rendering time and low in rendering quality.

In addition, in the use of the home decoration design software DR, the static rendering is realized in a manner of performing illumination construction baking (light building) in software. Although the integration of the VRAY plug-in the new version can be translated into VRAY material and rendered in the VRAY renderer, the IRAY renderer for NVIDIA does not have similar tools and flows, and the material files of the home design software cannot be identified and rendered in the IRAY renderer.

Disclosure of Invention

The technical problem to be solved by the invention is as follows: the method is characterized in that a special method based on an indoor space area is provided for solving the problems of long rendering time, large calculated amount and low rendering quality in the graphic rendering process of the indoor scene, and a partition-based indoor scene rendering and object-based rendering optimization method is provided according to the space regional characteristics of the indoor space and the certain regularity characteristics of the placement of objects in the space. In addition, the invention can also solve the problem that the material file cannot be directly processed by the IRAY renderer of NVIDIA when the rendering operation is carried out in the using process of the home design software DR.

The method for semi-automatic partition divides the indoor space to be rendered into different functional spaces, each divided functional space contains different placing objects, each article contains a default space position in the space, a user can design an indoor scene conveniently, and design efficiency is improved. Based on the indoor scene data structure formed by the semi-automatic partition method, a partition object-based rendering method is adopted during scene rendering, and a convolution-based deep learning neural network algorithm is added to perform noise reduction optimization processing on the rendered image.

The indoor scene rendering and optimizing method based on the subareas comprises the following steps:

1, partitioning each room for a house type design needing rendering;

and step 2, rendering each partition obtained in the step 1 respectively, and releasing corresponding rendering resource occupation after rendering of one partition is completed.

In one embodiment, partitioning a room comprises: partitioning is performed according to the function of the room.

In one embodiment, the functions of the room include the following categories: study room, bedroom, bathroom, kitchen or sitting room etc.

In one embodiment, partitioning a room comprises: the space inside the room is divided into: wall areas, areas in the space containing items, areas in the space without items, etc.

In one embodiment, when the placement position of the article to be placed is unknown, the article is placed in the middle of the room, and the position of the article can be adjusted by the user.

In one embodiment, for two adjacent partitions, after one rendering is completed, if the materials of adjacent positions between the two adjacent partitions are similar, the two adjacent partitions are subjected to a fusion linking process.

In one embodiment, in step 1, when performing the partitioning operation, the next-level partitioning operation is performed on the obtained partition.

In one embodiment, the rendering result is subjected to noise reduction processing through a convolutional neural network-based deep learning noise reduction algorithm.

In one embodiment, the above rendering process is performed by house design using home design software, and is performed by an IRAY renderer of NVIDIA.

In one embodiment, the rendering process includes the steps of:

s1, generating a grid object in the design software, and designating a corresponding map file;

s2, storing the mapping data, the material parameters and the UE material reference as a self-defined SX file; storing the chartlet data, the material parameters, the UE material references and the model information as a self-defined mx file;

s3, mapping data and UE texture reference in the SX file and/or the mx file establish association with mdl texture data through a mapping rule;

and S4, rendering the mdl material data through the IRAY renderer to obtain a rendering effect.

In one embodiment, in step S2, the mapping data included in the UE material refers to a mapping file.

In one embodiment, the step S2 is performed on a local computer.

In one embodiment, the step S3 is performed in the server.

In one embodiment, in step S3, the association of the texture data includes three types: FlataParameter, Vector Parameter, and Tex Parameter.

In one embodiment, in step S3, the "high luminosity" in float Parameter is named as "specula" corresponding to mdl Parameter; "color" in Vector Parameter, the corresponding mdl Parameter named "lerp _ color"; "Diffuse" in Tex Parameter, the corresponding mdl Parameter is named "albedo".

Advantageous effects

The area division method mainly aims at the area characteristics of the indoor space and the regularity of the placement of indoor articles, provides a rendering method aiming at the indoor scene, and improves the rendering efficiency and the image rendering quality of the indoor scene.

Drawings

FIG. 1 is a diagram showing an entire house object in an indoor scene

FIG. 2 is a classification tree showing individual house objects

FIG. 3 is a tree representing items of furniture in a room

FIG. 4 is a method of optimizing a rendering process

FIG. 5 is a schematic diagram of a data mapping process in a rendering process using an IRAY renderer in the present invention.

Detailed Description

The invention provides an indoor scene rendering method based on a partition idea, which is a semi-automatic partition method.

The process of the method is detailed as follows:

1. providing an identifier with editable functional attributes according to the particularity of the indoor scene and the purpose of each space; for example: in fig. 1, 1 denotes the entire room object in an indoor scene, which contains a plurality of functional spatial areas, each with its corresponding spatial function, including, for example, study, main bed, sub bed, kitchen, bathroom, etc. 2, 3, 4 in FIG. 1 represent functional more spatial regions in an indoor scene, which can be mapped based on spatial data of the walls and doors that make up the scene;

2. according to the space ID identification and the functional identification of the space area, identifying the space area as the unique rendering space in the rendering scene at this time and identifying the space area as ID _ using, namely the combination of the space ID and the functional identification field; namely: the data of each room is divided at least by the room ID and its corresponding function; for each room, the data information includes both the number of the room and the functional definition of the room. In this way, all rooms in the house are identified.

3. According to the space region identification divided in the above steps and the region boundary information of the space, according to a tree hierarchy organization structure method, organizing and storing the space data of the indoor scene according to a tree structure; in the method, the function division of the type rooms in the house type is considered, the articles in the rooms of the same type have the same or similar properties, and are combined into the same rendering task, so that the resources required by the rendering of the same or similar rendering resources used by the same or similar rendering resources can be immediately released after the rendering is finished, other tasks waiting for the resources can enter the rendering as soon as possible, the GPU utilization rate is improved, and the rendering efficiency is improved.

The first major category of spatial region classification categories in indoor scenes includes: bedrooms, study, kitchen, bathroom, others; wherein bedroom, bathroom, other contain the secondary classification, for example the functional identification contains in the bedroom: primary bed, secondary bed, child room, etc.; the bathroom function identification comprises: a primary toilet and a secondary toilet; other function identifiers include: living room, dining room, etc. The user of the divided region containing the secondary classification can customize the function identification of the divided region. The primary classification level map is shown in fig. 2. In fig. 2, 5 represents a root node of the indoor rendering task data, 6 represents a primary classification node type of the indoor space region division, and 7 and 8 represent bedrooms in the indoor space primary classification node and other secondary classification nodes respectively contained in the bedrooms. In the invention, the purpose of classifying and dividing the functions of each room in the house type is to perform centralized rendering on the rooms with the same function, in the rendering process, the rooms with the same function usually have similar internal materials and similar article arrangement, when the rooms with similar functions are classified into one type of rendering, the rendering processing on the rooms can be completed more quickly, after the rendering of one type of rooms is completed, the thread occupied by a CPU or a GPU can be quitted, so that the system resource is occupied clearly, and the rendering efficiency can be effectively improved in the whole view; in addition, in some cases, if different functional rooms are rendered in a centralized manner, rendering results are sometimes unusable due to great differences in design styles.

4. Next, corresponding furniture decorations are required to be arranged in each functional area, the furniture decorations serve as main articles in an indoor scene and target objects for rendering and drawing the scene, and the target objects are stored in a data storage as sub-objects of the space area, so that automatic partition loading of the scene is facilitated; each space region dividing node comprises a default space placing position containing an article by default, if the selected article does not belong to the default loaded data in the region, the loaded article is located at the center position in the space by default, and the space region article storage data structure is shown in fig. 3. 9 represents a primary classification node in the spatial region partition, 10 represents item objects contained in the spatial region, each item object having its default relative position in the spatial region, and 11 represents a secondary classification item of the items in the spatial region, having an attaching relationship to its upper-layer item. The other node positions in 10 are defaulted to the central position of the space region, and the user can select a more appropriate position in the space through a drag operation in the visual interface after adding. The specific principle of the placement positions of other primary nodes in the space region is shown in fig. 3, and the default space positions are all available. The nodes of each functional area in the graph comprise default space positions of corresponding articles in the space and default loading models; after the user selects the identification of the indoor space area during drawing, the indoor space scene containing the default model can be generated in the three-dimensional scene, the user can select the model which the user wants to load through replacement operation in later use, the placing position of the model in the functional space can be changed through simple interactive operations such as mouse dragging, the operation of the user is facilitated, and the operation efficiency is improved.

5. By the aid of the scene-based semi-automatic partitioning steps, functional areas in the house type are divided and articles and ornaments are arranged, and then the rendering task can be divided into multiple partitions for parallel rendering according to the partitioned indoor scene data structure.

The target object in each functional space area of the indoor space has a default space position in the space area, and can be used as a default initialization attribute value for loading and displaying in the rendering process, so that the object ornaments to be arranged are displayed after being rendered through a default position arrangement result;

in the rendering process, because the most important influence factor in the indoor scene rendering is the illumination parameter, when the rendering task is divided according to the region division, usually, the rendering region of each region is calculated and rendered by a part of adjacent regions more than the actual region, and after the rendering of each region is finished, the rendering part with the same adjacent region is subjected to fusion and linking according to the rendering result values of the adjacent regions calculated according to the rendering results of the adjacent regions by an image fusion algorithm, so that the rendering image of the whole region space is formed. The region division rendering is to reduce the calculated amount of rendering, and the other part is to consider that the same region material or map is the same or similar, so that the occupation of the material or model on GPU and CPU resources during rendering can be reduced according to the region rendering, and the rendering speed is accelerated; when rendering of a certain material or a map in the used area is completed, the memory and video memory resources occupied by the rendering task after the rendering task is completed can be released, and other rendering tasks can use the resources. If the area is not divided, rendering is carried out according to the whole house space, the rendering of objects using certain materials or mapping resources is completed, but the whole rendering task is not completed, and the occupied CPU and GPU resources are not released, so that the occupied resources are wasted, and the resource utilization rate and the rendering efficiency are reduced. Setting a threshold R for merging some similar features; and according to the rendering structure of the region division, searching whether adjacent regions have similar characteristics or not according to a data table of the topological relation among the storage regions, if so, comparing the similarity value with a threshold value R, and if so, merging, namely, the regions are the same region, otherwise, not merging. The method for optimizing the rendering process mainly comprises the following steps:

target detection or an image splitting method is used in each partition, the areas are divided into different drawing grades based on the quadtree, and GPU or CPU resource occupation in drawing is reduced; based on the space region division method, the rendering task can be divided into the structure shown in fig. 4, and the rendering task of the indoor space is performed. In fig. 4, the space to be rendered may be divided into two parts, namely a wall and an indoor space, and the rendering process of the wall is relatively simple and can be completed relatively quickly; meanwhile, the rendering of the indoor space area can be divided into the rendering task containing the target object (such as an article ornament) and the rendering of the area without the article according to the grade, and the rendering of the space without the article can be quickly finished, so that the rendering is carried out after the part of space is divided separately, the whole rendering calculation amount can be effectively reduced, and the system resources can be quickly released after the rendering space is finished.

For the region division and rendering method of the quadtree shown in fig. 4, let R represent the entire image region and select one predicate P. One method of segmenting R is to repeatedly divide the resulting image into four regions again until p (Ri) TRUE is present for any region Ri. Here starting from the whole image. If p (r) ═ FALSE, the image is divided into 4 regions. If the value of P is false for any region, each of the 4 regions is again divided into 4 regions, respectively, and so on. This particular partitioning technique is most conveniently represented in the form of a so-called quadtree (that is, there are exactly 4 subtrees per non-leaf node), as illustrated in the figure. Note that the root of the tree corresponds to the entire image, and each node corresponds to a divided subsection. At this time, only R4 was further subdivided.

The splitting and aggregation are completed by using the key point of the forward speed algorithm. The threshold value is chosen to be the midpoint between the two major peaks in the histogram. Through threshold processing, noise data in the image can be well eliminated.

Rendering results based on the space region objects are merged according to the images, and the regions rendered respectively are synthesized into the rendering space of the whole space region, so that the forced occupation of certain resource on GPU resources is reduced, even though the rendering object using the resource completes rendering, the rendering task of the whole space is not completed, the resources can not be released, and other tasks can not obtain the GPU resources during rendering. For example: if the rendering of the kitchen and the rendering of the restaurant are finished and the rendering of the restaurant is not finished when the rendering of the kitchen and the rendering of the restaurant are simultaneously performed, the rendering results of the kitchen and the restaurant can be integrated through image splicing and fusion, and the rendering process is continuously performed on the fused area, so that the occupation of a GPU in the rendering process of the kitchen can be released; the method can improve the rendering efficiency and the use efficiency of GPU and CPU resources.

And a deep learning noise reduction algorithm based on a convolutional neural network is introduced in the rendering, the size of a convolutional kernel is set only according to the size of the generated image, the result of rendering a certain space object is used as an input data source of a noise reducer, the circular iteration processing is carried out on the result, and the noise reduction processing is carried out on the space rendering result by utilizing GPU resources, so that a high-quality rendering result image is obtained.

In the above rendering process, in order to perform the house type rendering by the IRAY renderer under the NVIDIA hardware condition, a method for transferring the material characteristics and unifying the rendering effect between different material systems is also provided, and the method is implemented by implementing the material data transfer between the home decoration design software DR and the independent renderer IRAY, and the specific process is as follows:

1: the rendering process of the home decoration design software

① for DR design software, first, an externally created mesh object in FBX format is introduced into a material editor, and a desired material is selected from a left-hand material library of the software and added to a material channel of a model.

② opening material details, appointing local mapping in mapping channel, adjusting corresponding material parameters to achieve needed effect, storing in MX. SX file and uploading to server, where the adjustable data includes three types of flow Parameter, Vector Parameter and Tex Parameter.

③ additionally, the designer can select the commodities in the commodity library from the home software and place them in the house type to make the design in the design result obtained in step ②. the stored scheme is clicked to render, the texture data contained in all MX/SX files in the scene is converted into mdl texture data which can be recognized by the cloud renderer through the building server, and the mdl texture data, the mesh data and the light data are reconstructed in the building server again to render the map.

The main invention point of the invention is that MX/SX files generated locally are converted into corresponding mdl material data on the server, so that the mdl material data can be processed by an IRAY renderer running on the server.

More specifically, 2: the specific data conversion method is as follows:

the material editor is used as a commodity input end of the home decoration design software and plays a role in storing model information, mapping data, material parameters and UE material quotation. Storing the mapping data, the material parameters and the UE material quotation as a self-defined SX file, and storing the mapping data, the material parameters, the UE material quotation and the model information as a self-defined mx file.

Mapping data/UE material quotation in the MX/SX file establishes association with mdl material through a mapping rule in BP _ Material Collection, and complete correspondence can be realized. For example, a UE instant material is named as "cloth", and the mdl material corresponding to the UE instant material is named as "cloth. The mapping data contained in the UE texture, such as Diffuse Map, Normal Map, and Roughress Map, is directly assigned to the referenced mapping file.

The UE instanceMaterial and mdl material establish the material parameter association correspondence through the naming mapping in the DT _ Parametermapping. The material parameters are divided into three main categories: float Parameter, Vector Parameter and TexParameter, different subclasses in each class have their own exclusive names, such as "high luminosity" in float Parameter, and the corresponding mdl Parameter is named "specula"; "color" in Vector Parameter, the corresponding mdl Parameter named "lerp _ color"; "Diffuse" in Tex Parameter, the corresponding mdl Parameter is named "albedo".

The mapping of the mdl and ue material shaders is realized through respective material nodes and codes, and all the logical relations are also corresponding. See fig. 1.

In fig. 1, "contrast of map", "superimposed color of map", "luminance of map", "color" and "mixedness" are converted into "albedo _ power", "albedo _ filter", "albedo _ scale", "lerp _ color" and "lerp _ alpha" respectively "

The mdl code corresponding to the code is:

the calculation between float parameters and the calculation between Tex parameters and Vector parameters are included, the obtained result is applied to all the same-class mdl materials, and the data after name mapping can be synchronously transmitted to mdl.

Claims

1. The indoor scene rendering and optimizing method based on the subareas is characterized by comprising the following steps of:

1, partitioning each room for a house type design needing rendering;

2. The partition-based indoor scene rendering and optimization method of claim 1, wherein partitioning a room in one embodiment comprises: partitioning according to the functions of the rooms; in one embodiment, the functions of the room include the following categories: study room, bedroom, bathroom, kitchen or sitting room etc.

3. The partition-based indoor scene rendering and optimization method of claim 1, wherein partitioning a room in one embodiment comprises: the space inside the room is divided into: wall areas, areas in the space containing items, areas in the space without items, etc.

4. The zone-based indoor scene rendering and optimization method of claim 1, wherein in one embodiment, when a placement position of an item to be placed is unknown, the item is placed in the middle of a room, and the position of the item can be adjusted by a user; in one embodiment, for two adjacent partitions, after one rendering is completed, if the materials of adjacent positions between the two adjacent partitions are similar, the two adjacent partitions are subjected to a fusion linking process.

5. The partition-based indoor scene rendering and optimizing method according to claim 1, wherein in step 1, when performing the partition operation, the next-level partition operation is performed on the obtained partition; in one embodiment, the rendering result is subjected to noise reduction processing through a convolutional neural network-based deep learning noise reduction algorithm.

6. The zone-based indoor scene rendering and optimization method of claim 1, wherein the rendering process is performed by a home design software Dramatic Reality for house design and an IRAY renderer of NVIDIA for rendering.

7. The partition-based indoor scene rendering and optimization method of claim 1, wherein the rendering process comprises the steps of: s1, generating a grid object in the design software, and designating a corresponding map file; s2, storing the mapping data, the material parameters and the UE material reference as a self-defined SX file; storing the chartlet data, the material parameters, the UE material references and the model information as a self-defined mx file; s3, mapping data and UE texture reference in the SX file and/or the mx file establish association with mdl texture data through a mapping rule; and S4, rendering the mdl material data through the IRAY renderer to obtain a rendering effect.

8. The partition-based indoor scene rendering and optimizing method of claim 1, wherein in an embodiment, in step S2, the map data included in the UE material refers to a map file.

9. The partition-based indoor scene rendering and optimization method of claim 1, wherein in one embodiment, the step S2 is performed on a local computer.

10. The partition-based indoor scene rendering and optimizing method of claim 1, wherein in one embodiment, the step S3 is performed in a server; in one embodiment, in step S3, the association of the texture data includes three types: float Parameter, Vector Parameter and Tex Parameter; in one embodiment, in step S3, the "high luminosity" in float Parameter is named as "specula" corresponding to mdl Parameter; "color" in Vector Parameter, the corresponding mdl Parameter named "lerp _ color"; "Diffuse" in TexParameter, the corresponding mdl parameter is named "albedo".