KR101853237B1 - 3D geometry denoising method and apparatus using deep learning - Google Patents

Abstract

A 3D geometry denoising method using deep running and an apparatus thereof are disclosed. The 3D geometry denoising method includes the steps of: learning a deep learning model by using a displacement map corresponding to an original 3D geometry model; (b) generating a noise displacement map including a displacement difference with a cage model to which smoothing is applied after receiving the 3D geometry model including noise; (c) acquiring a denoised displacement map by applying the noise displacement map to the learned deep learning model; and (d) restoring the 3D geometry model by applying the denoised displacement map to the cage model. Accordingly, the present invention can perform a denoising operation in various models without complicated preprocessing.

Description

TECHNICAL FIELD The present invention relates to a 3D geometry denoising method and apparatus using deep running,

BACKGROUND OF THE INVENTION Field of the Invention [0001] The present invention relates to a three-dimensional geometry dew pointing method using deep running and an apparatus therefor.

As a conventional technique using learning for dinoing of a three-dimensional model, there is a method of using a stepwise regression analysis of a normal in a three-dimensional polygonal model. In the prior art, a filtered Facet Normal Descriptor (FND) extracted from model information of a model is applied to a regression analysis to generate a dinoized mesh, which is relatively intuitive and shows excellent performance in terms of using only normal information.

However, the conventional technique can be applied only to the polygon model, and there is a limitation in that accuracy is lowered in a high-resolution model having a small polygon size and a large number of vertices.

(01) P. S. Wang, Y. Liu and X. Tong, "Mesh Denoising via Cascaded Normal Regression ", ACM Trasactions on Graphics, 35 (6), pp. 232: 1-232: 12, 2016.

The present invention is to provide a 3D geometry dew pointing method and apparatus using deep running capable of grooming in various models without complicated preprocessing unlike the prior art.

In addition, the present invention provides a three-dimensional geometric dew pointing method and apparatus using deep running capable of restoring main details of a three-dimensional geometric model while removing only noise.

According to an aspect of the present invention, a three-dimensional geometric dNG method using deep running is provided.

According to an embodiment of the present invention, there is provided a method of estimating a 3D model, comprising: (a) learning a deep learning model using a displacement map corresponding to an original three-dimensional geometric model; (b) generating a noise displacement map including a displacement difference between a three-dimensional geometric model including noise and a cage model to which smoothing is applied; (c) applying the noise displacement map to the learned deep learning model to obtain a dinoized displacement map; And (d) restoring the three-dimensional geometric model by applying the glazed displacement map to the cage model.

The step (a) may include generating a cage model in which smoothing is applied to the original three-dimensional geometric model; Generating a displacement map between the original three-dimensional geometric model and the cage model; And learning to output the deformation-based displacement map by using the depletion model using the displacement map.

The step (a) may include: generating a noise model by adding noise to the original three-dimensional geometric model using a Gaussian random function; And generating a noise displacement map between the original three-dimensional geometric model and the noise model, wherein the noise displacement map may further be used when learning the deep learning model.

The step (a) may include the steps of: randomly dividing the displacement map into patches of a predetermined size; Expanding each patch by randomly rotating each of the divided patches; And a step of inputting the divided patch and the rotated patch into a deep learning model and outputting a dinoized displacement map, respectively.

The cage model may be generated by applying Laplacian smoothing.

Wherein the displacement map is generated including displacement information for each vertex by calculating a distance from the cage model to a surface corresponding to the three-dimensional geometric model in a normal direction of each vertex, The pixel color value of the area in which the pixel value is not present may be generated by designating a constant value different from the displacement information.

According to another aspect of the present invention, there is provided an apparatus for gonioscopic 3D geometry reconstruction.

According to one embodiment of the present invention, a computing device comprises: a memory for storing at least one instruction; And a processor coupled to the memory and executing instructions stored in the memory, the instructions executed by the processor comprising instructions for: (a) learning a deep learning model using a displacement map corresponding to the original three- ; (b) generating a noise displacement map including a displacement difference between a three-dimensional geometric model including noise and a cage model to which smoothing is applied; (c) applying the noise displacement map to the learned deep learning model to obtain a dinoized displacement map; And (d) performing the step of restoring the three-dimensional geometric model by applying the de-NOxing displacement map to the cage model.

According to an embodiment of the present invention, there is provided a method of 3D geometry grooming using deep running and an apparatus therefor, which is advantageous in that gowning can be performed in various models without complicated preprocessing unlike the prior art.

In addition, the present invention has an advantage in that main details of a three-dimensional geometric model are preserved, but only noise is removed and restored.

1 is a flowchart illustrating a three-dimensional geometric grooming method using deep learning according to an embodiment of the present invention.
FIG. 2 is a flowchart illustrating a method of learning a deep learning model according to an embodiment of the present invention; FIG.
3 is a diagram illustrating an original three-dimensional geometric model and a cage model generated based thereon according to an embodiment of the present invention.
FIG. 4 illustrates a displacement between an original 3D geometry model and a cage model according to an embodiment of the present invention; FIG.
5 illustrates a displacement map according to an embodiment of the present invention.
6 is a view showing a restoration result according to an embodiment of the present invention.
FIG. 7 is a block diagram schematically illustrating an internal configuration of a computing device according to an embodiment of the present invention; FIG.

As used herein, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. In this specification, the terms "comprising ", or" comprising "and the like should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps. Also, the terms "part," " module, "and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a flowchart illustrating a method of learning a deep learning model according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating a method of learning a deep learning model according to an embodiment of the present invention. 3 is a diagram illustrating an original three-dimensional geometric model and a cage model generated based on the original three-dimensional geometric model according to an embodiment of the present invention. FIG. 4 is a diagram illustrating a displacement between an original three- FIG. 5 is a view showing a displacement map according to an embodiment of the present invention, and FIG. 6 is a view showing a restoration result according to an embodiment of the present invention.

In operation 110, the computing device 100 learns a deep learning model using a displacement map.

In order to facilitate understanding and explanation, a method of learning a deep learning model will be described in detail with reference to FIG.

In step 210, the computing device 100 receives the original three-dimensional geometric model. In this specification, it is assumed that the original three-dimensional geometric model is an original model including details about the texture.

In step 215, the computing device 100 creates a cage model for the original three-dimensional geometric model. The computing device 100 proceeds to parameterization of the UV map so as to easily change the UV coordinates of the original three-dimensional geometry model and expand it in the form of a texture map, in order to generate a displacement map.

When the parameterization of the 3D original geometry model is completed, the computing device 100 applies Laplacian smoothing to generate a cage model in which texture details are removed separately from the original model. In order to generate the cage model, the application process of laplacian smoothing can be repeated within n (for example, n is a natural number) times. By performing laplacian smoothing multiple times on the parameterized original model, a cage model can be created in which the detailed representation of the surface is removed.

This can be expressed by the following equation (1).

은 스무딩된 모델을 나타내며,

는 i번째 노드에 해당하는 로컬 기하의 중심점을 나타낸다.here,

Represents a smoothed model,

Represents the center point of the local geometry corresponding to the i-th node.

)에 대해 N(자연수)개의 이웃 정점들을 고려한 스무딩된 모델(

)이 생성될 수 있다. The center point of the local geometry corresponding to the i-th node

) Smoothed model considering N (natural numbers) neighboring vertices

Can be generated.

In operation 220, the computing device 100 derives a difference between the original three-dimensional geometric model and the cage model to generate a displacement map.

When a smoothing cage model is generated for a 3D original geometry model, the displacement difference between the original geometry model and the cage model is calculated and stored on the parameter plane again to generate a displacement map to be used for the training.

The displacement information can be calculated through Equation (2).

The displacement map can be obtained by calculating the distance from the cage model to the vertex corresponding to the original model in the normal direction of the vertex.

The computing device 100 may designate the color value of the empty space in which the displacement information is not stored on the displacement map as the first value (e.g., 127).

For example, when creating a displacement map for each model, one displacement map can be generated for each model. In order to generate a constant displacement map, the real-valued displacement data is stored in the texture coordinates corresponding to the image of the resolution (1024 x 1024) designated, and a constant displacement that is not dependent on the resolution (number of vertices) You can create a map.

FIG. 3 illustrates an original geometric model and a cage model generated based on the original geometric model. FIG. 4 illustrates a displacement of a partial region of the two models of FIG. 3. FIG. As shown in FIG.

As shown in FIG. 3, it can be seen that the cage model has a more detailed representation than the original model. As the detailed representation is removed, there is a displacement difference between the original model and the cage model.

When the displacement map is generated, in step 225, the computing device 100 divides the displacement map into a patch shape of a predetermined size to generate patch data.

That is, the computing device 100 can generate the patch data by dividing the displacement map into patch units of an appropriate size so that geometric feature learning is easy. For example, suppose the resolution of the displacement map is 1024 x 1024 as in the original geometric model. The computing device 100 can generate 64 pieces of patch data by dividing the displacement map into a patch having a size of 128 x 128. [

In actual training, not only 64 pieces of patch data are used, but patch data can be generated by randomly rotating each patch data at various angles by a maximum of 10 degrees in order to construct a structure that is robust against rotation when restoring a geometric model. With this, if the patch size is 128 x 128, more than 640 patch data per model can be used for training.

In operation 230, the computing device 100 learns the deep learning model using the patch data.

Here, the deep learning model is formed by modifying the VDSR. The deep learning model according to an embodiment of the present invention includes a convolution filter, a batch normalization, and a ReLU as one set, and is composed of 20 layers in total. The patch learning data of the displacement map, which is input data, The displacement map of 1024 resolution is restored as a training result.

During the learning of the deep learning model, the patch data is processed by using both the displacement map extracted between the original model and the smoothed cage model and the displacement map extracted from the model adding the noise to the original model.

For example, the computing device 100 may add noise to the original geometry model using a Gaussian random function. In order to facilitate understanding and explanation, a model in which noise is added to the original geometric model using a Gaussian random function will be referred to as a noise model.

In this specification, it is assumed that a noise model is generated at three levels for the original geometric model, and then each of the displacement maps is generated and used as learning data of the deep learning model in order to facilitate understanding and explanation. Of course, it should be understood that each displacement map is inputted as patch data of a predetermined size.

The equation of the Gaussian random function for adding noise to the original geometric model is shown in Equation (3).

)를 0.01, 0.02, 0.03과 같이 달리 적용하여 각 정점의 노멀 방향으로의 정점의 변위를 변경함으로써 노이즈 모델을 각각 생성할 수 있다. The computing device 100 may calculate the standard deviation (< RTI ID = 0.0 >

) Can be applied differently, such as 0.01, 0.02 and 0.03, respectively, so that the noise model can be generated by changing the displacement of the vertex in the normal direction of each vertex.

The method of learning the deep learning model based on the displacement map has been described in detail with reference to FIG.

A method of restoring a three-dimensional geometric model using the deep learning model learned with reference to FIG. 1 will be described later.

In step 115, the computing device 100 receives a three-dimensional geometric model including noise.

In step 120, the computing device 100 generates a noise displacement map for a three-dimensional geometric model (including noise).

At this time, the computing device 100 generates a cage model using smoothing based on a three-dimensional geometric model including noise, and then generates a displacement map between the three-dimensional geometric model and the cage model. Since the cage model is a model in which the details of the geometric model are smoothed, a noise displacement map including displacement information including noise can be generated by calculating the distance difference between each vertex of the three-dimensional geometric model including the cage model and the noise. Here, the method of generating the noise displacement map itself is the same as the method of generating the displacement map described with reference to FIG. 2, and thus a detailed description thereof will be omitted.

In step 125, the computing device 100 applies the learned de-learning model using the noise displacement map to obtain a de-novoized displacement map. Here, the degaussed displacement map may have the same resolution as the original resolution.

In operation 130, the computing device 100 applies the acquired de-NOx displacement map to the cage model to restore the 3D geometry model.

For example, the computing device 100 may map the texture coordinates stored in the smoothed cage model to the dinoized displacement map. Then, by changing the displacement of the vertexes included in the model by increasing or decreasing the displacement value of the vertex in the normal direction based on the displacement information of the dinoized displacement map at the same position as the texture coordinates of each vertex of the smoothed cage model, Dimensional geometry model can be restored.

FIG. 6 is a diagram showing the error verification result of the restoration model for each noise level.

FIG. 6 shows a result of calculating an error between the original model including the detail using the Hausdorff distance and the restored model. At this time, when the range of the RMSE converges to 0.003, it is visually determined that the two models are almost the same.

As shown in FIG. 6, although there is a slight difference according to the noise level, it can be seen that there is virtually no difference between the original model and the two models visually, and it is virtually restored to the same level.

7 is a block diagram schematically illustrating an internal configuration of a computing device according to an embodiment of the present invention.

Referring to FIG. 7, a computing device 100 according to an embodiment of the present invention includes a memory 710 and a processor 715.

Memory 710 is a means for storing at least one instruction.

Processor 715 is a means for interfacing with memory 710 and executing instructions stored on memory 710.

The instructions executed by the processor 715 may be performed by learning a deep learning model based on a displacement map as described with reference to FIGS. 1 to 6, and by using the learned deep learning model, You can run the restore method. This is the same as that described above with reference to FIG. 1 to FIG. 6, so that a duplicate description will be omitted.

In addition, embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Examples of program instructions, such as magneto-optical and ROM, RAM, flash memory and the like, can be executed by a computer using an interpreter or the like, as well as machine code, Includes a high-level language code. The hardware devices described above may be configured to operate as one or more software modules to perform operations of one embodiment of the present invention, and vice versa.

As described above, the present invention has been described with reference to particular embodiments, such as specific elements, and limited embodiments and drawings. However, it should be understood that the present invention is not limited to the above- Various modifications and variations may be made thereto by those skilled in the art to which the present invention pertains. Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

100: computing device
710: Memory
715: Processor

Claims (9)

(a) learning a deep learning model using a displacement map corresponding to an original three-dimensional geometric model;
(b) generating a noise displacement map including a displacement difference between a three-dimensional geometric model including noise and a cage model to which smoothing is applied;
(c) applying the noise displacement map to the learned deep learning model to obtain a dinoized displacement map; And
and (d) restoring the three-dimensional geometric model by applying the de-NOxing displacement map to the cage model.
The method according to claim 1,
The step (a)
Generating a cage model in which smoothing is applied to the original three-dimensional geometric model;
Generating a displacement map between the original three-dimensional geometric model and the cage model; And
And a step of learning to output the deformation-based displacement map by using the depletion model using the displacement map.
3. The method of claim 2,
The step (a)
Generating a noise model by adding noise to the original three-dimensional geometric model using a Gaussian random function;
Further comprising generating a noise displacement map between the original three-dimensional geometric model and the noise model,
Wherein the noise drift map is further used when learning the deep learning model.
3. The method of claim 2,
The step (a)
Dividing the displacement map into random patches of a predetermined size;
Expanding each patch by randomly rotating each of the divided patches; And
And a step of inputting the divided patch and the rotated patch to a deep learning model, respectively, so as to output a dinoized displacement map, wherein the three dimensional geometry dinoing method using deep running.
The method according to claim 1,
Wherein the cage model is generated by applying Laplacian smoothing. ≪ RTI ID = 0.0 > [10] < / RTI >
The method according to claim 1,
Wherein the noise displacement map is generated including displacement information for each vertex by calculating a distance from the cage model to a surface corresponding to the three-dimensional geometric model in a normal direction of each vertex, Wherein a pixel color value of an area in which displacement information is not present is generated by designating a constant value different from the displacement information.
A computer-readable recording medium on which program codes for carrying out the method according to any one of claims 1 to 6 are recorded.
A computing device comprising:
A memory for storing at least one instruction; And
A processor coupled to the memory and executing instructions stored in the memory,
The instructions executed by the processor,
(a) learning a deep learning model using a displacement map corresponding to an original three-dimensional geometric model;
(b) generating a noise displacement map including a displacement difference between a three-dimensional geometric model including noise and a cage model to which smoothing is applied;
(c) applying the noise displacement map to the learned deep learning model to obtain a dinoized displacement map; And
(d) applying the deginated displacement map to the cage model to restore the three-dimensional geometric model.
9. The method of claim 8,
The step (a)
Generating a displacement map between the original three-dimensional geometric model and the cage model to which the smoothing is applied; And
Further comprising the step of learning to output the deformation-based displacement map using the displacement map.

Images