WO2022103190A1

WO2022103190A1 - Method and device for generating texture of three-dimensional model

Info

Publication number: WO2022103190A1
Application number: PCT/KR2021/016503
Authority: WO
Inventors: 박병재; 김희근; 이재민
Original assignee: 주식회사 예간아이티
Priority date: 2020-11-13
Filing date: 2021-11-12
Publication date: 2022-05-19
Also published as: KR20220065301A; KR102551194B1

Abstract

The present invention is to generate a texture of a three-dimensional (3-dimension) model, and an image processing method performed by an image processing device according to an embodiment of the present invention may comprise the steps of: generating a basic texture by using a reference camera among a plurality of cameras for photographing a target; generating at least one candidate image by using at least one candidate camera among the plurality of cameras; and removing noise of the basic texture by using the at least one candidate image.

Description

Method and apparatus for generating a texture of a three-dimensional model

The present invention relates to image processing, and to a method and apparatus for generating a texture of a three-dimensional model.

Three-dimensional (3-demension) modeling refers to modeling or virtualizing the real world or the unreal world in a three-dimensional form and implementing it. Such three-dimensional modeling can be accomplished by expressing a three-dimensional target object as geometric data.

In the past, as the method of constructing a three-dimensional depiction based on the survey performance of the real world was considered inefficient, approximate 3D modeling based on the neck side through photos or site visits was made without using survey results, or Modeling was mainly done. However, in recent years, 3D modeling work for the real world has been gradually progressed due to the development of surveying technology, the development of photogrammetry tools, and the development of computer graphics technology.

As an example, the 3D modeling may be generated by 3D scanning the target object. A picture created by scanning describes the distance from each point of interest to a surface. Accordingly, the three-dimensional position of each point in the picture can be identified. In the case of this technique, a scan operation in several directions is required to obtain information in all directions of the target object, and such a scanning operation takes a considerable amount of time.

An object of the present invention is to provide a method and apparatus for effectively generating a texture of a three-dimensional model.

An object of the present invention is to provide a method and apparatus for effectively removing noise from a texture of a 3D model.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able

An image processing method performed by an image processing apparatus according to an embodiment of the present invention includes generating a basic texture using a reference camera among a plurality of cameras for photographing a target, the plurality of cameras The method may include generating at least one candidate image using at least one candidate camera, and removing noise of the base texture by using the at least one candidate image.

Here, the reference camera may include a camera that includes a photographing surface of the target among the plurality of cameras in a photographing range and has a lens direction most similar to a direction perpendicular to the surface.

In addition, the at least one candidate camera may include at least one camera including a photographing surface of the target among the plurality of cameras in a photographing range.

The removing of the noise may include: determining a corrected value by combining values representing pixels on the same three-dimensional coordinates in the base texture and the at least one candidate image; It may include the step of setting as

Combination of the values includes averaging the values, and weights for the averaging may vary according to a plurality of regions in which the basic texture is divided.

The removing of the noise may include: determining differential values for each candidate image with respect to values representing pixels on the same three-dimensional coordinates in the base texture and the at least one candidate image; This may include applying the value to the base texture.

The removing of the noise includes: morphing a 3-dimensional object according to the shape of the boundary line of the basic texture, and projecting the basic texture onto the morphed 3-dimensional object; The method may include morphing the 3D object according to a shape of a boundary line of the auxiliary texture generated from the at least one candidate image, and projecting the auxiliary texture onto the morphed 3D object.

The removing of the noise may include: obtaining information related to the noise; extracting noise from the basic texture based on the information related to the noise; and removing the noise in an area in which the noise exists. It may include the step of performing an operation for.

The method may further include adjusting the color of the candidate images based on the reference texture. Here, the adjustment value for adjusting the color may be determined based on pixel values of the noise-free portion.

An image processing apparatus according to an embodiment of the present invention includes a processor and a memory. The processor generates a basic texture by using a reference camera among a plurality of cameras for photographing a target, and generates at least one candidate image by using at least one candidate camera among the plurality of cameras; , the noise of the base texture may be removed by using the at least one candidate image.

The computer-readable recording medium according to an embodiment of the present invention may be a computer-readable recording medium in which a computer program for performing the image processing method according to the above-described embodiment is recorded.

According to the present invention, it is possible to generate a texture (texture) of better quality for a three-dimensional (3-dimension) model.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be.

1 is a diagram illustrating an image processing apparatus according to an embodiment of the present invention.

2 is a diagram illustrating an example of a source image capturing environment of a texture of a three-dimensional (3-dimension) model according to an embodiment of the present invention.

3 is a diagram illustrating an embodiment of a procedure for generating a texture of a 3D model according to an embodiment of the present invention.

4A and 4B are diagrams illustrating examples of mapping of a 3D model and a 2D image expressed in a 3D coordinate system according to an embodiment of the present invention.

5 is a diagram illustrating another embodiment of a procedure for generating a texture of a 3D model according to an embodiment of the present invention.

The following is merely illustrative of the principles of the invention. Therefore, those skilled in the art can devise various devices that, although not explicitly described or shown herein, embody the principles of the invention and are included in the spirit and scope of the invention. In addition, it should be understood that all conditional terms and examples listed herein are, in principle, expressly intended only for the purpose of understanding the inventive concept and are not limited to the specifically enumerated embodiments and states as such. .

The above-described objects, features and advantages will become more apparent through the following detailed description in relation to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the invention pertains will be able to easily practice the technical idea of the invention. .

In the specification and claims, terms such as "first," "second," "third," and "fourth," are used, if any, to distinguish between like elements and, if not necessarily, in a specific sequence. or to describe the order of occurrence. It will be understood that the terms so used are interchangeable under appropriate circumstances to enable the embodiments of the invention described herein to operate, for example, in sequences other than those shown or described herein. Likewise, where methods are described herein as comprising a series of steps, the order of those steps presented herein is not necessarily the order in which those steps may be performed, and any described steps may be omitted and/or Any other steps not described may be added to the method.

Also, terms such as "left", "right", "front", "behind", "top", "bottom", "above", "below" in the specification and claims are used for descriptive purposes, It is not necessarily intended to describe an invariant relative position. It will be understood that the terms so used are interchangeable under appropriate circumstances to enable the embodiments of the invention described herein to operate otherwise than, for example, as shown or described herein. As used herein, the term “connected” is defined as being directly or indirectly connected in an electrical or non-electrical manner. Objects described herein as being "adjacent" to one another may be in physical contact with one another, in proximity to one another, or in the same general scope or area as appropriate for the context in which the phrase is used. The presence of the phrase “in one embodiment” herein refers to the same, but not necessarily, embodiment.

In addition, in the specification and claims, references to 'connected', 'connecting', 'fastened', 'fastening', 'coupled', 'coupled', etc., and various variations of these expressions, refer to other elements directly It is used in the sense of being connected or indirectly connected through other elements.

In addition, the suffixes "module" and "part" for components used in this specification are given or used in consideration of ease of writing the specification, and do not have a meaning or role distinct from each other by themselves.

In addition, the terms used herein are for the purpose of describing the embodiments and are not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, 'comprise' and/or 'comprising' means that a referenced component, step, operation and/or element is the presence of one or more other components, steps, operations and/or elements. or addition is not excluded.

In addition, in the description of the invention, if it is determined that a detailed description of a known technology related to the invention may unnecessarily obscure the gist of the invention, the detailed description thereof will be omitted. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Hereinafter, the present invention proposes a method and apparatus for generating a texture of a three-dimensional model. In particular, the present invention proposes a technique for reducing or removing noise included in a source image in generating a texture of a 3D model.

To map a texture to a three-dimensional model, a two-dimensional (2-dimension) image can be used as a source. In this case, the two-dimensional image used as the source may include noise depending on the photographing environment. For example, the noise may be caused by lighting, etc. present in a photographing environment of a two-dimensional image. As a specific example, noise may be caused by distortion of the color of pixels in a captured image due to light reflection of lighting. Therefore, in order to improve the quality of a three-dimensional model, it is necessary to remove or reduce noise in a two-dimensional image used as a source of a texture of the three-dimensional model.

1 is a diagram illustrating an image processing apparatus according to an embodiment of the present invention. Referring to FIG. 1 , the image processing apparatus 100 includes a processor 110 , a memory 120 , a communication unit 130 , an input unit 140 , and an output unit 150 . The configuration of FIG. 1 is exemplary, and some of the listed components may be omitted, or other components not listed may be further included.

The processor 110 controls the operation of the image processing apparatus 100 . In order to perform the image processing method described below, the processor 110 may control at least one of the memory 120 , the communication unit 130 , the input unit 140 , and the output unit 150 . That is, the processor 110 may control other components so that the image processing apparatus 100 performs operations according to various embodiments to be described later.

The memory 120 stores data such as a basic program, an application program, and setting information for the operation of the image processing apparatus 100 . Also, the memory 120 stores temporary data generated during the operation of the image processing apparatus 100 . The memory 120 may provide stored data according to the request of the processor 110 .

The communication unit 130 performs functions for transmitting and receiving signals to and from another external device (eg, an external camera). The communication unit 130 may support wired communication or wireless communication. For example, when performing wireless communication, the communication unit 130 may perform channel encoding, modulation, demodulation, and channel decoding.

The input unit 140 performs functions for detecting a user's input. The input unit 140 may transmit a command or data input by a user to the processor 110 to the processor 110 . To this end, the input unit 140 may include at least one hardware module for input. For example, the hardware module may include at least one of a keyboard, a mouse, a digitizer, a sensor, a touchpad, and a button.

The output unit 150 outputs data and information to be recognized by a user. For example, the output unit 150 may configure and display a visually recognizable screen. However, according to various embodiments, the output unit 150 may be replaced with another device capable of outputting a form recognizable by other senses in addition to being visually recognizable. To this end, the output unit 150 may include at least one hardware module for output. For example, hardware modules include speakers, liquid crystal display (LCD), light emitting diode (LED), light emitting polymer display (LPD), organic light emitting diode (OLED), active matrix organic light emitting diode (AMOLED), and FLED. (Flexible LED) may include at least one of.

Noise removal according to various embodiments of the present disclosure is performed using a plurality of cameras. An environment for acquiring a source image of a texture according to an embodiment of the present invention is shown in FIG. 2 below.

2 is a diagram illustrating an example of a source image capturing environment of a texture of a 3D model according to an embodiment of the present invention. Referring to FIG. 2 , a plurality of

cameras

211 , 212a , 212b , and 212c are disposed around a photographing target 210 for acquiring a texture. One of the plurality of

cameras

211 , 212a , 212b , 212c may be selected as the reference camera 211 , and the

others

212a , 212b , 212c may be used as a candidate camera or a child camera. Here, the reference camera 211 among the plurality of

cameras

211 , 212a , 212b and 212c is at least one of a field of view (FOV), a distance to the target 210, and an angle to the target 210 . can be selected based on For example, a camera in which a surface to be photographed and a lens direction of the target 210 are closest to a vertical direction and all of the targets 210 are included in the photographing area may be selected as the reference camera 211 . In this case, the image captured by the reference camera 2110 becomes a basic texture, and the images captured by the

candidate cameras

212a, 212b, and 212c become an auxiliary texture for compensating for noise of the basic texture.

Here, the plurality of

cameras

211 , 212a , 212b , and 212c may be understood as some components of the image processing apparatus or may be understood as separate apparatuses. That is, referring to FIGS. 1 and 2 , the plurality of

cameras

211 , 212a , 212b , and 212c may be a part of the input unit 140 . Alternatively, the plurality of

cameras

211 , 212a , 212b , and 212c may be connected to another device having a communication function, and the image processing apparatus 100 may receive captured images from the other device through the communication unit 130 . .

3 is a diagram illustrating an embodiment of a procedure for generating a texture of a 3D model according to an embodiment of the present invention. 3 illustrates an operation method of the image processing apparatus 100 of FIG. 1 .

Referring to FIG. 3 , in step 301 , the image processing apparatus generates a basic texture using a reference camera. The image processing apparatus selects a camera having a camera view that best expresses a texture among a plurality of cameras as a reference camera, and generates a basic texture based on an image captured by the reference camera. For example, the camera view that best expresses the texture may mean that the target for photographing for obtaining the texture includes all, and the direction of the view and the angle between the targets are closest to vertical.

In operation 303, the image processing apparatus generates at least one candidate image by using at least one candidate camera. The image processing apparatus acquires at least one image captured by at least one candidate camera different from the primary camera. In this case, the at least one candidate camera has a camera view different from the default camera, but has a camera view representing a texture to be used in a 3D model. For example, the at least one candidate camera may be a camera that captures the target at a different angle or distance from the primary camera.

In operation 305, the image processing apparatus removes noise from the base texture by using at least one candidate image. For example, the image processing apparatus generates at least one auxiliary texture from at least one candidate image, and corrects at least one pixel value of the basic texture using pixel values of at least one auxiliary texture having the same 3D coordinates. .

In the embodiment described with reference to FIG. 3 , the image processing apparatus removes noise from the base texture by using at least one candidate image. In this case, the noise removal operation using the candidate image may be defined in various ways.

According to an embodiment, the image processing apparatus may combine values representing the same pixel having the same 3D coordinates in the base texture and at least one auxiliary texture, and set the combined value as a value of the corresponding pixel. That is, the image processing apparatus may determine a corrected value for each coordinate by combining values of the base texture and at least one auxiliary texture for each 3D coordinate, and set the corrected value as a pixel value on the corresponding coordinate. Here, combining may include an averaging operation.

In this case, the averaging may be performed with the same weight regardless of the region of the texture. However, according to another embodiment, the image processing apparatus may divide the texture into a plurality of regions and apply different weights according to the regions. For example, in an area in which noise is not generated or generated in a small amount, the image processing apparatus may set the weight of the base texture to be relatively higher than that of the auxiliary textures. On the other hand, with respect to an area in which noise is generated above a certain level, the image processing apparatus may set the weights for the auxiliary textures to be relatively higher than that of the basic textures. The adaptive setting of these weights can be made by leveling the level of noise and defining a combination of weights for each level. Alternatively, the adaptive setting of the weight may be made by quantifying the level of noise and defining a weight determining function having a value indicating the level of noise as an input.

According to an embodiment, the image processing apparatus may remove noise by removing a difference in expression between the base texture and the auxiliary textures. In other words, the image processing apparatus may remove a visual expression expressed in the basic texture but not expressed in the auxiliary texture with respect to the same pixel on 3D coordinates. Specifically, the image processing apparatus may remove noise included in the basic texture based on a difference value for each coordinate between the primary texture and the secondary texture. For example, the image processing apparatus generates a plurality of difference values such as 'basic texture-auxiliary texture #1', 'basic texture-auxiliary texture #2', ... 'basic texture-auxiliary texture #n' for each three-dimensional coordinates. created, and the maximum difference value among the difference values for each coordinate may be applied to the corresponding coordinates of the basic texture. For example, 'primary texture-secondary texture#1', 'primary texture-secondary texture#2', ... of 'primary texture-secondary texture#n', 'primary texture-secondary texture#2' has the maximum value. , the image processing apparatus may subtract the 'basic texture-auxiliary texture #2' from the pixel value of the corresponding coordinates.

In this case, according to another embodiment, the noise removal operation of removing the difference in expression may be performed based on the average value of two or more auxiliary textures, not the pixel value of one specific auxiliary texture. For example, the image processing apparatus may operate as follows in units of coordinates. The image processing apparatus classifies the pixel values of the primary texture and the auxiliary texture into a plurality of value sections, and then sets the target value by averaging the pixel values of the textures included in the value section to which the most textures belong. Then, the image processing apparatus sets the current value by averaging the pixel values of at least one texture included in the value section to which the basic texture belongs. Finally, the image processing apparatus uses the difference value between the target value and the current value to correct the pixel value of the corresponding coordinate of the base texture (eg, add the difference value to the pixel value).

As in the embodiment described with reference to FIG. 3 , the image processing apparatus may remove noise from the base texture by using the auxiliary textures. In this case, in order to map each of the textures to a 3D object, morphing and projection operations may be performed. The morphing and projection operations will be described as follows.

4A and 4B are diagrams illustrating examples of mapping of a 3D model and a 2D image expressed in a 3D coordinate system according to an embodiment of the present invention. Figure 4a is a three-dimensional model by loading the mesh of the three-dimensional model in the three-dimensional coordinate system, and then projecting the obtained two-dimensional image into the three-dimensional coordinate system according to the shooting information in the direction taken from the position of the camera in the three-dimensional coordinate system. An example in which a mesh and a two-dimensional image are superimposed is shown. Here, projecting a two-dimensional image into a three-dimensional coordinate system means that the two-dimensional image is captured in the shooting direction of the camera from which the two-dimensional image was acquired at the position of the three-dimensional coordinate system corresponding to the position of the camera from which the two-dimensional image was acquired. It means that the 3D space maps the 2D image according to the acquired angle of view of the camera. This may be performed by setting the user's screen view point to the field of view of the corresponding camera, expressing the 3D model to the user, and superimposing the 2D image therewith. 4A shows a result screen according to this.

As shown in FIG. 4A , the two-dimensional image of the three-dimensional model may not coincide with the corresponding surface of the three-dimensional shape represented by the mesh. In the example of FIG. 4A , one vertex 410 of the 3D model and one vertex 420 in the 2D image are displaced from each other. This phenomenon may occur depending on the distortion characteristics of the camera and the surface area characteristics of the scan target (eg, a transparent surface).

In an embodiment, the image processing apparatus may modify the shape of the 3D model so that the expression of the corresponding part of the 3D model matches the 2D image based on the 2D image representing the specific region of the 3D model. For example, the image processing apparatus may display coordinates representing a part (eg, vertices) of the 3D model mesh in a part (eg, vertices) corresponding to the part (eg, vertices) expressed in the 2D image projected in the 3D coordinate system. : By moving to the coordinates where the vertices are expressed), the mesh of the 3D model and the representation of the 2D image can be matched with each other. Alternatively, the image processing apparatus may display coordinates representing a part (eg, vertices) of the 3D model mesh in a part (eg, vertices) corresponding to the part (eg, vertices) expressed in the two-dimensional image projected in the 3D coordinate system. coordinates expressed) is received, and accordingly, the mesh of the 3D model and the representation of the 2D image can be matched with each other.

As in the example of FIG. 4A , the image processing apparatus may match expressions by moving one vertex 410 of the 3D model to one vertex 420 in the 2D image. The result according to this is shown in FIG. 4B. In this case, the image processing apparatus may record the modified details in order to restore the shape of the 3D model modified later.

In addition, the image processing apparatus may generate a texture using a 2D image corresponding to the mesh of the modified 3D model. For example, the image processing apparatus selects a portion of a two-dimensional image corresponding to a face of a mesh, generates a coordinate correspondence relationship between both data as a UV map, and converts a portion of the two-dimensional image into a texture can be saved as

In addition, the image processing apparatus restores the shape of the modified 3D model to the original shape by using the shape modification history of the 3D model or the 3D model node position correspondence relationship before and after modification, and restores the texture according to the restored shape to the original 3 It can be applied to the texture of the dimensional model. By restoring the shape of the 3D model, even if the mesh of the 3D model is modified, since UV expresses the correspondence between the modified mesh and the 2D image as it is, even on the deformed mesh of the modified 3D model, the A texture by a part may be expressed in a deformed shape.

In addition, when a plurality of cameras are used, by using the views of individual cameras sequentially in a predetermined order and a two-dimensional image generated based thereon, by transforming the shape of the three-dimensional model for each camera looking at the three-dimensional space The texture can be improved. In an embodiment, the image processing apparatus identifies a difference between a shape of a 2D image corresponding to a view of the camera and a shape of a 3D model corresponding thereto for each view of the camera, and a task list for texture improvement in the order of the camera with the highest difference can be created and presented to the user.

A texture creation procedure including the aforementioned morphing and projection operations is described below with reference to FIG. 4 .

5 is a diagram illustrating another embodiment of a procedure for generating a texture of a 3D model according to an embodiment of the present invention. 5 exemplifies an operation method of the image processing apparatus 100 of FIG. 1 .

Referring to FIG. 5 , in step 501 , the image processing apparatus selects a reference camera specified as one surface. According to an embodiment, the image processing apparatus may select a reference camera based on location information, direction information, and location information of a target photographed to obtain a texture and direction information of the cameras. For example, the image processing apparatus may select a camera whose angle between the surface and the lens direction is closest to a right angle as the reference camera while including the entire surface to be used as a texture among the surfaces of the target in the camera view. According to another embodiment, the image processing apparatus may select a reference camera based on images captured by each of the cameras. For example, the image processing apparatus may select a reference camera as a camera that has captured an image having the highest sharpness among captured images. According to another embodiment, the image processing apparatus may display images captured by a plurality of cameras, and select a camera that has captured an image designated by a user as a reference camera.

In operation 403, the image processing apparatus selects a candidate camera group corresponding to the selected reference camera. That is, the image processing apparatus selects other cameras for capturing other images to be used to remove noise of the basic texture obtained by the reference camera. According to an exemplary embodiment, other cameras included in the candidate camera group may be selected from among cameras included in the shooting range without cutting off the shooting surface of the target. According to another embodiment, other cameras included in the candidate camera group may be selected from among cameras including all feature points of textures within a shooting range. According to another embodiment, other cameras included in the candidate camera group may be predefined. That is, a candidate camera group corresponding to each camera may be predefined.

In operation 505, the image processing apparatus morphs the 3D object based on the feature point of the target of the reference camera. In the image obtained by capturing the target, the shape of the surface of the 3D object and the boundary line may not match due to a shooting angle or the like. Accordingly, the image processing apparatus may change the shape of the 3D object to match the shape of the boundary line of the surface to which the texture is to be mapped with the captured image.

In operation 507, the image processing apparatus projects the basic texture captured by the reference camera. In other words, the image processing apparatus maps the basic texture to the morphed 3D object. Here, a mask may be applied to the base texture in consideration of the subsequent projection of the auxiliary textures. Although not shown in FIG. 5 , the image processing apparatus may restore the shape of the morphed 3D object after projection.

In step 509, the image processing device projects the auxiliary texture captured by the candidate camera. In other words, the image processing apparatus maps the auxiliary texture to the 3D object. In this case, the image processing apparatus may morph the 3D object according to the shape of the boundary line of the auxiliary texture and then project the auxiliary texture. In addition, the image processing apparatus may restore the shape of the morphed 3D object. That is, morphing and restoration operations may be performed for each camera before and after texture projection.

In operation 511, the image processing apparatus adjusts a color value based on the color of the reference camera. Color values for the same part may be different from each other due to camera settings that capture the target, product characteristics of the camera, and external interference at the time of shooting. Accordingly, the image processing apparatus adjusts color values of candidate images captured by the candidate cameras based on the reference image captured by the reference camera. For example, the image processing apparatus calculates difference values of pixel values for each coordinate between the reference image and one candidate image, and then calculates the average value of the difference values corresponding to the noise-free portion among all the difference values for all pixels of the candidate image. Values can be reflected (eg subtracted). That is, the average value uses the difference values of the noise-free portion, but the color value adjustment is applied to the entire image. Here, in order to select the difference values corresponding to the noise-free portion, the image processing apparatus may select a portion having a relatively low value from among all the distributions of the difference values. Alternatively, if the location of the noise can be specified, the image processing apparatus may select some difference values based on the coordinate values.

In operation 513, the image processing apparatus merges the projected textures. The image processing apparatus merges the plurality of projected textures. In this case, merging may be performed on the entire texture or part of the texture. According to an embodiment, the image processing apparatus may divide the texture into a plurality of regions, identify at least one region in which noise is present, and then merge the textures with respect to the at least one identified region.

In the embodiment described with reference to FIG. 5 , the image processing apparatus adjusts color values of candidate images according to the reference image. In this case, it has been described that the color values are adjusted based on the analysis result of the pixel values. However, according to another embodiment, the image processing apparatus may generate a plurality of color adjustment results and then display the generated results. Accordingly, when the user designates one result, the image processing apparatus may apply the designated result. This is in consideration of errors in judgment of the noise region, and the like.

According to the above-described embodiments, noise included in the basic texture obtained through the primary camera may be removed by using the auxiliary textures obtained through the candidate cameras. In this case, if the characteristics of noise are known, it is expected that noise can be removed more effectively.

According to an embodiment, the image processing apparatus may utilize information on a cause of noise. For example, noise may be caused by lighting. In this case, the image processing apparatus may obtain information on the location, brightness, color, etc. of the installed lighting, and predict the noise characteristics based on the obtained information. Furthermore, the image processing apparatus may obtain information on the surface material of the target to be photographed, and predict noise characteristics based on the material. When the noise characteristic is predicted, the image processing apparatus may extract the noise from the basic texture based on the noise characteristic. Accordingly, the image processing apparatus may perform a noise removal operation only in an area in which the extracted noise exists. In this case, the noise removal operation is prevented from being performed even in a noise-free region, so that the amount of computation can be reduced.

The image processing apparatus and the image processing method according to the embodiment described above may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured according to the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Each of the drawings referenced in the description of the previous embodiment is merely an embodiment shown for convenience of description, and items, contents, and images of information displayed on each screen may be modified and displayed in various forms.

Although the present invention has been described with reference to an embodiment shown in the drawings, this is merely exemplary, and those of ordinary skill in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical protection scope of the present invention should be defined by the technical spirit of the appended claims.

The content disclosed in this specification may be used to remove noise of a texture.

Claims

In the image processing method performed by the image processing apparatus,

generating a basic texture by using a reference camera among a plurality of cameras for photographing a target;

generating at least one candidate image by using at least one candidate camera among the plurality of cameras; and

and removing noise from the base texture by using the at least one candidate image.
The method according to claim 1,

The reference camera includes, among the plurality of cameras, a photographing surface of the target in a photographing range, and includes a camera having a lens direction most similar to a direction perpendicular to the surface.
The method according to claim 1,

The at least one candidate camera includes at least one camera including a photographing surface of the target among the plurality of cameras in a photographing range.
The method according to claim 1,

The step of removing the noise is

determining a corrected value by combining values representing pixels on the same three-dimensional coordinates in the base texture and the at least one candidate image; and

and setting the corrected value as a value of a corresponding pixel.
5. The method according to claim 4,

Combining the values comprises averaging the values,

The weights for the averaging are different according to a plurality of regions obtained by dividing the basic texture.
The method according to claim 1,

The step of removing the noise is

determining differential values for each candidate image with respect to values representing pixels on the same three-dimensional coordinates in the base texture and the at least one candidate image; and

and applying a maximum value among the difference values to a base texture.
The method according to claim 1,

The step of removing the noise is

morphing a 3-dimensional object according to the shape of the boundary line of the basic texture, and projecting the basic texture onto the morphed 3-dimensional object; and

and morphing the 3D object according to a shape of a boundary line of the auxiliary texture generated from the at least one candidate image, and projecting the auxiliary texture onto the morphed 3D object.
The method according to claim 1,

The step of removing the noise is

obtaining information related to the noise;

extracting noise from the basic texture based on the information related to the noise; and

and performing an operation for removing the noise with respect to the region where the noise exists.
The method according to claim 1,

and adjusting the color of the candidate images based on the reference texture.
10. The method of claim 9,

The adjustment value for adjusting the color is determined based on pixel values of the noise-free portion.
In the image processing apparatus,

processor and memory;

The processor is

A basic texture is generated using a reference camera among a plurality of cameras for photographing a target,

generating at least one candidate image by using at least one candidate camera among the plurality of cameras;

An image processing apparatus for removing noise from the base texture by using the at least one candidate image.
A computer-readable recording medium in which a computer program for performing the method of claim 1 is recorded.