KR101630856B1 - Multispectral photometric stereo system and operating method of the same - Google Patents

Abstract

Disclosed are a multispectral photometric stereo system and a method for operating the multispectral photometric stereo system, which can acquire a three-dimensional shape by eliminating inter-reflection from surfaces of an object. The method comprises the steps of: calculating spectral reflectance of a subject by using multispectral images of the subject and a reference object; calculating spectral illuminance by using the multispectral images; eliminating inter-reflection from the multispectral images based on the spectral reflectance of the subject and the spectral illuminance; and performing photometric stereo operation by using a result of the elimination of the inter-reflection from the multispectral images.

Description

TECHNICAL FIELD [0001] The present invention relates to a multispectral photometric stereo system and a method of operating the same.

The present invention relates to imaging techniques.

The photometric stereo technique is a technique for digitizing the three-dimensional shape of an object using information obtained by illuminating the object with illumination, and has been used in many fields over the past 30 years. The binocular stereo technique uses two cameras to acquire depth information, while the photometric stereo technique acquires surface normal information of an object with a single camera while varying illumination conditions. Thus, the photometric stereo technique has the advantage of simply creating a high-resolution normal map.

On the other hand, phenomena that interfere with surface diffuse reflection, which is the fundamental assumption of photometric stereo, such as interreflection, indirect illumination, specular reflection, self shadow, And so on, act as an obstacle to the accurate acquisition of the three-dimensional shape of the object by the photometric stereo method. However, since it is very difficult to remove the mutual reflection on the object surface, it is inevitable to apply the photometric stereo without removing the mutual reflection. This is because three-dimensional information of an object is needed to eliminate the mutual reflection, since the three-dimensional information of the object is the final result obtained by the photometric stereo. As a result, the photometric stereo technique has a limitation in that it can not accurately obtain a three-dimensional shape of a concave object where mutual reflection occurs.

SUMMARY OF THE INVENTION A problem to be solved by the present invention is to provide a multispectral photometric stereo system that obtains a three-dimensional shape by removing mutual reflection on an object surface.

A method of operating a multispectral photometric stereo system according to an embodiment of the present invention includes calculating a reflection function for each wavelength of a photographed object using multispectral images of a photographed object and a reference object, Removing the mutual reflection in the multispectral images based on the reflection function for each wavelength of the object to be imaged and the illuminance for each wavelength by using the mutual reflection in the multispectral images, And performing the photometric stereo using the result of removing the second signal.

Wherein the step of calculating the reflection function for each wavelength of the object to be imaged comprises the steps of: extracting the brightness of the object and the reference object in the image per wavelength channel included in the multi-spectral images; Can be calculated by a reflection function for each wavelength of the object to be imaged.

The step of removing the mutual reflection may distinguish the direct reflection and the mutual reflection radiance at the reflection radiance of each wavelength obtained from the multi-spectral images based on the reflection function for each wavelength of the object to be imaged and the illuminance for each wavelength , It is possible to eliminate the radiation luminance due to the mutual reflection at the reflection emission luminance for each wavelength.

The step of removing the mutual reflection may be a natural number smaller than the number of wavelength channels by dividing the energy due to the direct reflection and the mutual reflection by obtaining a solution of the n-th order polynomial related to the reflection radiance of each wavelength.

Wherein the step of performing the photometric stereo comprises: performing a photometric stereo using direct reflection included in the multispectral images to obtain surface normal vectors, restoring the three-dimensional shape of the object to be imaged using the surface normal vectors The method comprising the steps of:

A method of operating a multispectral photometric stereo system according to another embodiment of the present invention includes the steps of acquiring multispectral images by photographing an object to be imaged and a reference object using a plurality of light sources, Separating the direct reflection and the mutual reflection at the surface of the object to be photographed, and removing the mutual reflection in the multispectral images based on the result of the division.

The step of distinguishing the direct reflection from the mutual reflection comprises the steps of calculating the illuminance for each wavelength and the reflection function for each wavelength of the object using the multi-spectral images, the reflection function for each wavelength of the object, As a basis, it may include a step of distinguishing the direct reflection from the reflection luminance by the wavelength obtained from the multi-spectral images and the radiation luminance by the mutual reflection.

Wherein the step of calculating the reflection function for each wavelength of the object to be imaged comprises the steps of: extracting the brightness of the object and the reference object in the image per wavelength channel included in the multi-spectral images; Can be calculated by a reflection function for each wavelength of the object to be imaged.

Wherein the step of distinguishing the radiation luminance by the direct reflection and the mutual reflection is performed by obtaining a solution of an n-th order polynomial related to the reflection radiance of each wavelength to distinguish energy due to direct reflection and mutual reflection, It can be a small natural water.

The method of operation may further comprise performing a photometric stereo using the result of removing the mutual reflection in the multispectral images.

A multispectral photometric stereo system according to an embodiment of the present invention is a multispectral photometric stereo system that receives multispectral images of an object to be photographed and a reference object, And a computing unit for separating the direct reflection and the mutual reflection, and performing the photometric stereo using the result of removing the mutual reflection in the multi-spectral images.

Wherein the computing unit calculates the illuminance for each wavelength and the reflection function for each wavelength of the object using the multispectral images and calculates a reflection function for each wavelength based on the reflection function for each wavelength and the illuminance for each wavelength, Reflection Radiation by Wavelength It is possible to distinguish between direct reflection and reflection by mutual reflection.

Wherein the computing unit is configured to extract the brightness of the object and the reference object in the image per wavelength channel included in the multi-spectral images, and calculate a brightness ratio of the object and the reference object as a reflection function for each wavelength of the object .

According to the embodiment of the present invention, accurate surface normal vectors can be measured by eliminating mutual reflection. According to the embodiment of the present invention, the three-dimensional shape can be accurately extracted as compared with the conventional method.

1 is a schematic illustration of a multi-spectral photometric stereo system according to an embodiment of the present invention.
Fig. 2 is a graph showing luminance values per spot in a spectral image.
Figure 3 is a comparison of a conventional photometric stereo and a 3D shape obtained by the present invention.
4 is a block diagram of a multi-spectral photometric stereo system in accordance with an embodiment of the present invention.
5 is an exemplary diagram illustrating a setup environment of a multispectral photometric stereo system according to an embodiment of the present invention.
6 is a flowchart of a method for acquiring a three-dimensional shape of a multispectral photometric stereo system according to an embodiment of the present invention.
7 to 10 are the results of comparing the present invention with the prior art.
11 is a block diagram of a computing unit according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as "comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise.

FIG. 1 is a schematic view of a multispectral photometric stereo system according to an embodiment of the present invention, FIG. 2 is a graph showing spot luminance values in a spectral image, and FIG. And comparing the 3D shapes obtained by the present invention.

First, referring to FIG. 1, a photometric stereo system obtains a three-dimensional shape of an object by using information acquired by illuminating an object. When a direct reflection on the object surface is used, a three- Can be estimated accurately. However, when light strikes a diffuse surface with a specific color, it causes not only direct reflection but also repeated reflection between neighboring surfaces. This is called interreflection, especially when light is applied to a concave surface.

In order to improve the performance of the photometric stereo, it is necessary to eliminate mutual reflection. However, the reciprocal reflection can be known from the three-dimensional information of the object, and since the three-dimensional information of the object is the final result obtained with the photometric stereo, it is very difficult to remove the mutual reflection in the photometric stereo.

As a result, since the conventional photometric stereo technique is difficult to remove the mutual reflection, if the shape of the L-shaped object is restored by the conventional photometric stereo technique, Of course.

The multispectral photometric stereo system of the present invention (hereinafter referred to as an "imaging system") 100 solves the problems of conventional photometric stereo techniques through the antireflection method described below, (high-fidelity surface normal).

Referring again to FIG. 1, the imaging system 100 uses multispectral images of an object using each illumination for a photometric stereo to measure a luminance value for each wavelength. The imaging system 100 measures a luminance value for each wavelength channel by using a local bandpass filter, for example, a liquid crystal tunable filter (LCTF) that passes light in a designated spectral range.

Referring to FIG. 2, it can be seen that there is reciprocal reflection in the radiometric power distribution according to the wavelength of each of the points P1-P7 on the multispectral image.

The imaging system 100 models the reciprocal reflections using a multispectral imaging method to distinguish between direct and indirect reflections. The imaging system 100 removes the mutual reflection from the multi-spectral image, obtains the remaining direct reflection, and uses it to restore the shape of the object. Referring to FIG. 3 (b), the imaging system 100 restores the shape of the L-shaped object more precisely than the conventional photometric stereo technique.

FIG. 4 is a block diagram of a multi-spectral photometric stereo system according to an embodiment of the present invention, and FIG. 5 is an exemplary view illustrating a setup environment of a multi-spectral photometric stereo system according to an embodiment of the present invention.

4, the imaging system 100 includes a multispectral imaging device 110, a computing unit 130, and a plurality of lights 150-1 through 150-k for a photometric stereo.

The multispectral imaging device 110 includes a local bandpass filter 111, a lens 113, and a camera 115.

The local bandpass filter 111 may be, for example, a liquid crystal tunable filter (LCTF). The local bandpass filter 111 is a filter that passes a specific wavelength band (also referred to as a "channel" or "wavelength channel") in a spectral region (for example, 440 nm to 720 nm) . ≪ / RTI >

The camera 115 may be a monochrome camera. The monochrome camera uses a 14-bit ADC (analog-to-digital converter), so you can store the image results in 16-bit format to make the most of it.

The camera 115 is set to obtain a linear camera response. For example, you can disable the camera's gamma correction option because the gamma correction process makes the camera response function nonlinear. The multispectral imaging device 110 selects the focal length of the lens so that the vignetting effect does not occur since the local band filter 111 is mounted in front of the camera lens.

Multispectral imaging device 110 measures a multispectral image in each of a plurality of lights 150-1 through 150-n. When the computing unit 130 performs photometric stereo on the basis of information photographed using six lights, the multi-spectral imaging apparatus 110 photographs six multi-spectral images. Here, the multi-spectral image is composed of spectral images photographed for each channel. If images are acquired at intervals of 10 nm (one channel) from 440 nm to 720 nm, the multi-spectral image is composed of images taken at each of the 29 channels.

The plurality of lights 150-1 to 150-k may be, for example, six LED lights. Referring to FIG. 5, a heat sink may be attached to the CREE CXA1512 high power LED. Such a combination provides a stable illumination, reduces the variation over time, and provides a light source with no peak at each light source wavelength band. The illumination is set at an appropriate distance from the object (subject) for the photometric stereo, and is set at 0.62 m to 1 m from the object in Fig.

The computing unit 130 separates the direct reflection and the mutual reflection using the multi-spectral images taken at the multi-spectral imaging device 110. The computing unit 130 can separate the mutual reflection and the direct reflection from the multi-spectral image acquired for each channel, and performs the photometric stereo using the radiation luminance of the direct reflection without mutual reflection. The photometric stereo can use a known method. The computing unit 130 restores the three-dimensional shape of the object using the surface normal vectors obtained by the photometric stereo method. At this time, since the LED illumination is within a distance of 1m, the illumination may not be uniformly irradiated to each part of the object. Therefore, the computing unit 130 can correct the nonuniformity of the illumination based on the illumination profile obtained by photographing the white plate. The computing unit 130 also corrects the imaging results of the multispectral imaging device 110 using the radiative and geometric characteristics of the multispectral imaging device 110 to obtain physically meaningful results. The computing unit 130 may perform a radiometric calibration, a geometric calibration, and the like.

The radiation correction can be performed as follows.

When the camera response function is linear, a linear transformation function between the input radiation luminance and the camera signal values is obtained for each wavelength channel. For radiation correction, you can use the X-rite ColorChecker with 24 color patches. Two LED lights illuminate the ColorChecker from both sides at a 45 degree angle. The spectral photocopier and multispectral imaging device 110 measure the reflected radiation luminance, respectively. Then, using two kinds of measured values, we find the linear mapping function between the input radiation luminance and the camera signal. At this time, the transmittance of LCTF is considered. Once the multispectral imaging is corrected, it is possible to obtain the physical radiation luminance value with any camera signal. That is, the multispectral imaging device 110 may be used as a two-dimensional spectroscopic copier.

The geometric correction can be performed as follows.

]를 이용해서 구할 수 있다. 모든 광원에 대해 기하 보정을 한다.Geometric correction is the task of locating the light source for photometric stereo. When a mirror reflection occurs in a chrome spheres of known diameter, information about the direction of light (R) reflected from that point and the surface normal vector (V) of the chrome spheres can be obtained. Using the symmetry rule of light reflection, the input light direction (L) for the viewing direction (equal to R)

] Can be obtained. Perform geometric correction on all light sources.

In the following, a description will be given of how the computing unit 130 distinguishes direct reflection from mutual reflection based on the radiation luminance per wavelength obtained from a multispectral image. Distinguishing between direct and mutual reflections means that the mutual reflections can be removed from the measured luminance values.

The computing unit 130 models the mutual reflection mathematically as follows.

The repetitive movement of light is mathematically modeled on the input and output directions of light as shown in Equation 1 (JT Kajiya, "The rendering equation," in Proc. ACM SIGGRAPH Computer Graphics '86, vol. , 1986, pp. 143-150).

는 표면 반사 함수이고, N_x는 x에서의 표면 법선 벡터이다.In Equation (1)

Is the surface reflection function, and N _x is the surface normal vector at x.

와 반사 함수

는 빛의 입/출력 방향에 영향을 받지 않는다. 비록 입력 방사휘도

는 입력 각도에 영향을 받지만, 난반사 조건에서의 빛 이동 공식은 수학식 2와 같이 간소화될 수 있다.In the diffuse reflection condition,

And reflection function

Is not influenced by the direction of light input / output. Although the input radiation luminance

Is influenced by the input angle, the light movement formula in the diffuse reflection condition can be simplified as shown in Equation (2).

에 대한 구형적분은 표면

에 대한 면적분으로 바뀔 수 있다. 이렇게 하면 수학식 2에서 방향에 대한 조건이 사라지게 되어 수학식 3과 같이 정리된다.hemisphere

Lt; RTI ID = 0.0 >

Can be changed to an area minute. In this way, the condition for the direction disappears in Equation (2) and is summarized as Equation (3).

는, 비져빌리티(visibility)를 나타내는 이진함수

와, x와 y의 기하학적 상관관계를 나타내는

의 곱이다. 여기서, x는 거리r에서의 반사 표면(reflected surface)이고, y는 조도 표면(illuminating surface)이다.here

Is a binary function that represents visibility

And a geometric relationship between x and y

. Where x is the reflected surface at distance r and y is the illuminating surface.

Equation (3) can be expressed as a discrete matrix-vector format as shown in Equation (4).

은 미소 영역

에서의 방사휘도 벡터(radiance vector)이고,

은 각 미소영역의 발광 방사휘도 벡터(self-emitted radiance vector)이며,

는 각 미소영역의 출력 조도[라디오시티(radiosity)라고도 불린다]이다. 빛의 이동이 평형상태라고 가정하면, 수학식 4는 수학식 5와 같이 표현될 수 있다. 수학식 5를 노이만 시리즈(Neumann series)로 표현하면 수학식 6과 같다.In Equation (4)

The micro region

Gt; is a radiance vector at < RTI ID = 0.0 >

Is a self-emitted radiance vector of each microdomain,

Is also called the output illuminance (also called radiosity) of each microdomain. Assuming that the movement of light is in an equilibrium state, Equation (4) can be expressed as Equation (5). Equation (5) can be expressed as Neumann series (Equation 6).

Referring to Equation (6), the energy that is totally reflected while the light originating from the light source becomes the n-th reflection from the surface is represented by the sum of the n-th polynomial. Based on the light movement expressed by Equation (6), the amount of energy of the light reflected from the surface starting from the light source can be known. Thus, the nth term in Equation (6) corresponds to the energy of the nth reflected light. Since the computing unit 130 can know the amount of energy of the n-th reflected light from the surface of the object, mutual reflection can be removed.

)를 제외하면, 반사 방사휘도(reflected radiance)(

)는 수학식 7과 같이 반사함수(

)에 대한 다항식으로 표현된다. 반사 방사휘도(

)는 광원에서 떨어진 물체를 촬영하여 구할 수 있다.In Equation 6, the first term is the emission luminous intensity (

), The reflected radiance (

) Is a reflection function (

) ≪ / RTI > Reflected emission luminance (

) Can be obtained by photographing an object away from the light source.

이 직접반사에 해당하는 항이고, 나머지 항들이 상호반사에 해당하는 항들이다.As a result, it is possible to distinguish direct reflection from mutual reflection. The first term of equation (7)

Is a term corresponding to direct reflection, and the remaining terms are terms corresponding to mutual reflection.

Conventionally, an experiment using a single color surface and an RGB projector was performed to distinguish direct reflection from mutual reflection, but it is very difficult to shoot an object with different surface functions. Therefore, conventionally, it has not been easy to distinguish mutual reflection.

The present invention utilizes the fact that an object has different reflection characteristics depending on the wavelength in order to solve this difficulty. The values observed for each wavelength channel are considered to have different illuminance and different reflection functions in the same object. That is, the present invention can distinguish the n-th reflected light without changing the color of the illumination or adjusting the reflection characteristic of the object by using the multispectral imaging technique used for measuring the physical luminance value.

)는 수학식 8과 같이

개의 파장 채널들의 n차 다항식으로 표현된다. 수학식 8을 선형 시스템 방정식으로 표현하면 수학식 9와 같다. 수학식 9를 행렬 벡터 형식으로 표현하면 수학식 10과 같다.The reflective emission luminance of (7)

Is expressed by Equation (8)

Order polynomials of the wavelength channels. Equation (8) can be expressed by Equation (9) as a linear system equation. Equation (9) can be expressed as a matrix vector form as shown in Equation (10).

는 상호반사가 포함되어 이미징 장치에 측정된 다분광 방사 휘도 벡터(multispectral radiance vector that includes interreflection)이고,

는 조도의 분광 프로파일(spectral profile of illumination)이며,

는 반사 함수(reflectance polynomials)이다.

의 각 성분은 조도(

)와 반사 함수(

)의 곱이다. In the equations (9) and (10)

Is a multispectral radiance vector that includes interreflection measured at the imaging device,

Is a spectral profile of illumination,

Is reflectance polynomials.

Each component of the illuminance

) And the reflection function (

).

)와 파장별 반사 함수(

) 각각은 기준 물체를 촬영한 다분광 이미지들로부터 구할 수 있다. 이때, 기준 물체는 예를 들면, Spectralon reference tile일 수 있다. 기준 물체(Spectralon)를 촬영 대상(피사체) 옆에 놓아 두면, 촬영 대상을 촬영하면서 기준 물체도 함께 촬영할 수 있다.Illuminance by wavelength (

) And the reflection function by wavelength

) Can be obtained from multi-spectral images of the reference object. At this time, the reference object may be, for example, a Spectralon reference tile. If the reference object (Spectralon) is placed next to the object to be photographed (object), the reference object can also be photographed while the object to be photographed is photographed.

The computing unit 130 obtains a multispectral reflectance of a single color object from the multispectral images of the reference object and the object to be imaged (object). The ratio of the brightness of the convex portion of the object to be photographed to the reference object (Spectralon) is a reflection function for each wavelength of the object to be imaged. The computing unit 130 can obtain a reflection function in the process of photographing the object to be photographed.

If the computing unit 130 solves the linear system of Equation (9), each n-th mutual reflection can be obtained, so that the mutual reflection of each term can be separated.

의 row-rank는 상호반사 회수 n보다 높다. 따라서, 수학식 9의 선형 시스템은 과잉 제한(over-constrained) 상태이므로, least-square 최적화나, QR decomposition 또는 SVD를 통해 풀 수 있다. For example, in the case of 29 channels,

Is higher than the number of mutual reflection n. Thus, the linear system of equation (9) is over-constrained and can be solved through least-square optimization, QR decomposition or SVD.

The computing unit 130 can remove the mutual reflection as many as the number of multispectral imaging channels (for example, 29), but if the number of reflections exceeds a certain degree, it is physically and numerically meaningless. Accordingly, the computing unit 130 can determine a specific value, and cut the polynomial term after a specific value to obtain a solution.

6 is a flowchart of a method for acquiring a three-dimensional shape of a multispectral photometric stereo system according to an embodiment of the present invention.

)를 알 수 있다.Referring to FIG. 6, the imaging system 100 acquires multi-spectral images by photographing an object to be imaged using each light source (S110). At this time, a reference object (Spectralon) for obtaining a reflection function is placed next to the object to be photographed, and the object to be photographed and the reference object are photographed together. Reflection radiance by wavelength from multi-spectral images (

).

)를 계산한다(S120). 촬영 대상의 파장별 반사함수는 파장 채널별 분광 이미지에서 추출된 촬영 대상과 기준 물체의 밝기 비율이다. 특히, 반사함수는 촬영 대상의 볼록한 부분과 기준 물체의 밝기 비율일 수 있다.The imaging system 100 uses the multi-spectral images to determine the reflection function

(S120). The reflection function for each wavelength is the ratio of the brightness of the object and the reference object extracted from the spectral image of each wavelength channel. In particular, the reflection function may be a ratio of the brightness of the reference object to the convex portion of the object to be imaged.

)를 계산한다(S130). 파장별 조도(

)는 Spectralon을 이용하여 구할 수 있다. The imaging system 100 uses multi-spectral images to measure the illuminance

(S130). Illuminance by wavelength (

) Can be obtained by using Spectralon.

) 그리고 파장별 조도(

)를 이용하여 파장별 반사 방사휘도(

)에서 직접반사와 상호반사에 의한 방사휘도를 구분한다(S140). 즉, 이미징 시스템(100)은 파장별 반사함수(

) 그리고 파장별 조도(

)를 이용하여 파장별 반사 방사휘도(

)에 관계된 n(여기서, n은 파장 채널수보다 작은 자연수이다)차 다항식의 해를 구함으로써, 빛의 1차 반사부터 n차 반사 각각에 의한 에너지를 구분할 수 있다.The imaging system 100 measures the reflection function

) And illuminance by wavelength

) Was used to measure the reflected emission luminance per wavelength

(S140). In the case of the reflection brightness, In other words, the imaging system 100 is able to determine the wavelength-

) And illuminance by wavelength

) Was used to measure the reflected emission luminance per wavelength

(N is a natural number smaller than the number of wavelength channels), which is related to the n-th order reflections of the first order light to the n-th order light.

The imaging system 100 performs the photometric stereo using the radiation luminance by direct reflection (S150).

The imaging system 100 reconstructs the three-dimensional shape of the object to be imaged using the surface normal vectors obtained by the photometric stereo (S160).

7 to 11 are the results of comparing the present invention with the prior art.

FIG. 7 is a result of evaluating the mutual reflection elimination algorithm of the imaging system 100 through simulation. The simulation uses a multi-spectral path-tracing renderer and Mitsuba.

Build a virtual environment, apply an orange color to the object, and then illuminate the directional light. The spectrum of the simulated illumination is the same as the spectrum of the LED light source used in the experiment. To obtain the reflection function by wavelength, a virtual white object (reference object) is added in addition to the object to be imaged (Happy Body and Ward). The imaging system 100 has removed up to the fourth interreflections.

Fig. 7 (a) is an input rendered image in which the opaque portions (between the wall and the wall) are brighter than the surrounding due to mutual reflection. 7 (b) and 7 (c) are the results of the imaging system 100 removing the mutual reflection. Figure 7 (d) is a virtual reference image.

Figure 7 (e) is the PSNR between the resulting image (b) produced by the imaging system 100 with mutual reflection removed and the virtually created reference image (d). The image (b) shows almost the same result as the reference image (d) when the third mutual reflection is removed.

Figure 7 (e) shows the PSNR by the method of Liao and colleagues. Referring to Figure 7 (e), the results by imaging system 100 are shown to be better than in the prior art.

Figure 8 shows the accuracy of the 3D shape obtained by the present invention.

Referring to FIG. 8, the geometric accuracy of the present invention is measured using a monochromatic L-shaped object.

FIG. 8A shows a 3D shape restoration result including mutual reflection, and FIG. 8B shows a 3D shape restoration result in which mutual reflection is removed.

8 (a) shows a less concave shape than a right angle, but FIG. 8 (b) has a flat surface, a straight edge, and an accurate cabinet. When calculating the cabinet, the angle between the symmetry points of two opposing surfaces on the surface normal vector map can be calculated by averaging the angles. When the mutual reflections were removed, the cabinet accuracy improved by 16.75% on average for the three test colors (yellow, red, green, and blue).

FIG. 8 (c) is a result of comparing the method of removing multi-spectral mutual reflection according to the present invention, the method of removing mutual reflection using an RGB camera, and the method of removing active mutual reflection using a beam projector.

The RGB camera is calibrated by applying the above-mentioned radiation correction as it is.

The multi-spectral antireflectance method of the present invention shows the highest accuracy with an average cabinet of 91.59 and a small standard deviation of 6.51. The active antireflective method has low accuracy with an average cabinet of 96.01 and a standard deviation of 18.35. Since the multi-spectral antireflective method of the present invention calculates the internal angle per pixel, a high standard deviation means that there is more noise. This means that the multi-spectral antithrombia method of the present invention works better than other methods.

9 shows the performance difference of the mutual reflection elimination method according to the number of input spectrum channels. Three-dimensional object scanning is described, and the subject to be photographed is concave soap. The concave shape maximizes the mutual reflection effect. Since soap has a mirror reflection characteristic, the polarizing filter is stitched in front of the sensor and the light source, respectively, to prevent mirror reflection from coming directly into the sensor.

9 (a) is a three-dimensional shape obtained by using a 3D laser scanner, and is a reference image.

Figure 9 (b) shows a surface normal vector map, which is a pure photometric stereo result without removing the mutual reflection, and a reconstructed 3D shape. The reconstructed 3D shape is flat compared to the reference image.

Figures 9 (c) and 9 (d) are the results of the multispectral antireflective method of the present invention using two types of cameras (RGB camera and multispectral imaging device). 9 (c) shows a flat shape compared to the reference image, but the sharpness at the corner portion is better represented. FIG. 9 (d) is a result of using the 29-channel multispectral photometric stereo system, and restores the 3D shape substantially identical to the reference image. Using a sufficient wavelength channel, the multiresponse mutual reflection elimination method of the present invention can obtain a high-frequency detailed shape of an object surface, which can restore a high-quality surface normal vector map and a 3D shape.

Fig. 10 shows a surface normal vector map and restored shape of a David plaster cast.

Referring to FIG. 10, the white plaster image reduces the stability of the linear system expressed by Equation (9) / (Equation 10) because the reflection function shows almost the same value for each wavelength band. Therefore, it is coated with a matte red paint to have different reflection function values for each wavelength channel.

10 (a) is a three-dimensional shape obtained by using a 3D laser scanner, FIG. 10 (b) is a three-dimensional shape obtained by a method of Liao and colleagues, and FIG. 10 (c) Dimensional shape obtained by using a multi-spectral antireflection method.

It is difficult to measure the exact shape due to the mutual reflection of the twisted head shape on the david. Because 3D laser scanners use trigonometry for 3D shape restoration, they can be used as reference data because the results are not affected by mutual reflection. The method of removing multi-spectral mutual reflection of the present invention in the surface normal vector map and the reconstructed shape shows that the high-frequency detailed information as much as the reference image is well represented.

FIG. 11 shows a three-dimensional face shape obtained by using the conventional method and the multispectral mutual reflection elimination method of the present invention.

11 (a) and 11 (c) are three-dimensional shapes obtained by the conventional method, and FIGS. 11 (b) and 11 (d) show the multi- Is a three-dimensional shape obtained by use. It can be seen that the multiresponse mutual reflection elimination method of the present invention has restored an accurate shape compared to the conventional method.

12 is a block diagram of a computing unit according to an embodiment of the present invention.

12, the computing unit 130 includes hardware including a processor 131, a memory device 133, a storage device 135, and an input / output device 137, Various software / programs are stored in designated places. The hardware has a configuration and performance capable of executing the method of the present invention.

The computing unit 130 loads a program written in a programming language into the hardware so as to execute the multi-spectral mutual reflection elimination method and the 3D shape restoration method using the same. The processor 131 executes the program in combination with hardware such as the memory device 133 to implement the present invention. The computing unit 130 may be a device, such as a computer, a server, a terminal, or the like, having a software / program for driving the method of the present invention and capable of executing the software / program.

The embodiments of the present invention described above are not implemented only by the apparatus and method, but may be implemented through a program for realizing the function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.

Claims (13)

As a method of operating a multispectral photometric stereo system,
Calculating a reflection function for each wavelength of the object to be photographed using multispectral images of the object to be imaged and the reference object,
Calculating illuminance for each wavelength using the multi-spectral images,
Removing the mutual reflection in the multispectral images based on the reflection function for each wavelength of the object and the illuminance for each wavelength; and
Performing photometric stereo using the result of removing the mutual reflection in the multi-spectral images
Lt; / RTI >
The step of calculating the reflection function for each wavelength of the object
Extracting the brightness of the object to be imaged and the reference object in an image for each wavelength channel included in the multi-spectral images, and calculating a brightness ratio of the object to be imaged and the reference object as a reflection function for each wavelength of the object to be imaged Way. delete The method of claim 1,
The step of removing the cross-
The reflection luminous intensity due to the direct reflection and the mutual reflection is distinguished from the reflection luminous intensity for each wavelength obtained from the multispectral images based on the reflection function for each wavelength of the object to be photographed and the illuminance for each wavelength, Thereby eliminating radiation brightness due to mutual reflection. 4. The method of claim 3,
The step of removing the cross-
The solution of the n-th order polynomial related to the reflection radiance of each wavelength is obtained, the energy due to the direct reflection and the mutual reflection is separated,
Wherein n is a natural number less than the number of wavelength channels. The method of claim 1,
Wherein performing the photometric stereo comprises:
Performing photometric stereo using direct reflection included in the multi-spectral images to obtain surface normal vectors,
Reconstructing the three-dimensional shape of the object to be imaged using the surface normal vectors
≪ / RTI > As a method of operating a multispectral photometric stereo system,
Acquiring multi-spectral images by capturing an object to be imaged and a reference object using a plurality of light sources,
Separating the direct reflection and the mutual reflection at the surface of the object to be photographed based on the reflection information for each wavelength of the multispectral images, and
Based on the distinguishing result, removing the mutual reflection in the multi-spectral images
Lt; / RTI >
The step of distinguishing the direct reflection from the mutual reflection
Calculating a reflection function for each wavelength and a reflection function for each wavelength of the object using the multi-spectral images, and
And separating the radiation luminance due to the direct reflection and the mutual reflection in the reflection radiation luminance for each wavelength obtained from the multispectral images based on the reflection function for each wavelength of the object to be photographed and the illuminance for each wavelength,
The step of calculating the reflection function for each wavelength of the object
An operation method of extracting the brightness of the object to be imaged and the reference object in an image for each wavelength channel included in the multispectral images and calculating a brightness ratio of the object to be imaged and the reference object as a reflection function for each wavelength of the object to be imaged . delete delete The method of claim 6,
The step of distinguishing between the direct reflection and the radiation luminance by mutual reflection
The solution of the n-th order polynomial related to the reflection radiance of each wavelength is obtained, the energy due to the direct reflection and the mutual reflection is separated,
Wherein n is a natural number less than the number of wavelength channels. The method of claim 6,
Performing photometric stereo using the result of removing the mutual reflection in the multi-spectral images
≪ / RTI > As a multispectral photometric stereo system,
The method includes receiving multi-spectral images obtained by photographing an object to be imaged and a reference object, distinguishing direct reflection and mutual reflection on a surface of the object to be imaged based on reflection information of each multi-spectral image by wavelength, A computing unit that performs photometric stereo using the result of removing the reflection
Lt; / RTI >
The computing unit
Extracting the brightness of the object to be imaged and the reference object in an image for each wavelength channel included in the multi-spectral images, calculating a brightness ratio of the object to be imaged and the reference object as a reflection function for each wavelength of the object,
Based on the reflection function for each wavelength of the object to be imaged and the illuminance for each wavelength, reflection and mutual reflection at the reflection radiance of each wavelength obtained from the multi-spectral images are calculated by using the multi- Lt; RTI ID = 0.0 > photometric < / RTI > stereo system. delete delete

Images