KR101390455B1 - A Physically-based Approach to Reflection Separation - Google Patents

Abstract

The present invention relates to a method for improving the acquisition image quality by physically based reflection separation, which is a completely automatic physically based reflection which deals with double-sided reflection and spatially varying mixing coefficients of both layers in a physically based reflection model. An object of the present invention is to provide a method for improving image quality obtained by separation.
The present invention for achieving this object comprises the steps of: (a) extracting a pair of orthogonal images from the three polarized images each captured at a polarization angle separated by an angle; (b) separating a reflective layer and a background layer from each of the orthogonal images by assuming a spatially varying angle of incidence; (c) purifying the reflective and background layers by applying multi-scale constrained optimization; And (d) an edge removing treatment step remaining on the reflective layer and the background layer purified through step (c). .

Description

A Physically-based Approach to Reflection Separation

FIELD OF THE INVENTION The present invention relates to a method for improving image detail, such as reflections that may occur on images such as photographs of scenes taken through glass windows or photographs of objects placed in glass showcases in shops and museum sets. A method for separation of an image, more particularly an acquisition image by physically based reflection separation for separating reflection and background layers using multiple polarized images when capturing a background scene behind a glass It is about quality improvement method.

Reflection separation methods can be largely classified into two methods: image-based methods, and physically based methods. The present invention proposes a physically based approach for separating reflections using a plurality of polarized images with a background scene captured behind the glass.

The input consists of three polarized images, each captured from the same point of view, but with different polarizer angles separated by 45 degrees.

The output is a high quality separation of the reflection and background layers from the respective input images. The main technical challenge to this problem is that the mixing coefficients for the reflection and background layers are determined by the angle of incidence and the direction of the plane of incidence.

Here, the angle of incidence and the direction of the plane of incidence vary spatially over the pixels of the image. Using the physical properties of polarization for glass media with double-surfaces, an optimal method has been devised in the present invention to automatically perform reflection and separation of background layers.

Experiments demonstrate that the method devised in the present invention yields better results than the existing methods. By separating unnecessary images, you can improve the captured image to better see the desired scene. Since light reflected off the surface of the reflecting medium is polarized, a common way to reduce the effect of reflection is to place a polarizer in front of the camera lens to filter out reflected light that is polarized. Nevertheless, such a method has the disadvantage that it is effective only when it is captured at the Brewster's angle (approximately 56 ° with respect to the glass reflection) which is rarely set for the image capture.

As a result, weak reflections remain in the filtered image.

For reflection separation, the physical properties of reflection and transmission are used for double surfaced glass media that are both polarized and reflected light. The effects of reflection and transmission are derived from the physical equations of polarization, and the model of the effects of reflection and transmission can be represented by the following [Equation 1].

[Equation 1]

Where I is the intensity of light received by the image sensor and L _R Is the intensity of light from the scene that is reflected off the glass surface, and L _B Is the intensity of light from the background scene behind the glass, and α is the mixing coefficient. The value of α is determined by the refractive index of the glass, the direction of the incident surface, the incident angle, and the polarizer at the pixel position x.

Assuming that the camera response function for the image sensor is linear, I is equal to the image stored by the image sensor. Under these assumptions, our goal is to take a polarizer captured given I, L _R And L _B.

In a typical matting problem, there are a large number of foreground / background pixels with α equal to 0 or 1. Nevertheless, in the reflection separation problem α is not nearly equal to 0 or 1, but varies across 0 and 1 between images. Moreover, existing matting algorithms tend to blur the foreground / background area. Here, the focus of the matting problem is that α is between 0 and 1, as if to synthesize a natural image.

On the other hand, it is desirable in the reflection separation problem that the reflection / background layers are clear and crisp. Thus, although [Equation 1] is similar to the mating equation, existing mating algorithms do not apply to the reflection separation problem in the present invention.

Conventional image-based reflection separation methods use the minimum and maximum intensity pixels or a Sparse ICA (SPICA) technique between a series of images taken by moving the polarizer in an arbitrary direction. However, the result may still have weak reflections. Although the natural scenes follow the Laplacian distribution, a method of separating images using algorithms derived from statistical methods has been used, but in this case, the user's involvement of displaying gradients belonging to each layer is involved in the process. There was a disadvantage that can not be automatically handled.

Conventional physics-based reflection separation method takes a polarizer in several predetermined directions to photograph input images, and separates the two layers using the physics of polarization phenomenon for a dielectric medium such as glass. However, this method did not deal with the phenomenon that the angle of incidence changed locally in the image.

In the present invention, solving [Equation 1] with a single input image is not suitable, and three input images are used to automatically solve the reflection separation problem. Each image is captured with polarizer angles separated by 45 degrees, while the mixing coefficient α is allowed to vary spatially. In most existing methods, α is assumed to be constant over the image, which is sometimes not valid for the actual images. Therefore, the results obtained through these methods may not be satisfactory for the actual examples. With the method based on the reflection model in [Equation 1] proposed in the present invention, high-quality reflection separation results can be obtained for various examples. The results of the proposed method are shown to be superior in quantity to the actual examples as well as to the same examples.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and deals with double-sided reflection and spatially varying mixing coefficients of both layers in a physically based reflection model and obtains images by completely automatic physically based reflection separation. The purpose is to provide a quality improvement method.

The present invention for achieving the technical problem relates to a method for improving the image quality obtained by the reflection separation based on physical, (a) a pair of orthogonal from three polarized images each captured at a predetermined polarized angle apart Extracting images; (b) separating a reflective layer and a background layer from each of the orthogonal images by assuming a spatially varying angle of incidence; (c) purifying the reflective and background layers by applying multi-scale constrained optimization; And (d) an edge removing treatment step remaining on the reflective layer and the background layer purified through step (c). .

According to the present invention as described above, there is an effect of improving the image quality of the automatic method having a high quality by physically separating the reflected image through the reflection model.

_⊥를 각각 보여주는 일예시도.
도 5 는 인위적 예에 대한 결과들로서 본 발명에 따른 방법의 결과들과 기타 기존 방법들에 대한 결과들을 보여주는 일예시도.
도 6 은 실제 예에 대한 결과들로서 본 발명에 따른 방법의 결과들과 기타 기존 방법들에 대한 결과들을 보여주는 일예시도.
도 7 은 도 6 의 실제 예에 대해서 물리에 기반한 본 발명에 따른 방법과 여러 편광 이미지 간의 이미지 기반 정보에만 의존하는 반사분리 방법과의 비교를 나타내는 일예시도.
도 8 은 두 개의 인위적 예에 대한 강건성 테스트의 결과들을 보여주는 일예시도. 1 is an overall flowchart of a method for improving acquired image quality by physically based reflection separation according to the present invention;
Figure 2 is an exemplary view showing a comparison of the results of the method according to the invention and the previous method.
FIG. 3 shows reflectance R or transmittance T, which is represented by the sum of two orthogonal polarization components, and R and T varying depending on the incident angle, and shows that the amount of light received by the camera depends on the amount of polarization in the reflected light and transmitted light. Degree.
4 shows θ and spatially varying when the camera is close to the reflective medium.

One example showing each _각각 .
5 shows an example showing the results of the method according to the invention and the results for other existing methods as results for an artificial example.
6 shows an example of the results of the method according to the invention and the results of other existing methods as results for a practical example.
FIG. 7 illustrates an example of a comparison between a method according to the present invention based on physics and a reflection separation method that relies solely on image based information between various polarized images for the practical example of FIG. 6.
8 shows an example showing the results of the robustness test for two artificial examples.

Specific features and advantages of the present invention will become more apparent from the following detailed description based on the accompanying drawings. It is to be noted that the detailed description of known functions and constructions related to the present invention is omitted when it is determined that the gist of the present invention may be unnecessarily blurred.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the accompanying drawings.

A method of improving acquired image quality by physically based reflection separation according to the present invention will be described with reference to FIGS. 1 to 8.

First, with respect to the reflection model, the light that is reflected off or passed through to the glass surface is partially or partially polarized and perpendicular to the plane of incidence, as shown in FIG. It is expressed as the sum of these.

는 편광자 각으로 각기 정의된다.R is defined as the reflectance, which represents the relative intensity of light reflected from the glass surface, T is the transmittance, which is the relative intensity of light transmitted through the glass surface, θ is the angle of incidence,

Are each defined by the polarizer angle.

Special subscripts ⊥ and ∥ are attached to R and T, respectively, corresponding to the polarized elements perpendicular and horizontal to the plane of incidence. Thus R _⊥ and R _∥ represent two orthogonal polarized elements of R as R = R _⊥ + R _∥ . Similarly, it is defined T = T _⊥ + T _∥.

∥ 는 입사면과 편광자사이의 교차선의 방향에 대한 각도 및

_⊥ =

∥ + 90°로써 정의된다. R과 T에서의 개개의 편광된 요소는 유리 반사에 대해 1:474로서 간단하게 고정될 수 있는 반사 매체의 굴절률과 θ에 의해 결정되는 함수이다. 도 3 의 (b) 에서 보이는 바와 같이, 각 편광된 요소들의 상대적 강도는 θ내에서 완만하게 변화한다.

∥ is the angle to the direction of the intersection line between the plane of incidence and the polarizer

_⊥ =

∥ It is defined as + 90 °. The individual polarized elements at R and T are functions determined by the refractive index and θ of the reflective medium, which can simply be fixed as 1: 474 for glass reflection. As shown in FIG. 3 (b), the relative intensities of each polarized element vary slowly in θ.

In addition, none of the transmission elements disappear at any angle of incidence, while the weak horizontal element of reflection disappears completely only when the angle of incidence is equal to the Brewster's angle (about 56 ° to glass reflection).

The polarizer can completely eliminate the effects of light reflected only at the Brewster's angle, which is rarely set for image capture.

When taking an image with a polarizer, the amount of light passing through the polarizer is given by Malus's law represented by Equation 2.

[Equation 2]

는 편광자의 전달축과 입사 빛의 편광 방향 사이의 각이다. Where L is the intensity of the incoming polarized light,

Is the angle between the transmission axis of the polarizer and the polarization direction of the incident light.

The intensities of the two elements in the incident light, L _R before reflection and L _B before passing, are depolarized and thus even energy is contained within each element.

That is, L _R⊥ = L _R∥ = L _R / 2 and L _B⊥ = L _B∥ = L _B / 2.

At this time, the intensity I of light at one pixel of the image sensor after passing through the polarizer is expressed by Equation 3 below.

[Equation 3]

, θ 와

_⊥ 의 함수이다. 따라서, L_R 을 위한 혼합계수 α= [R_⊥(θ)cos²(

-

_⊥)+R_∥(θ)sin²(

-

_⊥)] 설정함으로써, [수식 1] 에서 반사 모델을 얻는다. this is

, θ and

_Is a function of _⊥ . Thus, L _R Mixing coefficient for R = (R _⊥ (θ) cos ² (

-

_⊥ ) + R _∥ (θ) sin ² (

-

_Iii )] to obtain a reflection model from [Equation 1].

Since T _⊥ (θ) = 1-R _⊥ (θ) and T _∥ (θ) = 1-R _∥ (θ), 1-α is L _R The mixing coefficient for

는 상수이고, θ와

_⊥ 는 도 4 에서 나타낸 바와 같이 이미지에 걸쳐서 공간적으로-변화할 뿐만 아니라, 마지막 두 수량의 변화들은 평면 유리 표면 혹은 작은 곡률을 갖는 유리 표면에 걸쳐서 공간적으로 완만하다는 것을 볼 수 있다.Now, for the input image captured with the polarizer in [Equation 3],

Is a constant, θ and

_In addition to being spatially-variable over the image as shown in FIG. 4, it can be seen that the last two quantities of change are spatially smooth over the flat glass surface or the glass surface with small curvature.

_⊥ 의 함수인데, 이미지에 걸쳐 공간적으로 완만하다. 도 3 의 (a) 는 반사율 R 혹은 투과율 T는 두개의 직교 편광 성분들의 합, 즉, R = R_⊥ + R_∥, T = T_⊥ + T_∥ 로서 표현된다. 도 3 의 (b) 는 R과 T는 입사각 θ에 대하여 변화하는데, 여기서 R은 브루우스터의 각(유리 반사에 대해서 56°)에서만 완전하게 편광된다. 도 3 의 (c) 는 카메라에 의해 수신된 빛의 양은 반사광과 투과광에서의 편광량에 따라 결정된다. The reflectance and transmittance are also gentle with respect to θ as shown in Fig. 3B. As a result, the mixing coefficient α is

_It is a function of _,, which is spatially smooth across the image. 3 (a), the reflectance R or transmittance T is expressed as the sum of two orthogonal polarization components, that is, R = R _⊥ + R _∥ , T = T _⊥ + T _∥ . 3 (b) shows that R and T change with respect to the incident angle θ, where R is completely polarized only at the Brewster's angle (56 ° to glass reflection). In FIG. 3C, the amount of light received by the camera is determined according to the amount of polarization in the reflected light and the transmitted light.

_⊥는 공간적으로 변화한다.
When the camera is close to the reflective medium, θ and as shown in Fig. 4 (a) and (b) respectively

_변화 changes spatially.

) 와 배경 레이어 (

) 가 [수식 20] 으로 주어진 반사 계수에 의해 선형 결합된 형태로 나타내어지고 반사 계수는 이미지 내에서 국지적으로 변화하고, 처리할 이미지의 크기에 비례하여 이미지 히스토그램 출력작업과 빌리프 프라퍼게이션(Belief Propagation)에 의해 증가하는 처리시간을 단축하기 위하여, 최저 스케일 팩터를 갖는 이미지들에 대해 빌리프 프라퍼게이션(Belief Propagation)을 수행할 수 있다. 또한 상기 (c) 단계에서, 다수의 스케일 팩터(scale factor)를 갖는 이미지들에 대해 최적화를 수행하고 입력 이미지의 가우시안 피라미드들에 기초한 멀티-스케일 반사 분리 방법을 채택하여 처리속도를 단축하게 할 수 있다. In order to obtain high quality reflection separation in the present invention, spatially varying mixing coefficients are explicitly used. The method proposed in the present invention consists of orthogonal image extraction, image separation, reflection enhancement, and weak edge suppression as shown in FIG. 1, which comprises: (a) three polarized images, each captured at a polarized angle separated by an angle; Extracting a pair of orthogonal images from the; (b) separating a reflective layer and a background layer from each of the orthogonal images by assuming a spatially varying angle of incidence; (c) purifying the reflective and background layers by applying multi-scale constrained optimization; And (d) an edge removing treatment step remaining on the reflective layer and the background layer purified through step (c). .
In the step (b), by analyzing the physical law of the polarization phenomenon, as shown in [Equation 1], the reflection mixed image (for each pixel (x)

) And background layer (

) Is represented linearly by the reflection coefficient given by Equation 20, and the reflection coefficient changes locally within the image, and the image histogram output and believe proportionality are proportional to the size of the image to be processed. In order to shorten the processing time which is increased by propagation, belief propagation may be performed on the images having the lowest scale factor. In addition, in step (c), the processing speed may be reduced by optimizing images having a plurality of scale factors and adopting a multi-scale reflection separation method based on Gaussian pyramids of the input image. have.

₁ 과

_⊥(x)의 명시적 값 대신에

₁-

_⊥(x)를 계산하고, 두 개의 직교 이미지들 I_⊥(x) 와 I_∥(x)를 계산한다. I_⊥(x)는 반사광과 투과광에서 수직 요소들에 대한 강도의 합이고, I_∥(x)의 구성요소도 이와 유사하게 정의되고 각각 아래의 [수식 4] 와 [수식 5] 와 같다.Taking into account three polarized images to extract an orthogonal image, each image was captured at a polarization angle separated by 45 degrees, ie I _i (x); i = 1, 2, 3, from these three images,

₁ lesson

_대신 instead of an explicit value of (x)

_1-

Calculating a _⊥ (x), and calculates the two orthogonal images, I _⊥ (x) and I _∥ (x). I _⊥ (x) is the sum of the intensities for the vertical elements in the reflected and transmitted light, and the components of I _∥ (x) are similarly defined and are shown in Equations 4 and 5, respectively.

[Equation 4]

[Equation 5]

₁-

_⊥(x) 에 관하여 입력 이미지 I_i(x); i = 1, 2, 3 가 표현될 수 있는데 아래의 [수식 6] 과 같다.Substituting the above equation into [Equation 3], I _⊥ (x), I _∥ (x) and

_1-

_입력 input image I _i (x) with respect to (x); i = 1, 2, 3 can be expressed as [Equation 6] below.

 [Equation 6]

_i)에 대해 (I₁,

₁), (I₂,

₁+45°) 그리고 (I₃,

₁+90°)에 대해서 각각 치환함으로써 다음의 [수식 7], [수식 8], [수식 9] 가 유도된다.Now, (I _i ,

_i ) for (I ₁ ,

₁ ), (I ₂ ,

₁ + 45 °) and (I ₃ ,

By substituting for ₁ + 90 °), the following [Formula 7], [Formula 8], and [Formula 9] are derived.

[Equation 7]

[Equation 8]

[Equation 9]

_i-

_⊥(x), I_⊥ 과 I_∥에 대한 이런 식들로부터 다음의 [수식 10], [수식 11], [수식 12] 가 도출된다.

_i -

_From these equations for _⊥ (x), I _⊥ and I _{∥ the} following [Equation 10], [Equation 11], [Equation 12] are derived.

[Equation 10]

[Equation 11]

[Equation 12]

₁-

_⊥(x)는 [-45°, 45°] 내에 있다고 가정한다. 만약

₁-

_⊥(x)이 -45°보다 작거나 혹은 45°보다 크다면, 계산된 값은 실제 값과 ±90° 만큼 다르다. 이러한 경우에, cos2(·)의 신호가 반대이기 때문에, 단순히 I_⊥ 와 I_∥ 의 값을 교환한다.
In the above formula, for the unique solution of arctan (·)

_1-

_⊥ (x) is assumed to be within the [-45 °, 45 °]. if

_1-

_If (x) is less than -45 ° or greater than 45 °, the calculated value differs from the actual value by ± 90 °. In this case, since the signal of cos2 (·) is reversed, we simply exchange the values of I _⊥ and I _∥ .

In the step of image separation, the image is taken as L _R by assuming the incident angle θ (x). And L _B , and a method for calculating the alpha matte α (x) for each input image, which is used in the next step.

Since T _⊥ (θ) = 1-R _⊥ (θ) and T _∥ (θ) = 1-R _∥ (θ), L _R from [Equation 4] and [Equation 5]. And L _B [Equation 13] and [Equation 14] below are derived.

[Equation 13]

[Equation 14]

Since θ = 0 ° or 90 ° when R _⊥ = R _∥ is in order to avoid division by zero, a range of θ (x) is limited between 5 ° and 85 °.

In Equations 13 and 14, L _R (x) and L _B (x) are functions of unknown θ (x). In order to estimate θ (x), it is assumed that there is no correlation between the image contents in the reflection scene and the background scene. Thus, the best θ (x) is obtained by finding θ (x) that minimizes mutual information between L _R and L _B.

By using Equations 13 and 14, all candidate pairs of L _R (x) and L _B (x) evaluated at sequential values of regularly sampled θ (x) are prepared.

Since θ (x) changes slowly over the image, we solve the following maximum a posteriori (MAP) problem to select the best θ (x) at every pixel across the image: Use belief propagation.

[Equation 15]

(θ(x))는 데이터 비용 함수이고,

는 네이버후드 비용 함수이고, N(x)는 x의 일차 네이버스이다.

(θ(x))는 θ(x)를 pixel x에 배정하는 비용을 측정하고,

는 θ(x)와 θ(y)를 한 쌍의 이웃하는 픽셀들 x와 y에 각각 배정하는 비용을 측정한다. here,

(θ (x)) is a data cost function,

Is the neighbor hood cost function and N (x) is the primary neighbor of x.

(θ (x)) measures the cost of assigning θ (x) to pixel x,

Measures the cost of assigning θ (x) and θ (y) to a pair of neighboring pixels x and y, respectively.

In the data cost function previously defined for mutual information, it measures the statistical dependence of image histograms between L _R (x) and L _B (x) at each angle θ (x), where θ is the Under the assumption that static and image contents in L _R and L _B are not statistically correlated, it has been shown that the image pairs (L _R ,, L _B ) at the highest θ have the least mutual information.

In the present invention, this function is adapted to the problem setting.

L _R and L _B mutual information between the two images is Kullback between the histogram of the image L _R and L _B - is derived from a labeler (KL) distance.

[Equation 16]

과

는 L_R 과 L_B 에서의 각각 어떠한 강도 레벨들 l_R과 l_B에 대한 확률들이다.

는 l_R 과 l_B 에 대한 결합확률이고, L 은 이미지 히스토그램들에서의 모든 가능한 강도 레벨들의 집합이다. here,

and

L _R Are the probabilities for any intensity levels l _R and l _B at and L _B , respectively.

L _r And l are the combined probabilities for l _B and L is the set of all possible intensity levels in the image histograms.

In order to reduce the dependence on the brightness of the image, I (L _R , L _B ) in Equation 16 is normalized by dividing it by the average magnetic information of L _R and L _B.

Since the KL distance in [Equation 16] is defined under the assumption of a static value of [theta] over the image, this cannot be applied directly to the spatially-changing problem setting. Instead, assume that θ is locally static, ie, spatially unchanged in a small patch around each pixel, based on the smoothness condition in the spatial domain.

At this time, the domain function is defined with respect to mutual information about the patch.

(θ(x)) = exp(-I_n(L_R ^[x] ,L_B ^[x])),

(θ (x)) = exp (-I _n (L _R ^[x] , L _B ^[x] )),

Here, In is mutual information about image patches L _R ^[x] and L _R ^[x] around ^x .

In addition, the data cost function was tested to select an image pair with a minimum number of corners and edges. Nevertheless, we found that mutual information generally yields better results for the relevant problem.

=exp

로써 정의되고 여기서 λ = 0.5 는 무게이다. 네이버후드 비용 함수를 사용하지 않고 각 픽셀에 대한 데이터 비용을 단순히 최대화하면, 이미지에 걸쳐 θ(x) 의 가정치는 노이즈가 섞여있고 정확하지 않을 것이다. θ(x) 를 선택한 후에, θ(x)를 가지고 [수식 13] 과 [수식 14] 를 평가함으로써 두 개의 분리된 레이어들 L_R(x) 과 L_B(x) 를 얻는다. 또한 θ(x) 를 빌리프 프라퍼게이션(belief propagation)으로부터 얻어진 값으로 세팅하고

₁-

_⊥(x) 를 이전 단계에서 얻어진 값으로 세팅함에 의하여 이것의 정의에 근거한 각 입력 이미지에 대한 α(x) 를 계산한다.
NAVER Hood cost function

= exp

Where λ = 0.5 is the weight. If you simply maximize the data cost for each pixel without using the NAVERHOD cost function, the assumption of θ (x) across the image will be noisy and inaccurate. After selecting θ (x), two separate layers L _R (x) and L _B (x) are obtained by evaluating [Equation 13] and [Equation 14] with [theta] (x). We also set θ (x) to the value obtained from the belief propagation

_1-

As set by the _⊥ (x) to the value obtained in the previous step and calculates the α (x) for each of the input images based on the definition thereof.

The separation results include artifacts that fill the gap due to the estimation of the patch of θ (x) and propose a method to further refine the separation results for their quality improvement.

α _i and (L _{R ,,} L _B ) create a problem of minimization with the objective function given in Eq.

[Equation 17]

The objective function consists of three terms, the first term being a data term derived from [Equation 1] for the input images.

와 유사하게 되도록 하는 유연한 제약인데, 이는 두 번째 단계에서 추정되는 혼합계수를 표시한다. 본 항은 각 입력 이미지에 걸쳐 α_i(x) 의 지속적인 가정을 강요한다. 세 번째 항은 α_i(x)의 변화를 최소화하는 평활도 항이다. λ_c 와 λ_s는 가중 파라미터들이다. The second term requires that α _i (x)

It is a flexible constraint to be similar to, which indicates the mixing coefficient estimated in the second step. This term imposes a continuous assumption of α _i (x) over each input image. The third term is a smoothness term that minimizes the change in α _i (x). λ _c and λ _s are weighting parameters.

, L_R 과 L_B 을, 이미지 분리 단계를 가지고 얻어진

,

,

을 가지고 각각 초기화한다. 그리고 나서 두 개의 볼록 하위문제들을 교차적으로 해결하고, 한 번의 반복에서 L_R 과 L_B를 수정하는 동안 α_i에 대해서 풀고, 다음 번 반복에서 α 를 수정하는 동안에 L_R 과 L_B 대해서 해결함으로써 목적함수를 최소화한다.
Optimization strategies evolved from traditional approaches first

, L _R and L _B , obtained with image separation steps

,

,

Initialize each with Then solve the two convex subproblems alternately, and solve L _R and L _B in one iteration. Solve for α _i during the correction and minimize the objective function by solving for L _R and L _B during the correction of α in the next iteration.

In the weak peripheral suppression stage after reflex refinement, L _R or L _B may still contain very weak false edges belonging to other layers. Cross-projection tensors are employed to suppress false edges at L _R and L _B. A true edge at L _R or L _B is assumed to have a gradient magnitude greater than the corresponding false edge at another layer.

The cross-projection tensor for L _R (x) is defined as

Equation 18

Where v ₁ and v ₂ are the eigen-vectors of each case of the smoothed structural tensor, derived from the gradient of L _B (x). These vectors are calculated after the color space for L _R and L _B is converted from RGB to YUV. The values of μ ₁ and μ ₂ to define the cross-projection tensor D _R (x) for L _R (x) are determined as follows.

In the present invention, when there is a true edge in L _R (x), it is determined to be μ ₁ = 1, μ ₂ = 1.

Μ ₁ = 0, μ ₂ = 1 when there is an edge at both L _R (x) and L _B (x), and the Y-channel gradient magnitude at L _R (x) is smaller than that at L _B (x) Decide on

When there is no edge at L _R (x), we set μ ₁ = 0 and μ ₂ = 0.

Symmetrically, a cross-projection tensor D _B (x) for L _B (x) can be defined.

To suppress false edges and maintain true edges, first project the gradients as follows:

Formula 19

과

는 통합되고, 결과들은 다시 RGB 공간으로 전환되어 최종 분리 결과들을 얻는다.
[수식 20]

R : 유리 표면으로부터 반사된 빛의 상대적 강도를 나타내는 반사율
R_⊥: R의 입사면에 수직인 편광된 요소
R_∥: R의 입사면에 수평인 편광된 요소
θ : 입사각

: 편광자 각

∥: 입사면과 편광자사이의 교차선의 방향에 대한 각도

_⊥ =

∥ + 90°At this time, the projected gradients

and

Are integrated and the results are converted back to RGB space to get the final separation results.
[Equation 20]

R: reflectance indicating the relative intensity of light reflected from the glass surface
R _⊥ : Polarized element perpendicular to the plane of incidence of R
R _∥ : polarized element horizontal to the incidence plane of R
θ: incident angle

Polarizer angle

∥ angle in the direction of the intersection line between the incident surface and the polarizer

_⊥ =

∥ + 90 °

_⊥,

₁ 를 가지고 지상 실측 정보 레이어들로부터 합성되었다. 공급물질들 내의 이러한 변수들의 지상 검증 값들을 정하는 방법에 대해 기술되고, 이 방법은 지상 검증 자료에 가까운 분리 결과들을 만들었다. 각 레이어의 제곱평균 오차값(RMSE)을 계산함으로써, 양적으로 본 특허에서 고안한 방법을 다른 방법들과 비교하는데, 이는 가정된 레이어와 이것의 지상 검증자료 사이의 차이를 수량화하는 것에 의해, 본 방법의 결과들이 기존의 다른 방법들의 결과보다 더 낮은 제곱평균 오차값들(RMSE)을 보여주었다. 시각적인 품질을 비교하면, 분리결과는 그것들의 실측 정보 레이어와 거의 유사해 보였다. 한편으로는, 기존의 방법들을 통한 결과들은 다른 레이어들로 부터의 약한 가장자리와 부자연스러운 단절들이 기존의 일부방식의 결과에서 관찰되었다. 도 5 의 (a) 는 본 발명의 결과들이며, 도 4 의 (b) 는 기존방법에 의한 결과의 한 예이며, 도 5 의 (c) 는 기존방법에 의한 결과의 한 예이며, 도 5 의 (d) 는 기존방법에 의한 결과의 한 예이며, 도 5 의 (e) 는 기존방법에 의한 결과의 한 예 (f) 입력 이미지들. (g) 지상실측정보 L_B 에서L_R(좌에서 우로)
5 shows the results of the synthesis example. [Equation 4], [Equation 5], and [Equation 6], the input images are ground measured information θ,

_⊥ ,

_It has been synthesized from ground survey information layers with ₁ . A method of setting ground verification values of these variables in feedstocks is described, which produced separation results that are close to ground verification data. By calculating the root mean square error value (RMSE) of each layer, the method proposed in this patent is quantitatively compared with other methods, by quantifying the difference between the hypothesized layer and its ground verification data. The results of the method showed lower mean square error values (RMSE) than the results of other existing methods. Comparing the visual qualities, the separation results looked almost similar to their actual information layers. On the one hand, the results from existing methods show weak edges and unnatural breaks from other layers in some of the existing methods. Figure 5 (a) is the results of the present invention, Figure 4 (b) is an example of the results by the existing method, Figure 5 (c) is an example of the results by the existing method, of Figure 5 (d) is an example of the result by the existing method, and FIG. 5 (e) is an example of the result by the existing method. (f) Input images. (g) Ground measurement information L _B to L _R (from left to right)

6 shows the results of an actual example. Similar to the synthesized examples, the present invention has consistently yielded better results. Better results were consistently produced for the starting quality compared to the results of the other methods. In Figure 6 (a) results of the present invention (b) an example of the results by the existing method (c) an example of the results by the existing method (d) an example of the results by the existing method (e) to the existing method Example of result by (f) Input images. In (a)-(e), the first column shows the background layers.

Next, the results of the image-based method are compared with the results of the physically-based method of the present invention. FIG. 7 shows this comparison using the practical example in FIG. 6. The results of the method presented in the present invention from robustness tests showed very small changes in the root mean square error (RMSE) and few visual differences.

와

의 각 변화에 대한 평균 RMSEs, (b) 도 5 의 예에 대하여 가장 큰 각 변화(+10°)를 갖는 결과, (c) [1.4,1.6]내의 범위를 갖는 반사 지수에 대한 두 개의 예들에 걸쳐 L_B와 L_R 에 대한 평균 RMSEs, (d) 도 5 에서의 예에 대한 지수 1.6을 갖는 결과들.
8 (a) shows the root mean square error values (RMSE) of each layer by changing small changes in angle. 8 (b) shows the estimated layers for small changes (+ 10 °) of the largest angle. (A) having a range of [-10 °, + 10 °] of FIG.

Wow

The mean RMSEs for each change in, (b) result in having the largest angular change (+ 10 °) for the example of FIG. 5, and (c) in two examples for the reflection index having a range within [1.4,1.6]. Across L _B and L _R Mean RMSEs for, (d) Results with index 1.6 for the example in FIG. 5.

For the same example, the present invention also conducted a sensitivity test for errors in refractive index. 8 (c) shows the average RMSE of each layer assumed by changing the refractive index from 1.4 to 1.6. Input images were generated by setting the refractive index to 1.471. 8 (d) shows the result of the refractive index having the largest error (1.6). These results support that the method presented in the present invention is robust to small changes in polarizer angle and refractive index.

In the present invention, a reflection separation method based on physical properties of polarization is proposed. A series of three polarized images are given, each producing a high quality separation of reflection and background layers in this method, captured with different polarizer angles separated by 45 degrees. In the present invention, a physically-based reflection model was derived to estimate the spatially-varying blending coefficients (ie, alpha mattes) of two layers for one input image.

On this model, an algorithm for fully automatic reflection separation is proposed.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concepts of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

Claims (8)

(a) extracting a pair of orthogonal images from three polarized images each captured at an angle of polarization separated by an angle;
(b) separating a reflective layer and a background layer from each of the orthogonal images by assuming a spatially varying angle of incidence;
(c) purifying the reflective and background layers by applying multi-scale constrained optimization; And
and (d) removing edges remaining on the reflective layer and the background layer, which have been purified through the step (c).
The method of claim 1,
The step (b)
[Formula 13] and L _B as shown in [Formula 14] as by using the incidence angle θ assumes a (x) separating the image with a reflective layer and a background layer L _R L _B, and [formula 13] and [formula 14] And L _R L _B because it is expressed as a function of the incident angle θ. And L _R Assuming that the image contents are statistically independent, patch angle is selected using belief propagation to select the incident angle θ where the mutual information is minimized and to obtain the locally changing incident angle. Expands to, but assumes that the angle of incidence remains unchanged within a small patch around every pixel, and formulates the maximum probability problem so that the calculated angle of incidence at every pixel changes in the image and solves using Belief Propagation. Method for improving the acquired image quality by physically based reflection separation.
[Equation 13]

[Equation 14]

Where L _R Is the intensity of light from the scene reflected away from the glass surface, L _B is the intensity of light from the background scene behind the glass, R _⊥ is the vertical polarization of the reflected light, R _∥ is the horizontal polarization of the reflected light element, I _∥ is the sum of the strength of the horizontal component in the reflected light and transmitted light, I _⊥ is the sum of the intensity of the vertical component in the reflected light and transmitted light, θ is the incident angle.
The method of claim 1,
The step (c)
In order to improve the quality of the results of step (b), it is necessary to minimize the objective function given by Equation 17 to purify α _i and (L _{R ,,} L _B ). How to improve the image quality obtained.
[Equation 17]

Where I _i (x) is an image function, α is a mixing coefficient, α 'is an estimated mixing coefficient, λ _c is a weighting parameter, λ _s is a weighting parameter, L _R Is the intensity of light from the scene reflected off the glass surface, and L _B is the intensity of light from the background scene behind the glass.
The method of claim 1,
The step (d)
Cross-projection to L _R (x), adopting the cross-projection tensor given by Equation 18, assuming that the true edge at L _R or L _B has a gradient magnitude greater than the corresponding false edge at another layer. - projection tensor D _R (x) may have the presence of a true edge to L _R (x) μ ₁ = 1, μ ₂ set = 1, and L _R (x) and L _B (x) both edges, L when the Y-channel gradient magnitude of the _R (x) is smaller than that in the _{L B (x) μ 1 =} 0, when set to μ ₂ = 1, and the edge is not in L _R (x) μ ₁ = 0, A method of improving acquired image quality by physically-based reflection separation, characterized by setting μ ₂ = 0 and symmetrically, cross-projection tensor D _B (x) for L _B (x) is defined.
[Equation 18]

Where v ₁ , v ₂ is the unique vector of the magnitude of the smoothed structural tensor, and μ ₁ μ ₂ is the edge relation constant
The method of claim 1,
In the step (b)
By analyzing the laws of physics about the polarization phenomenon, as shown in [Equation 1], the reflection-mixed image

) And background layer (

) Is represented in a linearly coupled form by the reflection coefficient given by [Equation 20], and the reflection coefficient changes locally in the image.
[Equation 1]

[Equation 20]

R: reflectance indicating the relative intensity of light reflected from the glass surface
R _⊥ : Polarized element perpendicular to the plane of incidence of R
R _∥ : polarized element horizontal to the incidence plane of R
θ: incident angle

Polarizer angle

∥ angle in the direction of the intersection line between the incident surface and the polarizer

_⊥ =

∥ + 90 °
The method of claim 1,
In the step (b)
Belief Propagation for images with the lowest scale factor, in order to shorten the processing time increase due to image histogram output and believ propagation in proportion to the size of the image to be processed. Method of improving the acquired image quality by the reflection separation of the physical basis, characterized in that for performing.
7. The method according to claim 1 or 6,
In the step (c)
Physically based reflection separation, characterized by optimizing the images with multiple scale factors and reducing the processing speed by adopting a multi-scale reflection separation method based on Gaussian pyramids of the input image. How to improve the image quality acquired.
delete

Images