KR101992527B1 - Method and Apparatus for Depth Image Estimation of Light Fields - Google Patents

Abstract

A light field depth image estimation method and apparatus are presented. The light field depth image estimation method proposed in the present invention includes a step of measuring a data cost using attributes of a light field image including each patch and a refocus image for estimating a data cost for a depth label candidate of a pixel of each patch Measuring the consistency of pixel colors of each patch by calculating each entropy cost to calculate the data cost in the corresponding relationship between pixels of each patch; Measuring the unfocused cost, measuring the data cost using the properties of the light field image, the data cost measuring the fitness for the depth label candidate, and the data cost for maintaining flatness between adjacent pixels, And adaptive unfocused costs to form the final data cost. And a step.

Description

FIELD OF THE INVENTION [0001] The present invention relates to a light field depth image estimation method and apparatus,

Field of the Invention [0002] The present invention relates to a method and apparatus for estimating a light field depth image, and more particularly, to a method and apparatus for estimating a light field depth that is more resistant to occlusion and less sensitive to noise.

4D light field cameras are a promising technology for acquiring images because they can get a lot of information about light in space. The light field camera acquires the brightness of each directional light beam rather than the brightness of the light ray accumulated in the pixel unlike the existing technology. The commercial light field camera is smaller than the existing light field camera and is convenient to operate, so the research in the field of light field and the consumer show a lot of interest. In the prior art, depth estimation algorithms have been developed with various features of light fields such as EPI (Epipolar Plane Image), angular patch, and image refocusing. However, existing techniques have problems with occlusion because they fail to satisfy two important assumptions: color consistency (correspondence point information) and focus agreement (non-focus information). Therefore, robust depth estimation techniques are needed in the presence of occlusion and noise.

SUMMARY OF THE INVENTION The present invention is directed to a method and apparatus for providing two new data costs robust to occlusion and noise, including correspondence and defocus information, less noise sensitivity than existing countermeasures, The cost of each angular entropy and the existing technology are improved to use the adaptive defocus cost which is strongly resistant to noise and occlusion and the cost volume filtering and the graph cut are postprocessed And to provide a method and apparatus for optimizing the data cost obtained by performing as a process.

In one aspect, the light field depth image estimating method proposed by the present invention estimates data cost for a depth label candidate of a pixel of each patch by using attributes of a light field image including each patch and a refocus image, Measuring the cost of each patch, calculating the entropy cost to calculate the data cost in the corresponding relationship between pixels of each patch, measuring the consistency of the pixel color of each patch, determining the pixel of each patch having consistency with the occlusion phenomenon Measuring the adaptive non-focal cost to obtain the final data cost, normalizing and integrating each entropy cost and the adaptive non-focal cost measured using the attributes of the light field image to form the final data cost.

The step of measuring the consistency of pixel colors of each patch by calculating each entropy cost to calculate the data cost in the correspondence relationship between pixels of each patch is used for the consistency evaluation of pixel colors of each patch by measuring each entropy cost , Each entropy cost is calculated independently for each channel of RGB without dividing the histogram into sections.

The step of measuring the adaptive non-focal cost to obtain the pixels of each patch having consistency with the occlusion phenomenon may include calculating the minimum cost by dividing the entire region affected by the occlusion into a plurality of sub-regions for the adaptive non-focal cost Thereby finding an auxiliary patch which is not affected by the occlusion of the original patch and has no blurring effect.

The step of finding the auxiliary patch without blur effect without being influenced by the occlusion of the original patch calculates the additional cost representing the difference between the average color of the auxiliary patch having the minimum cost and the center pixel color to distinguish the foreground and background of the occlusion region do.

Measuring the adaptive non-focal cost to obtain pixels of each patch that are consistent with the occlusion phenomenon includes calculating the non-focal cost for each of the auxiliary patches in the original patch to remove noise, Find the secondary patch with.

In another aspect, the light field depth image estimating apparatus proposed in the present invention uses attributes of a light field image including each patch and a refocus image to estimate a data cost for a depth label candidate of a pixel of each patch To measure data costs; Measure the consistency of pixel colors of each patch by calculating each entropy cost to calculate the data cost in the corresponding relationship between pixels of each patch; A data cost calculator for measuring the adaptive non-focal cost to obtain the pixels of each patch having consistency with the occlusion phenomenon, and the entropy cost and the adaptive non-focal cost measured using the attributes of the light field image, Thereby forming a final data cost.

The data cost calculator measures each entropy cost and uses it to evaluate the consistency of the pixel colors of each patch. Each entropy cost is independently calculated for each channel of RGB without dividing the histogram into sections.

The data cost calculator finds an auxiliary patch that is not affected by the occlusion of the original patch and has no blurring effect by calculating the minimum cost by dividing the entire area affected by the occlusion by a plurality of subregions for the adaptive unfocussed cost, In order to distinguish the foreground and the background of the occlusion region, it is necessary to calculate the additional cost indicating the difference between the average color and the center pixel color of the auxiliary patch having the minimum cost, or to calculate the supplementary patch with respect to all auxiliary patches in the original patch, , And finds a secondary patch with the lowest cost.

According to embodiments of the present invention, for two new data costs that are robust to occlusion and noise, there is a need to provide for both correspondence and defocus information, less susceptibility to noise than existing counterfeit costs, The cost of each angular entropy and the existing technology are improved to use the adaptive defocus cost which is strongly resistant to noise and occlusion and the cost volume filtering and the graph cut are postprocessed It can be performed as a process to optimize the data cost.

1 is a flowchart illustrating a light field depth image estimation method according to an embodiment of the present invention.
2 is a diagram for explaining each patch analysis according to an embodiment of the present invention.
3 is a diagram illustrating a data cost curve of each patch analysis according to an embodiment of the present invention.
4 is a diagram for explaining an unfocused cost analysis according to an embodiment of the present invention.
5 is a view illustrating a light field depth image estimating apparatus according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating a comparison result of an optimized depth map using an integrated data cost according to an exemplary embodiment of the present invention.
FIG. 7 is a diagram illustrating a depth map comparison result generated by performing global optimization on the result of FIG. 10 according to an embodiment of the present invention.

Depth estimation is a typical field of light field applications. Several algorithms have been developed using various light field properties, but the existing techniques do not work well for occlusion and handling noise-dominated data. The present invention proposes a light field depth estimation method that is more robust in occlusion and less sensitive to noise. The proposed technique proposes two new data costs, and two new data costs are measured using each angular patch and image refocusing technique. Each entropy (AE) reduces the effect of noise and occlusion on each patch at low cost. Adaptive defocus (AD) provides low cost of occlusion while maintaining robustness to noise. Through the integration of the two data costs, the light field depth estimation according to the present invention operates to be robust to occlusion and noise. As a post-processing, cost volume filtering and graph cut optimization are applied to improve the accuracy of depth maps. The proposed method has superior performance both qualitatively and quantitatively compared with the existing light field depth estimation method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a light field depth image estimation method according to an embodiment of the present invention.

To measure two new data costs robust to occlusion and noise, the present invention uses correspondence and defocus information. Each angular entropy cost is less susceptible to noise than the existing countermeasures and effectively addresses the occluded area. The adaptive defocus cost is also resistant to noise and occlusion by improving the existing technology. The data cost obtained by integrating them is optimized by performing cost volume filtering and graph cut as a post-processing process.

The proposed improved data cost is divided into two categories: angular entropy (AE) and adaptive defocus (AD). At each data cost, adjacent pixels should have a color value similar to the center pixel value. Rather than using all pixels of each patch as they are, AE weights each pixel according to the color similarity. Thus, the pixels that induce occlusion of each patch have less effect on the entropy calculation. In the present invention, the original refocusing patch is divided into auxiliary patches and the existing nonfocusing cost is measured in the auxiliary patch since the occlusion results in blurred refocused images. And adds a color similarity constraint to each auxiliary patch cost. AD is calculated as the lowest cost of the non-blur auxiliary patches.

According to an embodiment of the present invention, an extensive comparison of the proposed data cost and the existing data cost is performed for accurate performance measurement. Quantitative performance evaluation is performed using commonly used light field datasets. In addition, light field images for various experiments were acquired directly using a Lytro Illum camera. For quantitative evaluation, mean square error (MSE) and bad pixel percentage (BP) were measured using all the pixels of the image and the pixels of the occlusion region. Experimental results show that the data cost of the proposed method is superior to that of the conventional approach in occluded areas and noise images.

The present invention provides a new data cost, angular entropy (AE) cost and adaptive defocus (AE) cost for obstruction and noise-robust light field depth estimation, which allows accurate observation of light field patches and refocused images, AE) costs.

The proposed light field depth image estimation method includes a step 110 of measuring a data cost using attributes of a light field image including each patch and a refocus image for estimating a data cost for a depth label candidate of a pixel of each patch, A step 120 of measuring a data cost for measuring a fitness for a depth label candidate of a pixel of each patch and a data cost for maintaining flatness between adjacent pixels, calculating a data cost in a correspondence relationship between pixels of each patch (130) measuring coherence of the pixel color of each patch by calculating the entropy cost for each patch, measuring 140 an adaptive non-focal cost to obtain pixels of each patch that are consistent with the occlusion phenomenon, The cost of the data measured using the attribute of the field image, the cost of the data for measuring the fitness for the depth label candidate, Using the data costs for maintaining the flatness of, and by normalizing the respective entropy cost and adaptive non-focus (AD) cost and integrating a step 150 to form the final cost data.

At step 110, the data cost is measured using the attributes of the light field image including each patch, refocused image for estimating the data cost for the depth label candidates of the pixels of each patch.

의 각각의 화소는 깊이 라벨 후보

를 기반으로 변형된(sheared) 라이트필드 영상

에 다시 대응된다. 이 과정은 다음 수식과 같이 표현된다.Three attributes of the light field image (e. G., EPI, each patch, refocused image) are used to measure the data cost in depth estimation. In the present invention, each patch and refocused image are used to estimate the data cost for each depth label candidate. Light field to make each patch

Each pixel of the depth label candidate

A sheared light field image

Lt; / RTI > This process is expressed as:

는 중심 핀홀 영상의 위치를 의미한다. 또한

와

는 각각 단위 변위(disparity) 라벨

에 대한

와

축 방향의 이동 값이다.

는 변위가 없을 때의 변위 라벨

값을 나타낸다. 이동 값은 라이트필드 영상의 서브어퍼쳐(subaperture) 영상과 중심 핀홀 영상의 거리가 증가할수록 증가한다. 일반성을 잃지 않는 한에서 본 발명의 실시예에 따른 깊이 및 변위 값은 실제 깊이 값이 아닌 라이트필드 깊이 라벨 값

을 의미한다. 각패치는 아래와 같이 변형된 라이트필드로부터 각 영상의 화소를 추출하여 생성될 수 있다. here

Is the position of the center pinhole image. Also

Wow

Are each a unit disparity label

For

Wow

And is a movement value in the axial direction.

Is the displacement label when there is no displacement

Value. The shift value increases as the distance between the subaperture image of the light field image and the center pinhole image increases. As long as the generality is not lost, the depth and displacement values according to the embodiment of the present invention are not the actual depth values but the light field depth label values

. Each patch can be generated by extracting pixels of each image from a modified light field as described below.

는 깊이 라벨

에서의 화소

의 각패치를 의미한다. 재초점 영상

는 아래 정의와 같이 모든 화소의 각패치의 평균으로 계산된다..here

Depth label

Pixel

Respectively. Refocus imaging

Is calculated as the average of each patch of every pixel as defined below.

는 각패치의 화소의 수이다.here

Is the number of pixels of each patch.

In step 120, the data cost for measuring the fitness for the depth label candidates of the pixels of each patch and the data cost for maintaining flatness between adjacent pixels are measured.

The light field depth estimation according to an embodiment of the present invention is modeled in the MAP-MRF structure as follows.

Where α (P) and N (p) are the set of pixel depth labels and adjacent pixels. E _unary (p, α (p)) is the data cost that measures how well the depth label α of a given pixel p is. E _unary is the cost to maintain flatness between neighboring pixels. And λ is a weight for the two costs

In step 130, the consistency of pixel colors of each patch is measured by calculating the entropy cost to calculate the data cost in the corresponding relationship between pixels of each patch. At this time, the constrained entropy cost is measured and used for the consistency evaluation of the pixel color of each patch, and the constrained entropy cost can be independently calculated for each RGB channel without dividing the histogram into sections.

In step 140, the adaptive non-focal cost is measured to obtain pixels of each patch that are consistent with occlusion phenomena. At this time, for the adaptive non-focal cost, the total area affected by the occlusion is divided into a plurality of sub-areas and the minimum cost is calculated to find an auxiliary patch that is not affected by the occlusion of the original patch and has no blurring effect. Then, the additional cost is calculated to show the difference between the average color of the auxiliary patch having the minimum cost and the center pixel color to distinguish the foreground and the background of the occlusion region.

Alternatively, in step 140, for each auxiliary patch in the original patch to remove noise, calculate the unfocused cost in each auxiliary patch and find the auxiliary patch with the lowest cost.

The consistency of pixel colors of each patch is measured by calculating constrained entropy for the data cost C (p,? (P)) in the correspondence relationship between pixels. Next, the constrained adaptive nonfocus (CAD) D (p, α (p)) is measured to obtain robust results for occlusion phenomena.

At step 150, the final data cost is formed by normalizing and integrating each entropy cost and adaptive non-focus (AD) cost measured using the attributes of the light field image.

Each data cost is normalized and integrated to form the final data cost. The final data cost and flattening cost are defined as follows.

Where ∇I (p, q) is the brightness difference between pixels p and q, τ is the threshold of the flattening cost, and β is the weight between the two data costs. The data cost volume is flattened using an edge-preserving filter. We then perform a graph cut to optimize the energy function.

2 is a diagram for explaining each patch analysis according to an embodiment of the present invention.

)에서의 히스토그램, 도 2(d)는 각패치와 (a=48)에서의 히스토그램을 나타낸다. 첫 번째 열은 폐색이 없는 실시예이고, 두 번째 열은 다중 폐색이 존재하는 실시예이며, 참값 a는 8이다.Fig. 2 (a) shows a central pinhole image of a spatial patch, Fig. 2 (b) shows a histogram at each patch and a = 8,

2 (d) shows a histogram at each patch and (a = 48). The first row is an embodiment without occlusion, the second row is an embodiment with multiple occlusion, and the true value a is 8.

The existing correspondence data cost measures the similarity between pixels between each patch without considering the occlusion. When the occlusion area is widened to affect each patch, the color consistency of the pixels of each patch is broken. However, many pixels still maintain color consistency. Therefore, a new corresponding relationship data cost is proposed to calculate the color consistency from the probability distribution of the brightness of the main pixels.

)을 갖는 각패치는 단일 색상을 갖는다. 그리고 밝기 히스토그램은 도 2(b)와 같이 높은 정점을 갖는다. 이러한 관찰을 통해 각엔트로피 비용(AE)이라 하는 각패치의 엔트로피를 측정하고, 이를 컬러 일관성 평가에 사용한다. 라이트필드는 기존의 다중 뷰 스테레오 셋보다 훨씬 더 많은 뷰를 생성하기 때문에, 각패치는 엔트로피가 안정적으로 계산되기에 충분한 화소를 포함한다. 각엔트로피 비용 H는 아래의 수식에 의해 구해진다. Fig. 2 shows a histogram of brightness values at several depth label candidates of each patch of a given pixel. As shown in the first column without occlusion, the exact depth value (

) Has a single color. And the brightness histogram has a high peak as shown in FIG. 2 (b). Through this observation, entropy of each patch called each entropy cost (AE) is measured and used for color consistency evaluation. Since the light field generates much more views than the conventional multi-view stereo set, each patch contains enough pixels for the entropy to be calculated stably. Each entropy cost H is obtained by the following equation.

Where h (i) is the probability of the brightness i of each patch A _u ^p . Unlike the conventional method, the entropy cost in the present invention independently calculates the histogram in each of the RGB channels without dividing the histogram into sections.

Each entropy cost is also robust to occlusion, since it depends on the distribution of the brightness of the multitude of episodes. For example, the second row of FIG. 2 shows each patch in which the brightness histogram still shows a high peak at true value alpha although the occlusion region is present. The proposed data cost shows the best response even though there are multiple occlusions in each patch. As long as the unobstructed pixels are dominant in each patch, the entropy cost is low.

3 is a diagram illustrating a data cost curve of each patch analysis according to an embodiment of the present invention.

The data cost curves of the patches when there is occlusion and when there is no occlusion are shown in Figs. 3 (a) and 3 (b), respectively.

4 is a diagram for explaining an unfocused cost analysis according to an embodiment of the present invention.

Fig. 4 (a) shows the center pinhole image and the spatial patch, Fig. 4 (b) shows the data cost curve comparison and Figs. 4 (c) 4 (a) and 4 (c) to 4 (f) show the difference map between the spatial patch of FIG. 4 (a) and the spatial patch of FIG. 4 Patch, and the true value a is 27.

The existing non-focal cost for light field depth estimation is robust to noisy scenes but does not work well in the presence of occlusions. This is because blurred refocused images increase the ambiguity of existing data costs. 4 (a), 4 (c) to 4 (f) show a center pinhole image and a refocus image, respectively. 4 (g) to 4 (j) show a difference map between the central image and the refocused image for a clear observation, the difference between the true value depth (a = 27) There are errors that show little cost.

To solve this problem, we proposed adaptive defocus (AD) cost that is robust to both noise and occlusion. Based on the analysis of FIG. 4, the adaptive unfocused cost finds the minimum cost in the subregion in which the entire region is divided. In other words, the original patch (15 × 15) was divided into 9 auxiliary patches (5 × 5) without measuring the cost in the entire area (15 × 15) affected by the occlusion, so that the blur Look for secondary patches that have no effect. At this time, the measurement of each auxiliary patch N _c (p) ratio _c ^res focus cost D (p, α) of independently as follows.

c is the index of the auxiliary patch and p is the center pinhole image. The initial unfocus cost (ID) is calculated as the cost in the auxiliary patch that minimizes the cost of the equation.

을 이용한다. 공식은 다음과 같다. However, the initial cost still has an ambiguity that can not distinguish the foreground and background of the occlusion region as shown in Fig. 4 (b). In order to distinguish between the two cases, the additional cost, which represents the difference between the average color of the auxiliary patch with the least cost and the center pixel color p (p)

. The formula is as follows.

AD is derived as follows.

는 두 비용 항의 상대적인 영향을 조절하는 상수이다. 도 4(b)에 제시한 바와 같이, 제안된 적응적 비초점 비용(AD)은 비초점 비용(CD)에 비해 참값에서 최소 비용을 가진다.
here

Is a constant that regulates the relative effect of the two cost terms. As shown in FIG. 4 (b), the proposed adaptive non-focal cost (AD) has a minimum cost in the true value compared to the non-focal cost (CD).

5 is a view illustrating a light field depth image estimating apparatus according to an embodiment of the present invention.

The proposed light field depth image estimating apparatus 500 includes a data cost calculating unit 510 and a final data cost integrating unit 520.

The data cost calculation unit 510 measures the data cost using the attributes of the light field image including each patch and the refocused image to estimate the data cost for the depth label candidates of the pixels of each patch. Also, by calculating the constrained entropy cost to calculate the data cost in the correspondence relationship between pixels of each patch, it is possible to measure the consistency of pixel colors of each patch and to obtain pixels of each patch having consistency with the occlusion phenomenon The constrained adaptive non-focal cost is measured.

The final data cost integrating unit 520 forms the final data cost by normalizing and integrating each entropy cost and the adaptive non-focal cost measured using the attributes of the light field image.

The data cost calculation unit 510 measures the data cost using the attributes of the light field image including each patch and the refocused image to estimate the data cost for the depth label candidates of the pixels of each patch.

의 각각의 화소는 깊이 라벨 후보

를 기반으로 변형된(sheared) 라이트필드 영상

에 다시 대응된다. 이 과정은 앞서 설명된 것과 같이 수학식(1) 내지 수학식(3)으로 표현된다. Three attributes of the light field image (e. G., EPI, each patch, refocused image) are used to measure the data cost in depth estimation. In the present invention, each patch and refocused image are used to estimate the data cost for each depth label candidate. Light field to make each patch

Each pixel of the depth label candidate

A sheared light field image

Lt; / RTI > This process is expressed by Equations (1) to (3) as described above.

을 의미한다. 각패치는 수학식(4)와 같이 변형된 라이트필드로부터 각 영상의 화소를 추출하여 생성될 수 있다. The shift value increases as the distance between the subaperture image of the light field image and the center pinhole image increases. As long as the generality is not lost, the depth and displacement values according to the embodiment of the present invention are not the actual depth values but the light field depth label values

. Each patch can be generated by extracting pixels of each image from a modified light field as shown in Equation (4).

는 수학식(5)와 같이 모든 화소의 각패치의 평균으로 계산된다..Refocus imaging

Is calculated as an average of each patch of all the pixels as shown in equation (5).

Then, the data cost calculation unit 510 measures the data cost for measuring the fitness for the depth label candidate of the pixel of each patch and the data cost for maintaining the flatness among the adjacent pixels.

The light field depth estimation according to the embodiment of the present invention is modeled in the MAP-MRF structure as in Equation (6).

Where α (p) and N (p) are the set of depth labels and adjacent pixels of pixel p. E _binary (p, α (p)) is the data cost that measures how well the depth label α of a given pixel p is. E _binary is the cost to maintain flatness between neighboring pixels. And λ is a weight for the two costs

The data cost calculation unit 510 measures the consistency of pixel colors of each patch by calculating the entropy cost to calculate the data cost in the corresponding relationship between pixels of each patch. At this time, each constrained entropy cost is measured and used for the consistency evaluation of pixel colors of each patch, and each constrained entropy cost is independently calculated for each RGB channel without dividing the histogram into sections.

Then, the adaptive non-focal cost is measured to obtain pixels of each patch that are consistent with the occlusion phenomenon. At this time, for the adaptive non-focal cost, the total area affected by the occlusion is divided into a plurality of sub-areas and the minimum cost is calculated to find an auxiliary patch that is not affected by the occlusion of the original patch and has no blurring effect. Then, the additional cost is calculated to show the difference between the average color of the auxiliary patch having the minimum cost and the center pixel color to distinguish the foreground and the background of the occlusion region.

Alternatively, the data cost calculator 510 may calculate the non-focal cost for each auxiliary patch, and find the auxiliary patch with the lowest cost, for all auxiliary patches in the original patch to remove noise.

을 위해서 제약된 각엔트로피를 계산함으로써 각패치의 화소 색상의 일관성을 측정한다. 다음으로 폐색 현상에 대해 강인한 결과를 얻기 위해 적응적 비초점(AD)

를 측정한다. The data cost in the pixel-to-pixel correspondence relationship

, The consistency of the pixel colors of each patch is measured by calculating each constrained entropy. Next, adaptive non-focus (AD)

.

The final data cost integrating unit 520 forms the final data cost by normalizing and integrating each entropy cost and the adaptive non-focal cost measured using the attributes of the light field image.

Each data cost is normalized and integrated to form the final data cost. The final data cost and flattening cost are defined by equations (7) and (8).

The data cost volume is flattened using an edge-preserving filter. We then perform a graph cut to optimize the energy function.

FIG. 6 is a diagram illustrating a comparison result of an optimized depth map using an integrated data cost according to an exemplary embodiment of the present invention.

6D is a V-LO, FIG. 6B is a SSD B-PRD, FIG. 6C is a SSD P-GRAD, FIG. 6D is a OV- OPRD, AD, the left frame of each drawing shows a light field with no noise, and the right frame shows a light field with noise of dispersion 0.10.

The proposed algorithm is implemented in Intel i7 4770 @ 3.4 GHz with 16 GB RAM. The results of the proposed data cost are compared with those of the current light field depth estimation data. In the present invention, we compared the individual data costs of 16 (10 corresponding to 6 non-focal costs) and 6 integrated data costs.

Conventional depth estimation methods have difficulty in making a fair comparison because various optimization methods are applied. In the present invention, depth estimation results are compared without global optimization to check the characteristics of each data cost. And globally optimized depths are compared using various test data. At this time, a combination of three global optimization methods, a graph cut (GC), an edge preservation filter (EPF), and an edge preservation filter and a graph cut (EPF-GC) are used. The quantitative evaluation uses the minimum square error (MSE) and error pixel ratio (BP) defined as follows.

와

는 화소

에서의 참값 깊이와 계산된 깊이이다.

는 깊이 오류를 판단하기 위한 임계값이다. 폐색지역에서의 강인함을 평가하기 위해, 폐색 지도에

와

를 측정한다 (각각

와

). 폐색 지도 O는 참값 깊이 지도에서 급격한 변화를 edge 주변 영역을 추출하여 만든다. here

Wow

The pixel

And the calculated depth.

Is a threshold value for judging a depth error. To assess robustness in the occluded area,

Wow

(Each

Wow

). The occlusion map O makes a sudden change in the true value depth map by extracting the area around the edge.

,

,

,

,

로 설정하였으며, 비용단면(cost slice) 필터링을 위해

과

로 설정했다. 모든 데이터 셋에서의 깊이 조사 범위는 1, 2, 3, . . . , 74, 75이다.Composite datasets were generated using existing 4D light field datasets. The actual light field image was acquired using a retro-room camera. The parameters used in the experiment are

,

,

,

,

And for cost slice filtering.

and

Respectively. The depth survey ranges for all data sets are 1, 2, 3,. . . , 74, and 75, respectively.

For qualitative evaluation, a depth map with an integrated data cost optimized for a noisy or noisy Mona data set is shown in FIG. The proposed data cost shows the best performance results for all datasets and noise levels.

7D, 7D and 7E show V-LO, SSD B-PRD, SSD P-GRAD, OV-OPRD and AE- AD. Figure 7 shows a globally optimized depth map for each integrated data cost. It can be seen that the proposed method preserves the edges of thin objects better than other methods.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device As shown in FIG. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (7)

Measuring a data cost using attributes of a light field image including each patch and a refocus image for estimating a data cost for a depth label candidate of a pixel of each patch;
Measuring a data cost for measuring a fitness for a depth label candidate of a pixel of each patch and a data cost for maintaining flatness between adjacent pixels;
Measuring coherence of pixel colors of each patch by calculating each entropy (AE) cost to calculate a data cost in a corresponding relationship between pixels of each patch;
Measuring an adaptive non-focussing (AD) cost to obtain pixels of each patch that are consistent with the occlusion phenomenon; And
Forming the final data cost by normalizing and integrating each entropy cost and the adaptive non-focal cost measured using the attributes of the light field image
Lt; / RTI >
The step of normalizing and integrating each entropy cost and the adaptive non-focal cost measured using the attributes of the light field image to form a final data cost,
The entropy cost and the adaptive non-focal cost are normalized by using the data cost measured using the attributes of the light field image, the data cost measuring the fitness for the depth label candidate, and the data cost for maintaining flatness between adjacent pixels To form the final data cost,
Measuring the consistency of pixel colors of each patch by calculating each entropy cost to calculate a data cost in a corresponding relationship between pixels of each patch,
In order to calculate color consistency from the probability distributions of the brightness of a plurality of patches, each entropy cost is measured by observing the brightness histogram of each patch having an accurate depth value and a single color, and is used for color consistency evaluation, Cost does not divide the histogram into sections but calculates independently for each channel of RGB
Light field depth image estimation method. delete The method according to claim 1,
Measuring the adaptive non-focal cost to obtain a pixel of each patch that is consistent with the occlusion phenomenon,
For the adaptive unfocussed cost, finding the auxiliary patch without blur effect without being affected by the occlusion of the original patch by calculating the minimum cost by dividing the whole area affected by the occlusion into a plurality of subregions
/ RTI > wherein the light field depth image estimating method comprises: The method of claim 3,
The step of finding an auxiliary patch that is not affected by occlusion of the original patch and has no blur effect,
To further distinguish the foreground and background of the occlusion region, an additional cost is calculated that represents the difference between the average color of the auxiliary patch with the least cost and the center pixel color
Light field depth image estimation method. Measuring a data cost using attributes of a light field image including each patch, a refocused image for estimating a data cost for a depth label candidate of a pixel of each patch; Measuring a data cost for measuring a fitness for a depth label candidate of a pixel of each patch and a data cost for maintaining flatness between adjacent pixels; Measure the consistency of the pixel colors of each patch by calculating the entropy (AE) cost to calculate the data cost in the pixel-to-pixel correspondence of each patch; A data cost calculator for measuring an adaptive non-focus (AD) cost to obtain pixels of each patch that are consistent with the occlusion phenomenon; And
A final data cost integrator that forms a final data cost by normalizing and integrating each entropy cost and adaptive non-focal cost measured using the attributes of the image,
Lt; / RTI >
The final data cost consolidator,
The entropy cost and the adaptive non-focal cost are normalized by using the data cost measured using the attributes of the light field image, the data cost measuring the fitness for the depth label candidate, and the data cost for maintaining flatness between adjacent pixels To form the final data cost,
The data cost calculator,
In order to calculate color consistency from the probability distributions of the brightness of a plurality of patches, each entropy cost is measured by observing the brightness histogram of each patch having an accurate depth value and a single color, and is used for color consistency evaluation, Cost does not divide the histogram into sections but calculates independently for each channel of RGB
Light field depth image estimator. delete 6. The method of claim 5,
The data cost calculator,
For the adaptive unfocussed cost, we calculate the minimum cost by dividing the whole area affected by the occlusion into a plurality of sub - regions and find the auxiliary patches without blur effect without being affected by the occlusion of the original patches.
To further distinguish the foreground and background of the occlusion region, an additional cost is calculated that represents the difference between the average color of the auxiliary patch with the least cost and the center pixel color
Light field depth image estimator.

Images