KR101088144B1 - Method for Measurement of Distance between Object and Camera using Stereo Camera - Google Patents

Abstract

The present disclosure relates to a method for measuring the distance of an object using a stereo camera that more accurately measures the distance between the camera and the object by using optimized disparity, and corresponds to the corresponding first and second images of the stereo camera. Estimating an initial disparity based on a first difference value between the first and second pixels; the first difference value, a second difference value of an nth order disparity between the first pixel and an adjacent pixel, and the first difference A weight for the second difference value that is set according to a first ratio between the probability that the value is '0' and '1' and the second ratio between the probability that the second difference value is '0' and '1', And estimating an n + th order disparity based on a determination value for determining whether to exclude the second difference value according to a third difference value between the first pixel and the adjacent pixel. Based on parity Calculating a distance between the camera and the object, wherein n is a natural number, where the first disparity when n = 1 is the initial disparity, and then n is sequentially increased to a predetermined order. The final disparity of the n + th order can be estimated.

Stereo Camera, Disparity, Distance Measurement, Disparity Space Image, Dynamic Programming

Description

Method for Measurement of Distance between Object and Camera using Stereo Camera}

The present disclosure relates to a method for measuring a distance of an object using a stereo camera, and more particularly, to measure a distance between an object and a camera using an optimized disparity more accurately. It is about.

In general, objects appearing in images taken using a stereo camera have various shapes such as size, direction, shape, color, and brightness. These deformation factors make it difficult to apply fixed algorithms and cause performance degradation.

An embodiment of the present invention is to solve the above conventional problems, the object of which is to obtain a disparity optimized by a method according to a fixed algorithm and the distance between the camera and the object using the optimized disparity To more accurately measure the distance of the object using a stereo camera to provide a method.

In order to achieve the above object, a method for measuring a distance of an object using a stereo camera according to an aspect of the present invention includes a first difference value between corresponding first and second pixels in corresponding first and second images of the stereo camera. Estimating an initial disparity based on the first difference value, the second difference value of the nth order disparity between the first pixel and the adjacent pixel, and the probability that the first difference value is '0' and '1'. A weight for the second difference value that is set based on a first ratio between the probability of 'to be equal to and the second ratio between the probability that the second difference to be' 0 'and the probability to become' 1 ', and the first Estimating an n + th order disparity based on a determination value for determining whether to exclude the second difference value according to a third difference value between a pixel and the adjacent pixel, and based on the n + first order disparity Calculate the distance between the camera and the object And n may be a natural number, and when n = 1, the first disparity is the initial disparity, and then n is sequentially increased to a predetermined order so that the n + first disparity is increased. Can be estimated. In addition, before the estimating of the initial disparity, the color adjustment of the first and second images may be performed.

를 최소화 하는 디스패리티이고, 여기서 U(d_i)는 상기 제 1 차이값이다. The initial disparity is determined in a disparity space image (DSI) generated according to the first difference value.

Is a disparity that minimizes, where U (d _i ) is the first difference value.

를 최소화하는 디스패리티이고, 여기서 U(d_i)는 상기 제 1 차이값, V(d_i, d_j)는 상기 제 2 차이값, λ는 상기 가중치, l_ij는 상기 결정값을 나타낸다.The n + 1st disparity is

U (d _i ) is the first difference value, V (d _i , d _j ) is the second difference value, λ is the weight, and l _ij is the determination value.

의 수식에 따라산출되고, 여기서

이고,

이며, P(e(di)=1/di)과 P(e(di)=0/di)는 각각 상기 제 1 차이값이 '1'이 될 확률과 '0'이 될 확률, P(Δd_ji=1/s)와 P(Δd_ji=0/s)는 각각 상기 제 2 차이값이 '1'이 될 확률과 '0'이 될 확률을 나타낸다.The weight λ is

Is calculated according to the formula

ego,

P (e (di) = 1 / di) and P (e (di) = 0 / di) are the probability that the first difference value is '1' and the probability that '0' is P (Δd), respectively. _ji = 1 / s) and P (Δd _ji = 0 / s) represent the probability that the second difference value is '1' and the probability that it becomes '0', respectively.

The determination value l _ij may be set to '1' or '0' depending on whether the third difference value is greater than a predetermined value.

의 수식에 의해 산출되고, 여기서 Z는 상기 거리, d는 상기 제 n+1차 디스패리티, f는 해당 카메라의 초점거리, B는 상기 스테레오 카메라의 두 카메라간의 거리를 나타낸다.The distance is

Where Z is the distance, d is the n + th order disparity, f is the focal length of the camera, and B is the distance between two cameras of the stereo camera.

As described above, according to an aspect of the present invention, an optimized disparity may be obtained by a method according to a fixed algorithm, and the distance between the camera and an object may be more accurately calculated using the optimized disparity. That effect occurs.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. First of all, in adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are used as much as possible even though they are shown in different drawings. In addition, in describing the present invention, when it is determined that the detailed description of the related well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

In addition, in describing the component of this invention, terms, such as 1st, 2nd, A, B, (a), (b), can be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. If a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected to or connected to that other component, but there may be another configuration between each component. It is to be understood that the elements may be "connected", "coupled" or "connected".

1 is a flowchart illustrating a method of measuring an object distance using a stereo camera according to an exemplary embodiment of the present invention.

First, the colors of the two images of the left and right images are adjusted by using the average color and the variance obtained from the corresponding left and right images of the stereo camera (S101).

Subsequently, when the user designates a first region including an object to be measured in the left image, a corresponding second region in the right image is simultaneously designated with the first region, and the first and the first images of the designated left and right images are simultaneously designated. Two regions (or referred to as images) are extracted and used as an input image for distance measurement (S103).

Subsequently, a disparity space image (DSI) having a difference of pixel values for each row is obtained for the extracted first and second images, and then a dynamic programming (DP) technique is performed for the corresponding DSI. The initial disparity (or first disparity) value is estimated by tracing the least cost path (S105). That is, the initial disparity is estimated based on the first difference value between the first and second pixels corresponding to each row in the first and second images, and the initial disparity is determined in the DSI generated according to the first difference value. By applying the DP technique, it can be estimated as the disparity d _i to minimize the following equation (1). In Equation (1), I means a set of pixels constituting the image. In Equation (1), U (d _i ) represents the first difference value, and is expressed as Equation (2) below, and I (x, y) and J (x, yd) in Equation (2). _i ) represents the values of the first and second pixels, respectively.

--- 수학식(1)

--- Equation (1)

=

--- 수학식(2)

=

--- Equation (2)

Subsequently, the DSI is obtained by using the estimated first disparity as well as the difference value of the left and right images (ie, the first difference value), and the minimum cost path is traced by applying the DP technique to the corresponding DSI. Estimate the disparity value. As described above, the DSI is obtained using the first difference value and the n th order disparity, and the DP method is applied to the DSI to track the disparity of the least cost path in which Equation (3) is minimized. A first primary disparity value is estimated (S107).

--- 수학식(3)

--- Equation (3)

In Equation (3), N means a set of pixels adjacent to the i-th pixel. According to Equation (3), the first difference value U (d _i ), the second difference value V (d _i , d _j ) of the nth order disparity between the first pixel and the adjacent pixel, and the first The first ratio between the probability that the difference value U (d _i ) becomes '0' and the probability that it becomes '1' and the second ratio between the probability that the second difference value becomes '0' and the probability that it becomes '1'. the second difference value that is set based on V (d _i, d _j) weights λ, and in accordance with a third difference value between the adjacent pixels with the first pixel and the second difference value V (d _i, d _j for ) Is estimated based on a decision value l _ij that determines whether to exclude?), Where n is a natural number and the first disparity when n = 1 is the initial disparity. n is sequentially increased to a predetermined order to estimate the n + th order disparity, and the n + 1st order disparity is set as the final disparity. As a result, the n + first order disparity is a disparity that minimizes Equation (3).

이고 상기

로 정의한다. 즉,

는 인근 화소

와

의 값이 유사하면 동일 영역에 속해 있다고 가정할 수 있으므로 1에 근사한 값을 갖도록하여 두 화소에 대한 디스패리티

와

가 동일한 값을 갖도록 유도하고, 위 두 화소의 값의 차가 기 설정된 값 이상으로 크면 다른 영역에속해 있을 확률이 높으므로 상호 영향을 주지 못하도록

이 되도록 조정한다. 마지막으로, 상기 가중치

는 다음과 같이 구한다. 대응되는 좌우 영상 두화소의 차인

(또는 U(d_i)라 표시함)와 인근 디스패리티의 차

(또는 V(d_i, d_j)라 표시함)를 각각

,

으로 정의하면 이들의 확률분포는 실험적으로 exponential 분포를 갖는다. 따라서, 다음의 수학식 (4) 및 수학식 (5)에 근거하여

,

를 구하고이를 이용하여 상기 가중치

를 구할 수 있다.In Equation (3), the second difference value

And said

. In other words,

Is a nearby pixel

Wow

If the values of are similar, it can be assumed that they belong to the same area.

Wow

Is derived to have the same value, and if the difference between the values of the two pixels is larger than the preset value, there is a high probability that they belong to different areas, so that they do not affect each other.

Adjust so that Finally, the weight

Is obtained as follows. Difference between the corresponding left and right video two pixels

(Or _denoted U (d _i )) and the nearby disparity

(Or denotes V (d _i , d _j ))

,

If they are defined as, their probability distribution has experimentally exponential distribution. Therefore, based on the following equations (4) and (5)

,

Find and use the weights

Can be obtained.

--- 수학식 (4)

--- Equation (4)

--- 수학식 (5)

--- Equation (5)

In Equation (4), P (e (di) = 1 / di) and P (e (di) = 0 / di) are respectively the probability that the first difference value U (d _i ) becomes' 1 'and' 0 (?), Where P (sd _ji = 1 / s) and P (∇d _ji = 0 / s) are respectively represented by the second difference value V (d _i , d _j ). Represents the probability of becoming '1' and the probability of becoming '0'. In addition, the determination value l _ij may be set to '1' if the third difference is less than a preset value, and to '0' if it is greater than or equal to.

를 구한다.Finally, given the map according to the final disparity, the final disparity of the object is obtained by combining the histogram of the map and the assumption that the desired object is located in the center of the extracted image. 6) the distance of the object according to

.

--- 수학식 (6)

--- Equation (6)

는 관측물체의 최종 디스패리티이고,

와

카메라의 초점거리 및 두 카메라 간의 거리를 각각 나타낸다.In equation (6)

Is the final disparity of the object,

Wow

The focal length of the camera and the distance between the two cameras are respectively shown.

For reference, the derivation process of the equation applied in the present embodiment will be described.

Stereo matching is modeled as two MRFs, the disparity map D corresponding to the coordinates of the reference image and the discontinuity point S of the disparity. Given the stereo images I and J according to the Bayesian law, the conditional probabilities for D and S can be expressed as

라고 가정할 수 있으므로

를 최대화하는

를 구함으로써 원하는디스패리티 맵을 구할 수 있다. 그런데,

이고

와

는 각각 다음의 식(8)(9)와 같이 가정한다.In this equation (7),

So we can assume

To maximize

By obtaining the desired disparity map can be obtained. By the way,

ego

Wow

Are respectively assumed as in the following equations (8) and (9).

는 두 화소

와

간의 차이를 나타내고,

는 두 화소

의 디스패리티

와

의 차이를 나타낸다. 따라서

는 다음의 식(10)과 같이 정리할 수 있다.here

Two pixels

Wow

Represents the difference between

Two pixels

Disparity of

Wow

Indicates a difference. therefore

Can be summarized as Equation (10) below.

는 세그멘테이션 정보

에 무관하므로

로 표현할 수 있다. By the way

Segmentation information

Is irrelevant to

.

와

는 다음과 같이 스테레오 정합에서 가장 보편적으로 사용되는 exponential 분포로 가정하였다.In this embodiment

Wow

Is assumed to be the most commonly used exponential distribution in stereo matching.

As a result, the problem of obtaining disparity results in a problem of solving the following equation (13).

를 취하면 다음의 식(14)와 같이 나타낼 수 있다.In order to reduce the calculation amount in the root of this equation (13),

It can be expressed as the following equation (14).

,

이므로식 (14)는 다음과 같이 나타낼수 있다.By the way,

,

Equation (14) can be expressed as

(14)

(14)

와

는 상수이어서 이를 제거해도 결과에는 영향을 주지 않는다. 또한, 양변을

로 나누면 식 (14)는 다음의 식(15)와 같이 간단히 정리할 수 있다. here,

Wow

Is a constant, so removing it does not affect the result. Also, both sides

By dividing by, Equation (14) can be simply summarized as in Equation (15) below.

이다. 위 식을 최소화하는

를 찾아냄으로써 디스패리티를 구할 수 있다. 이 식을 식 (3)과 비교하면

이고

에 각각 해당하는 메트릭이다. 본 실시예에서는 구현의편의를 위하여

와

를 각각

,

으로 정의하였으며, 전 화소에 대한 디스패리티

를 추론하였다.here,

to be. To minimize the above expression

We can find the disparity by finding Comparing this expression with equation (3)

ego

Each metric corresponds to In this embodiment, for convenience of implementation

Wow

Each

,

Disparity for all pixels

Deduced.

In the above description, all elements constituting the embodiments of the present invention are described as being combined or operating in combination, but the present invention is not necessarily limited to the embodiments. In other words, within the scope of the present invention, all of the components may be selectively operated in combination with one or more. In addition, although all of the components may be implemented in one independent hardware, a program in which some or all of each component is selectively combined to perform some or all functions combined in one or a plurality of hardware. It may be implemented as a computer program having a module. Codes and code segments constituting the computer program may be easily inferred by those skilled in the art. Such a computer program may be stored in a computer readable storage medium and read and executed by a computer, thereby implementing embodiments of the present invention. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like.

In addition, the terms "comprise", "comprise" or "having" described above mean that the corresponding component may be included, unless otherwise stated, and thus excludes other components. It should be construed that it may further include other components instead. All terms, including technical and scientific terms, have the same meanings as commonly understood by one of ordinary skill in the art unless otherwise defined. Terms commonly used, such as terms defined in a dictionary, should be interpreted to coincide with the contextual meaning of the related art, and shall not be construed in an ideal or excessively formal sense unless explicitly defined in the present invention.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention but to describe the present invention, and the scope of the technical spirit of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

As described above, in the method of measuring the distance of an object using a stereo camera according to an embodiment of the present invention, the recent research to measure the distance to a camera with respect to a specific object in a stereo image is increasing rapidly, the application field also continues Increasing trends, for example, in fixed algorithms, such as in applications where you want to take a stereo camera of an object on the sea floor or in a heavily polluted area, and then measure the distance of the object remotely. The optimized disparity can be obtained, and the optimized disparity can be used to more accurately calculate the distance between the camera and the object.

1 is a flowchart illustrating a method of measuring an object distance using a stereo camera according to an exemplary embodiment of the present invention.

<Description of Symbols for Main Parts of Drawings>

S101: Stereo Image Color Adjustment Steps

S103: Input image excerpt for distance measurement

S105: Initial disparity estimation step

S107: estimating final disparity

S109: distance calculation step

Claims (7)

In the method of measuring the distance of an object using a stereo camera, Estimating initial disparity based on a first difference value between corresponding first and second pixels in corresponding first and second images of the stereo camera The first difference value, the second difference value of the nth order disparity between the first pixel and the adjacent pixel, a first ratio between a probability that the first difference value is '0' and a probability that it becomes '1' and the A weight for the second difference value that is set according to a second ratio between a probability that the second difference value becomes '0' and a probability that it becomes '1', and a third difference value between the first pixel and the adjacent pixel. Estimating an n + 1st order disparity based on a determination value for determining whether to exclude the second difference value; Calculating a distance between the camera and the object based on the n + 1st disparity; Wherein n is a natural number and the initial disparity is a first disparity when n = 1, and thereafter, n is sequentially increased to a predetermined order to estimate the n + th first disparity. How to measure the distance of an object using a camera. The method of claim 1, The initial disparity is determined in a disparity space image (DSI) generated according to the first difference value.

Wherein I is a disparity that minimizes the set of pixels constituting the image, and U (d _i ) is the first difference value. The method of claim 1, The n + 1st disparity is

Where I is a set of pixels constituting an image, N is a set of pixels adjacent to an i-th pixel, and U (d _i ) is the first difference value, V (d _i , d _j ) is the second difference value, λ is the weight, l _ij represents the determination value, the distance measuring method of the object using a stereo camera. The method of claim 3, wherein The weight λ is

Calculated according to the formula of,

ego,

P (e (di) = 1 / di) and P (e (di) = 0 / di) are the probability that the first difference value is '1' and the probability that '0' is P, d _ji = 1 / s) and P (∇d _ji = 0 / s) respectively indicate the probability that the second difference is '1' and the probability of becoming '0'. Distance measurement method. The method of claim 3, wherein The determination value l _ij is set to '1' if the third difference is less than a preset value, and to '0' if the third difference is greater than or equal to a predetermined value. The method of claim 1, The distance is

Wherein the distance Z is the distance, d is the n + th order disparity, f is the focal length of the camera, and B is the distance between the two cameras of the stereo camera. Distance measurement method of the used object. The method of claim 1, And adjusting the colors of the first and second images before the estimating the initial disparity.

Images