KR100962329B1 - Road area detection method and system from a stereo camera image and the recording media storing the program performing the said method - Google Patents

Abstract

PURPOSE: A road area extracting method from a stereo camera image, a device thereof and a recording medium which records a program realizing the method are provided to extract a road area more exactly by removing an obstacle area in a perpendicular disparity image and extracting the road area with Lagrangian support vector regression which a weighted value is applied. CONSTITUTION: A perpendicular disparity image generator(200) generates a perpendicular disparity image with comparison of right and left images of a stereo camera(100). A probablistic disparity filtering unit(300) removes an obstacle area from the perpendicular disparity image. A road area estimator(400) acquires a road function about a road area from the perpendicular disparity image. The road area estimator presumes the road function with WLSVR(Weighted Lagrangian Support Vector Regression). A disparity map generator(500) generates a disparity map.

Description

Road area detection method and system from a stereo camera image and the recording media storing the program performing the said method

The present invention relates to a method and apparatus for extracting a ground from a stereo camera image, and a recording medium on which a program for implementing the method is recorded. More particularly, the present invention relates to an acquired vertical disparity image (v-disparity image). Method and apparatus for extracting the ground region from the camera image that removes the obstacle region and extracts the ground region using weighted Lagrangian Support Vector Regression (WLSVR) from the disparity vertical disparity image And a recording medium on which a program for implementing such a method is recorded.

Adaptive Cruise Control (ACC) and collision safety technologies are the most active research areas in ITS to provide efficient driver assistance and protection.

In a typical ACC scenario, the safety distance between cars is automatically maintained by a system that minimizes the difference between the desired speed and the actual speed. At dangerous moments, for example, the warning and braking aids are activated or the seat belts are tightened more firmly as a function of the distance between the two operators.

Such driving environment analysis systems typically calculate the location of obstacles relative to the vehicle by extracting road surfaces, finding obstacles, and using range information. Typically, active sensing systems such as laser radars or millimeter wave radars have shown good performance in ACC systems.

The laser radar measures the distance between two cars by illuminating the front part of the car with an infrared laser beam, receiving reflected light and measuring the time delay. Such laser radar systems are not only very sensitive to bad weather such as rain, snow and snow, but will also be affected by lasers fired by other vehicles, despite providing very accurate measurements at relatively low cost. . And millimeter-wave radars measure distances by detecting reflected waves. Since millimeter wave radars provide sufficiently good accuracy despite bad weather conditions, millimeter wave radars have recently been widely applied in ACC devices and crash safety devices.

But such radar sensors have some fatal weaknesses. Radars generally have very narrow viewing angles (less than 15 degrees) and relatively low accuracy in the plane direction.

This means that the radar system can hardly see vehicles cutting in or passing from neighboring lanes unless the vehicle is very ahead.

In addition, since there is no information about road shape information in the radar system, it cannot distinguish between roadside obstacles and roadside obstacles, especially if the obstacles are located on an incoming curve.

Stereo vision sensors can compensate for all these drawbacks of radar systems. Stereo analysis is the process of measuring the range of an object by comparing the projection of the object to two or more images.

Until recently, real-time stereo analysis could only be satisfied in bulky and conventional hardware arrangements. However, recent advances in computational power and algorithms have made it possible for multipurpose personal computers to perform real-time stereo processing.

As such, in the field of ITS, many approaches have been aimed at analyzing the driving environment based on stereovision technology. Compared to the radar system, the stereovision sensor has the advantages of wide viewing angle, good surface accuracy and relatively low cost.

The passive nature of the stereo vision sensor suggests that it is relatively free of signal interference present in other sensors.

In addition, it can provide image density and depth information together. However, like other optical sensors, stereovision sensors are sensitive to bad weather and lighting conditions.

It is also problematic because of the high computational cost for disparity map calculations, namely the analysis of 3D image reconstruction and its application which significantly limited when solving real world problems.

Another prior art to remedy this problem is to classify 3D road images into relative surfaces, based on road image analysis algorithms using a UV-disparity concept that characterizes road side structures, obstacles, and asphalt road surfaces. Provides a high speed stereo vision solution.

To this end, automotive engineering engineers face the following challenges in analyzing 3D driving environments:

1) Detect road surfaces and lane areas, 2) Recognize obstacles on the road to measure the size of obstacles and 3D models 3) Measure the relative distance between cars and obstacles 4) Not obstacles, but can be confusing Recognize roadside structures such as guardrails, streetlights, and bridges passing over them

Optical flow forms the basis of vision based on 3D image analysis and obstacle detection. Most optical flow algorithms only calculate the movement between the camera and the environment. This may not be true in most driving situations.

Stereovision is also widely used in image analysis. Stereo analysis is a processor that measures the range for an object by comparing two or more projected objects.

In order to properly extract obstacles from the image, it is necessary to separate those belonging to the obstacle candidate from the image features belonging to the road surface.

Most algorithms will use a clustering method that is applied successively to synthesize distracting disparity pixels into distinct obstacle candidates by using a method of suppressing disparity continuity.

However, because of asymmetrical road surfaces and car stops, such algorithms are not sufficient to separate obstacle candidates from road surfaces. In addition, clustering that separates dispersive disparity pixels into distinct obstacle candidates is not only limited in accuracy, but also very time consuming.

Some obstacles are simply eliminated because the number of detected disparity pixels is very small or because of limited dimensions in depth.

Labayrade proposed the concept of a vertical disparity image, aimed at simplifying the processor that separates obstacles from the road surface.

In the v-disparity image, the hardness profile for the road surface can be shown by a distinct linear curve, and obstacles in the vertical plane will be projected in the vertical line. So extraction of 3D road surfaces and obstacles will be simplified like 2D linear extraction.

The vertical disparity image is mainly used for surface extraction perpendicular to the vertical plane for stereo equipment such as road surfaces and car fronts.

Meanwhile, the conventional technique of extracting the ground using the vertical disparity image (v-disparity image) also includes the obstacle region or the ground region in the vertical disparity image (v-disparity image) including both the obstacle region and the ground region. In this case, since the obstacle area acts as a noise to the ground area and the ground area acts as a noise to the obstacle area, there is a problem in the safety such as the accuracy of the extracted obstacle area or the ground area decreases.

In addition, there is a problem in that accuracy is lowered by using a function estimating method that does not give a weight to a point including a disparity value in a vertical disparity image.

In order to solve the above problem, the present invention removes an obstacle region by using a stochastic method from the obtained vertical disparity image, and extracts a ground region from the vertical disparity image from which the obstacle is removed. By using weighted Lagrangian Support Vector Regression (WLSVR), we can extract ground area more stably and accurately from the vertical disparity image containing a lot of noise. It is an object of the present invention to provide a recording medium on which a method and apparatus for extracting a ground from a stereo camera image and a program for implementing the method are recorded.

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

According to an exemplary embodiment of the present invention, a method of extracting ground from a stereo camera image is probabilistically determining whether each point including a disparity value in an acquired vertical disparity image is an obstacle region or a ground region. A probabilistic disparity filtering step of calculating a method to remove an obstacle area from a vertical disparity image; And a ground estimation step of obtaining a ground function for the ground area from the vertical disparity image from which the obstacle area is removed, and estimating the ground area from the ground function.

 Here, the ground function can be defined by the following equation.

[Equation]

Here, d is a disparity value of the vertical disparity image, v is a value of the vertical coordinate of the vertical disparity image (v-disparity image), w is a weight value for the ground area, and is unknown. Is the bias value of the function, which is unknown.

Here, in order to estimate the ground area from the ground function, the ground function may use the following equation as the objective function.

[Equation]

Limitation condition;

은 데이터 샘플이 부과되지 않은 무감지 지역에 대한 값,

,

는

,

으로 여유(slack) 변수 벡터, S는

으로 수직 디스패러티 이미지(v-disparity image)에서 각 좌표의 점들에 대한 가중치가 부여된 대각행렬, c는 w^2와 b^2을 최소화시키기 위한 매개 변수, T는 트랜스포머(transformer)이다.Where e is a vector filled with 1, the parameter

Is the value for the undetected region where no data sample was charged,

,

Is

,

Slack variable vector, S

As a weighted diagonal matrix for points of each coordinate in a vertical disparity image, c is a parameter for minimizing w ^ 2 and b ^ 2, and T is a transformer.

Here, the function for the ground area may be obtained by recursively calculating the following equation.

[Equation]

는

으로 2N 변수와 kernel 함수 K로 이루어진 라그랑지안 상관계수 벡터,

,

, I는 임의의 차수를 가지는 단위 행렬이다.here,

Is

Lagrangian correlation coefficient vector consisting of 2N variables and kernel function K,

,

, I is an identity matrix of arbitrary order.

Here, the stochastic disparity filtering step includes a disparity value by using a disparity value of each point in a vertical disparity image and a weighted standard deviation value of the vertical coordinate value in the disparity value. A probability value calculating step of calculating a probability value whether each point is an obstacle region or a ground region; And filtering each point estimated as an obstacle from a vertical disparity image by using the calculated probability value.

The method of extracting the ground from the stereo camera image may further include generating a vertical disparity image by comparing the left and right images obtained by using the stereo camera.

In addition, the ground extraction method from the stereo camera image is a disparity map in which the obstacle region and the ground region are separated from the vertical disparity image (v-disparity image) before removing the obstacle region by using a function of the obtained ground region. Generating a; may further include.

The computer-readable recording medium according to the present invention also includes a recording medium on which a program for implementing the method is recorded.

In addition, the apparatus for extracting the ground from the stereo camera image calculates whether each point including the disparity value in the acquired vertical disparity image is an obstacle region or a ground region by a stochastic method. A probabilistic disparity filtering unit that removes an obstacle region from a vertical disparity image; And a ground estimator for obtaining a ground function for the ground area from the vertical disparity image from which the obstacle area is removed and estimating the ground area from the ground function.

Here, the ground function can be defined by the following equation.

[Equation]

Here, d is a disparity value of the vertical disparity image, v is a value of the vertical coordinate of the vertical disparity image (v-disparity image), w is a weight value for the ground area, and is unknown. Is the bias value of the function, which is unknown.

Here, the ground function may use the following equation as an objective function to obtain a weight value for the ground area and a bias value of the function from the ground function.

[Equation]

Limitation condition;

은 데이터 샘플이 부과되지 않은 무감지 지역에 대한 값,

,

는

,

으로 여유(slack) 변수 벡터, S는

으로 수직 디스패러티 이미지(v-disparity image)에서 각 좌표의 점들에 대한 가중치가 부여된 대각행렬, c는 w^2와 b^2을 최소화시키기 위한 매개 변수, T는 트랜스포머(transformer)이다.Where e is a vector filled with 1, the parameter

Is the value for the undetected region where no data sample was charged,

,

Is

,

Slack variable vector, S

As a weighted diagonal matrix for points of each coordinate in a vertical disparity image, c is a parameter for minimizing w ^ 2 and b ^ 2, and T is a transformer.

Here, the function for the ground area may be obtained by recursively calculating the following equation.

[Equation]

는

으로 2N 변수와 kernel 함수 K로 이루어진 라그랑지안 상관계수 벡터,

,

, I는 임의의 차수를 가지는 단위행렬이다.here,

Is

Lagrangian correlation coefficient vector consisting of 2N variables and kernel function K,

,

, I is a unit matrix of arbitrary order.

Here, the probabilistic disparity filtering unit includes a disparity value by using the disparity value of each point in the vertical disparity image and the weight-based standard deviation value of the vertical coordinate value in the disparity value. A probability value calculator for calculating a probability value whether each point is an obstacle region or a ground region; And a filtering unit which removes each point estimated as an obstacle from the vertical disparity image using the calculated probability value.

The apparatus for extracting a ground from a stereo camera image may include a vertical disparity image generation unit configured to generate a vertical disparity image by comparing a left and right image obtained using a stereo camera; It may further include.

In addition, the ground extraction apparatus from the stereo camera image is a disparity map in which the obstacle region and the ground region are separated from the vertical disparity image (v-disparity image) before removing the obstacle region by using a function of the obtained ground region. It may further include a disparity map generator for generating a.

Method and apparatus for extracting the ground from a stereo camera image according to the present invention and a recording medium on which a program for implementing such a method are recorded are recorded from the vertical disparity image with the obstacle area removed from the vertical disparity image containing a lot of noise. By using weighted Lagrangian Support Vector Regression (WLSVR) to extract the ground area, the ground area can be extracted more stably and accurately.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

First, in adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible, even if shown on different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the following will describe a preferred embodiment of the present invention, but the technical idea of the present invention is not limited thereto and may be variously modified and modified by those skilled in the art.

1A and 1B are diagrams for explaining an example of an apparatus and method for extracting ground from a stereo camera image according to the present invention.

As shown in FIG. 1A, the apparatus for extracting a ground from a stereo camera image includes a stereo camera 100, a v-disparity image generator 200, and a probabilistic disparity filter 320 (Probablistic Disparity). Filtering 300, a road plain estimator 400, and a disparity map generator 500.

The stereo camera 100 generates two left and right images by photographing the front of the camera while two cameras are arranged in parallel with each other.

The vertical disparity image generation unit 200 generates the vertical disparity image by comparing the left and right images obtained using the stereo camera 100.

As such, a method for generating a vertical disparity image using a stereo camera is described in Zhencheng Hu, Francisco Lamosa and Keiichi Uchimura. "A complete UV-disparity study for stereovision based 3D driving environment analysis" 3-D Digital Imaging and Modeling, 2005. 3DIM 2005. Fifth International Conference on, Volume, Issue, 13-16 June 2005 Page (s): 204-211 And papers by Stephen J. Krotosky and Mohan M. Trivedi. "A Comparison of Color and Infrared Stereo Approaches to Pedestrian Detection" Intelligent Vehicles Symposium, 2007 IEEE Volume, Issue, 13-15 June 2007 Page (s): 81-86.

The vertical disparity image generated as described above is illustrated in FIG. 2.

A vertical disparity image as shown in FIG. 2 is a histogram in which the number of pixels for a given disparity value is accumulated for each image row. Where v is the vertical coordinate value of the vertical disparity image (v-disparity image) and corresponds to the vertical coordinate value of the image captured by the stereo camera 100, and d is the disparity of the vertical disparity image (v-disparity image). Value.

As a result, the vertical disparity image (v-disparity image) represents the density of the same disparity value for each image row. In more detail, in FIG. 2, a bright point is a point where a large number of pixels include the same disparity value, and a dark point is a point where there are few pixels including the same disparity value. to be.

Therefore, in the vertical disparity image as shown in FIG. 2, the ground area 210 has a larger value of the disparity value d as the distance between the stereo camera 100 and the captured image is closer, and as the distance is longer. It has a small disparity value d. In addition, since the obstacle area 220 such as a car or a traffic light is disposed perpendicular to the ground, the obstacle area 220 has a disparity value approximately equal to the height of the obstacle in the captured image.

The probabilistic disparity filtering unit 300 calculates whether each point including the disparity value in the acquired vertical disparity image is an obstacle area or a ground area by using a probabilistic method. And removing an obstacle area from the vertical disparity image. The probabilistic disparity filtering unit 320 may include a probability value calculating unit 310 and a filtering unit 320 as shown in FIG. 1B.

Here, the probability value calculating unit 310 is a weighted standard deviation value of the vertical coordinate value (V) at the same disparity value (d) as the disparity value (d) of the vertical disparity image (v-disparity image) Using the function to calculate the probability value whether each point including the disparity value is an obstacle area or a ground area.

Here, the probability value calculator 310 will be described in more detail as follows. In FIG. 2 and the vertical disparity image (v-disparity image), since the obstacles generally exist in the vertical direction from the ground, the weight-based standard deviation value of the vertical coordinate value of the same disparity value is high.

However, since the disparity values are distributed in a vertically narrow region in the vertical disparity image, only the ground-based region has a smaller weight-based standard deviation value of the vertical coordinate value at the same disparity image.

As such, the probability value for the obstacle area in the vertical disparity image (v-disparity image) as shown in FIG. 2 may be expressed as Equation 1 below.

는 수직 디스패러티 이미지(v-disparity image)의 동일한 디스패러티 값(d)에서 수직 좌표 값(v)이 가지는 가중치 기반의 표준 편차 값이다. 이와 같은 수학식 1은 장애물이 자동차인 경우를 일례로 한 것이다.Where p is a probability probability function and d is a disparity value of a vertical disparity image, sigma.

Is a weighted standard deviation value of the vertical coordinate value v in the same disparity value d of the vertical disparity image. Equation 1 is an example in which the obstacle is a car.

In addition, the probability value for the ground area in the vertical disparity image may be expressed as Equation 2 below.

Since it is impossible to obtain Equations 1 and 2 as they are, it can be expressed as Equation 3 by applying a Bayes rule using conditional probabilities.

In the case of Equation 3, the right term is impossible to obtain, but the left term can be estimated using actual data.

In addition, since it is difficult to estimate prior probability and normalization constraints, each disparity of the vertical disparity image (MLE) using the Maximum Likelihood Estimation (MLE) method is used. The probability value is calculated whether the value is an obstacle area or a ground area. To this end, it may be expressed by Equation 4 as to whether the area having the disparity value and the standard deviation value of each vertical coordinate is an obstacle area or a ground area.

In order to solve the above Equation 4, there is a problem of finding a likelihood probability whether a specific disparity value and a standard deviation value are an obstacle region or a ground region. In order to obtain such probability probability, it is assumed that each probability is composed of a Gaussian Mixture model such as Equation (5).

By solving Equation 4 using Equation 5, it is possible to estimate whether each point including the disparity value in the vertical disparity image is an obstacle region or a ground region.

Next, the filtering unit 320 functions to remove each point estimated as an obstacle from the vertical disparity image using the calculated probability value. As such, when the obstacle region is removed, the vertical disparity image (v-disparity image) as shown in FIG. 2 is converted into the shape as shown in FIG. 3.

Then, the ground plain estimator 400 obtains a ground function for the ground area from the vertical disparity image from which the obstacle area is removed, and adds a weighted Lagrangian support to obtain the ground function. It estimates the ground function using Weighted Lagrangian Support Vector Regression (WLSVR).

Here, the ground function is defined by the following equation (6).

Here, d is a disparity value of the vertical disparity image, v is a value of the vertical coordinate of the vertical disparity image (v-disparity image), w is a weight value for the ground area, and is unknown. Is the bias value of the function, which is unknown.

Here, w and b values may be obtained through a weighted Lagrangian Support Vector Regression (WLSVR) algorithm with weights added.

Here, the weighted Lagrangian Support Vector Regression (WLSVR) is a function estimation method having the following Equation 7 as the objective function, which is obtained for each point of the vertical disparity image (v-disparity image). Applicable when disparity values are different.

와

인 가중치를 가지고 있다. 그리고, 수학식 7과 같은 목적함수의 표현을 간결하게 하기 위하여, 벡터

,

로 정의한다.Here, the apparatus and method for extracting the ground from a stereo camera image according to the present invention includes N pieces of training data.

Wow

Has a weight. And, in order to simplify the expression of the objective function as shown in equation (7), the vector

,

.

minimize

minimize

subject to;

은 데이터 샘플이 부과되지 않은 무감지 지역에 대한 값,

,

는

,

으로 여유 변수 벡터(slack variable vector), S는

으로 수직 디스패러티 이미지(v-disparity image)에서 디스패러티 값을 포함하는 각 지점에 대한 가중치가 부여된 대각행렬, c는 w^2와 b^2을 최소화시키기 위한 매개 변수, T는 트랜스포머(transformer)이다.Where e is a vector filled with 1, the parameter

Is the value for the undetected region where no data sample was charged,

,

Is

,

Slack variable vector, S

Weighted diagonal matrix for each point containing the disparity value in the vertical disparity image, c is a parameter for minimizing w ^ 2 and b ^ 2, and T is a transformer. )to be.

과

의 제곱을 최소화시키는 문제이기 때문이다.Here, the reason for using c as a parameter for minimizing w ^ 2 and b ^ 2 is as follows. c is used to optimize two or more constraints simultaneously if they have more than one constraint. In this case, the weights of w and b are minimized while

and

This is because it minimizes the square of.

Here, the points considered in order to introduce the objective function as shown in Equation 7 will be briefly described as follows.

,

는 오류이기 때문에 그 값은 최소화되어야 하고, 그 값을 최소화시키는 선이 지면 영역을 알려주는 선이 된다. 따라서 이를 반영하기 위해

이 도입된 것이다. 또한, 이를 반영하여 지면 영역에 대한 파라미터 w와 b를 구하기 위하여 w^2 와 b^2를 반영하고, w^2 와 b^2의 영향을 조절하기 위하여 매개변수 c를 사용하는 것이다.First of all, since the number of points with v and d values in the vertical disparity image are all different, a weight S for each point is introduced to take this into account.

,

Since is an error, the value should be minimized, and the line that minimizes the value becomes a line indicating the ground area. So to reflect this

This is introduced. Also, reflect w ^ 2 and b ^ 2 to obtain the parameters w and b for the ground area, and use the parameter c to adjust the influence of w ^ 2 and b ^ 2.

,

는 도 4와 같이 표현될 수 있다. Where a slack variable vector

,

May be expressed as shown in FIG. 4.

By transforming the equation (7) to create a dual problem (dual problem), and to find a Lagrangian can be expressed by the following equation (8).

는 수학식 7에서

,

모두 0이상이라는 조건이 포함되어 있기 때문에 이를 수식에 반영하기 위하여 도입한 라그랑지안 상관 계수이다.Where is a Lagrangian correlation coefficient vector composed of 2N variables and kernel function K,

In equation (7)

,

Since all of the conditions include zero or more, the Lagrangian correlation coefficient is introduced to reflect this in the equation.

,

에 대하여 편미분 하면 다음의 수학식 9와 같은 결과를 얻을 수 있다.Equation 8 is converted into each variable w, b,

,

By partial derivative with respect to Equation 9, the following results can be obtained.

Here, by substituting Equation 9 into Equation 8, a Lagrangian function can be obtained as shown in Equation 10 below.

,

을 도입한다. 여기서, I는 단위행렬을 의미한다.To make this more concise, the following two matrices

,

Introduce. Here, I means unit matrix.

A new objective function such as Equation 11 can be found from Equation 10 arranged as described above.

minimize

minimize

subject to;

이다.here,

to be.

값을 구하고, 그 결과값을 이용하면 원 지면 함수의 파라미터 w와 b는 다음과 같은 수학식 12와 같이 표현된다.In equation (11)

When the value is obtained and the result value is used, the parameters w and b of the surface function are expressed by Equation 12 as follows.

By using Equations 11 and 12, w and b in Equation 5 can be obtained.

As described above, when the ground function of Equation 5 is obtained, the ground area on the vertical disparity image may be represented by a red line w. Here, b is a bias, d is a disparity value, and v is a vertical coordinate value of a vertical disparity image.

Thereafter, the disparity map generator 500 uses the function of the obtained ground area to distinguish the obstacle area from the ground area from the vertical disparity image (v-disparity image) before removing the obstacle area. It is used to create a disparity map.

The disparity map generated as described above may be represented as shown in FIG. 6. In FIG. 6, the red area 510 indicates a ground area, the green area 520 indicates an obstacle area, and the blue area 530 indicates an area that is neither an obstacle area nor a ground area.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer operating the program using a computer-readable recording medium. The computer-readable recording medium may be a magnetic storage medium (for example, a ROM, a floppy disk, a hard disk, a magnetic tape, etc.), an optical reading medium (for example, a CD-ROM, a DVD, an optical data storage device, etc.). And storage media such as carrier waves (eg, transmission over the Internet).

The above description is merely illustrative of the technical idea of the present invention, and various modifications, changes, and substitutions may be made by those skilled in the art without departing from the essential characteristics of the present invention. will be. Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical spirit of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by the embodiments and the accompanying drawings. . The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

As described above, a method and apparatus for extracting a ground from a stereo camera image according to the present invention and a recording medium on which a program for implementing the method are recorded add weight to extract a ground area from a vertical disparity image. By using the weighted Lagrangian Support Vector Regression (WLSVR), it is possible to extract the ground area more stably and accurately from the vertical disparity image (v-disparity image) containing a lot of noise. It can be used in various fields where it is necessary to find an object.

In particular, the present invention is likely to be used as an essential system for stable driving in the field of intelligent vehicle systems.

In addition, in the case of various intelligent robots, it is necessary to determine the area with the ground and the area with the object when driving indoors or on the road.

1A is a view for explaining an example of a ground extraction apparatus from a stereo camera image according to the present invention.

1B illustrates a stochastic disparity filtering unit in an example of an apparatus for extracting a ground from a stereo camera image according to the present invention.

2 is a view for explaining an example of the vertical disparity image generated by the vertical disparity image generating unit according to the present invention.

3 is a view for explaining an example of a vertical disparity image from which an obstacle region is removed by a probabilistic disparity filtering unit according to the present invention;

4 illustrates an example of a slack variable vector and FIG.

5 is a view showing an example of the vertical disparity image in which the ground area extracted by the ground estimating unit according to the present invention is displayed.

6 is a diagram illustrating an example of a disparity map in which an obstacle and a ground generated by the disparity map generating unit according to the present invention are distinguished.

<Description of reference numerals for the main parts of the drawings>

100: stereo camera 200: vertical disparity image generator

300: stochastic disparity filtering unit 400: ground estimation unit

500: disparity map generator

Claims (18)

In the acquired vertical disparity image, the vertical disparity image is calculated by probabilistically calculating whether each point including a disparity value is an obstacle region or a ground region. A stochastic disparity filtering step of removing the obstacle area from And A ground estimating step of obtaining a ground function for the ground area from a vertical disparity image from which the obstacle area is removed, and estimating the ground area from the ground function; Ground extraction method from a stereo camera image comprising a. The method of claim 1, The ground function is a ground extraction method from a stereo camera image, characterized in that defined by the following equation. [Equation]

Here, d is a disparity value of the vertical disparity image, v is a value of the vertical coordinate of the vertical disparity image (v-disparity image), w is a weight value for the ground area, and is unknown. Is the bias value of the function, which is unknown. The method of claim 2, And the ground function uses the following equation as an objective function to obtain a weight value for the ground area and a bias value of the function from the ground function. [Equation]

Limitation condition;

Where e is a vector filled with 1, the parameter

Is the value for the undetected region where no data sample was charged,

,

Is

,

Slack variable vector, S

In the vertical disparity image (v-disparity image), a weighted diagonal matrix for points of each coordinate, c is a parameter to minimize w ^ 2 and b ^ 2, and T is a transformer. The method of claim 3, wherein And a function for the ground area is calculated by recursively calculating the following equation. [Equation]

here,

Is

Lagrangian correlation coefficient vector consisting of 2N variables and kernel function K,

,

, I is an identity matrix of arbitrary order. The method of claim 1, Ground extraction method from the stereo camera image Generating a vertical disparity image by comparing the left and right images obtained by using a stereo camera; The method of extracting ground from a stereo camera image further comprising. The method of claim 1, The probabilistic disparity filtering step Each point including the disparity value is an obstacle using a disparity value of each point in the vertical disparity image and a weighted standard deviation value of the vertical coordinate value of the disparity value. A probability value calculating step of calculating a probability value whether the area or the ground area; And Filtering each point estimated as an obstacle from a vertical disparity image using the calculated probability value; Ground extraction method from a stereo camera image, characterized in that it comprises a. The method of claim 1, Ground extraction method from the stereo camera image Generating a disparity map in which an obstacle area and a ground area are divided from a vertical disparity image (v-disparity image) before removing the obstacle area by using the obtained function of the ground area; The method of extracting ground from a stereo camera image further comprising. Generating a vertical disparity image by comparing the left and right images obtained by using a stereo camera; In the generated vertical disparity image, the vertical disparity image (v-disparity) is calculated by probabilistically calculating whether each point including a disparity value is an obstacle region or a ground region. probabilistic disparity filtering to remove the obstacle area from the image; A ground estimating step of obtaining a ground function for the ground area from a vertical disparity image from which the obstacle area is removed, and estimating the ground area from the ground function; And Generating a disparity map in which an obstacle area and a ground area are divided from a vertical disparity image (v-disparity image) before removing the obstacle area by using the obtained function of the ground area; Ground extraction method from a stereo camera image comprising a. The method of claim 8, The ground function is a ground extraction method from a stereo camera image, characterized in that defined by the following equation. [Equation]

Here, d is a disparity value of the vertical disparity image, v is a value of the vertical coordinate of the vertical disparity image (v-disparity image), w is a weight value for the ground area, and is unknown. Is the bias value of the function, which is unknown. The method of claim 9, And the ground function uses the following equation as an objective function to obtain a weight value for the ground area and a bias value of the function from the ground function. [Equation]

Limitation condition;

Where e is a vector filled with 1, the parameter

Is the value for the undetected region where no data sample was charged,

,

Is

,

Slack variable vector, S

As a weighted diagonal matrix for points of each coordinate in a vertical disparity image, c is a parameter for minimizing w ^ 2 and b ^ 2, and T is a transformer. In a computer-readable recording medium, A recording medium on which a program for implementing the method according to any one of claims 1 to 7 is recorded. In the acquired vertical disparity image, the vertical disparity image is calculated by probabilistically calculating whether each point including a disparity value is an obstacle region or a ground region. A probabilistic disparity filtering unit for removing an obstacle region from And A ground estimator for obtaining a ground function for the ground area from a vertical disparity image from which the obstacle area is removed, and estimating the ground area from the ground function; Ground extraction apparatus from a stereo camera image comprising a. 13. The method of claim 12, The ground function is a ground extraction apparatus from a stereo camera image, characterized in that defined by the following equation. [Equation]

Here, d is a disparity value of the vertical disparity image, v is a value of the vertical coordinate of the vertical disparity image (v-disparity image), w is a weight value for the ground area, and is unknown. Is the bias value of the function, which is unknown. The method of claim 13, And the ground function uses the following equation as an objective function to obtain a weight value for the ground area and a bias value of the function from the ground function. [Equation]

Limitation condition;

Where e is a vector filled with 1, the parameter

Is the value for the undetected region where no data sample was charged,

,

Is

,

Slack variable vector, S

As a weighted diagonal matrix for points of each coordinate in a vertical disparity image, c is a parameter for minimizing w ^ 2 and b ^ 2, and T is a transformer. The method of claim 14, And a function for the ground area is obtained by recursively calculating the following equation. [Equation]

here,

Is

Lagrangian correlation coefficient vector consisting of 2N variables and kernel function K,

,

, I is an identity matrix of arbitrary order. 13. The method of claim 12, The ground extraction apparatus from the stereo camera image A vertical disparity image generator configured to generate the vertical disparity image by comparing the left and right images obtained using a stereo camera; Apparatus for extracting the ground from the stereo camera image further comprising. 13. The method of claim 12, The probabilistic disparity filtering unit Each point including the disparity value is an obstacle using a disparity value of each point in the vertical disparity image and a weighted standard deviation value of the vertical coordinate value of the disparity value. A probability value calculator for calculating a probability value that is an area or a ground area; And A filtering unit for removing points estimated as obstacles from a vertical disparity image using the calculated probability values; An apparatus for extracting ground from a stereo camera image, comprising: a. 13. The method of claim 12, The ground extraction apparatus from the stereo camera image A disparity map generator configured to generate a disparity map in which an obstacle region and a ground region are divided from a vertical disparity image (v-disparity image) before removing the obstacle region by using a function of the obtained ground region; Apparatus for extracting the ground from the stereo camera image further comprising.

Images