KR20170079373A - Apparatus for object detection on the road and method thereof - Google Patents

Abstract

The present invention relates to an apparatus and method for detecting an object on a road.
An apparatus for detecting an object on a road according to the present invention includes a depth image generator for generating a depth image of the road from a left and right image of a road taken from a stereo camera; Estimating the road information by using the depth image of the generated road, and estimating the road information by reflecting previously estimated road information when it is determined that an error occurred during the road information estimation; An interest region extractor for extracting a region of interest to be searched using the estimated road information; And an object detecting unit for searching for the extracted ROI and detecting an object existing on the road.
As described above, according to the present invention, when an error occurs in the current image while estimating the road information for detecting an object on the road, the posterior probability is calculated using the road information estimated from the previous image and the road information estimated from the current image The object detection speed and the detection performance are improved as the search area is reduced by searching for a specific object such as a pedestrian or a vehicle in the ROI accurately extracted from the road information estimated to be high.

Description

[0001] APPARATUS FOR OBJECT DETECTION ON THE ROAD AND METHOD THEREOF [0002]

The present invention relates to an apparatus and method for detecting an object on a road, and more particularly, to an apparatus and method for detecting an object on a road, in which, when a loss occurs in the road information estimated in the current image, And a method for detecting an object on the road.

Conventionally, a method using a single camera has been mainly used for detecting an object on a road. FIG. 1 is a diagram for explaining an object detection process using a conventional single camera. In order to detect an object using a single camera, as shown in FIG. 1, a method of detecting an object by searching all regions in the image using a multi-scale sliding window technique is used. Since this method has a large number of search regions, it requires a long calculation time and increases the undetectable rate.

In order to solve this problem, in recent years, many researches have been conducted on a method of detecting an object on the road using a stereo camera in the fields of intelligent automobiles and robots.

The object detection method using a stereo camera has an advantage of extracting a depth image, estimating road information, and searching for an object only in a region of interest on the road, thereby drastically reducing the search area. As a result, the object detection speed and the detection performance can be improved at the same time.

In order to provide excellent performance while maintaining the fast searching speed, it is most important to accurately extract the region of interest by robustly estimating the road information in various external environments. However, in general, a depth image calculated from a stereo image causes a stereo image matching error due to a difference in brightness between left and right pixels, a difference in viewpoint between left and right cameras, and a shielding. An estimation error occurs.

FIG. 2 is a view showing a stereo matching error due to shielding in an object detecting process using a conventional stereo camera, and FIG. 3 is an example of a road information loss caused by a front obstacle in the object detecting process using a conventional stereo camera FIG.

That is, as the stereo matching error occurs due to the partial shielding as shown in FIG. 2, an estimation error may also occur when estimating the road information. In addition, as shown in FIG. 3, when there are a large number of obstacle objects in front, it is difficult to accurately estimate the road information from the depth image because the road information is lost. ROI: region of interest), which may cause a problem of increasing errors of object detection as a whole.

The technology of the background of the present invention is disclosed in Korean Patent Registration No. 10-0959246 (published on May 20, 2010).

According to an aspect of the present invention, there is provided a method for estimating road information for detecting an object on a road, the method comprising the steps of: And an object and a method for detecting an object on the road that detects a specific object such as a pedestrian or a vehicle by searching in a region of interest accurately extracted from the road information estimated to be the highest.

According to an aspect of the present invention, there is provided an apparatus for detecting an object on a road, the apparatus comprising: a depth image generator for generating a depth image of the road from a left and right image of a road taken by a stereo camera; Estimating the road information by using the depth image of the generated road, and estimating the road information by reflecting previously estimated road information when it is determined that an error occurred during the road information estimation; An interest region extractor for extracting a region of interest to be searched using the estimated road information; And an object detecting unit for searching for the extracted ROI and detecting an object existing on the road.

The road information estimating unit generates a v-disparity map by accumulating in the y-axis using the generated depth image, and calculates a v-disparity map by using a Hough transformation algorithm for a portion corresponding to the gradient in the generated v-disparity map And estimates the road information.

The road information estimating unit estimates the road information by reflecting the road information estimated by the depth image of the road generated before the depth image of the generated road, when it is determined that an error has occurred during the road information estimation, The road information may be updated by assigning weights according to the degree of similarity.

The ROI extractor may extract the ROI using the stixel technique that expresses the estimated road information and the image in a group of a predetermined width.

Wherein the object detection unit searches the ROI to determine whether or not an object exists on the road, and when it is determined that the object exists, The type of object can be detected.

According to another aspect of the present invention, there is provided a method of detecting an object on a road using an object detection apparatus, the method comprising: generating a depth image of the road from a left and right image of a road taken by a stereo camera; Estimating the road information by using the depth image of the road; estimating the road information by reflecting previously estimated road information when an error occurs during the road information estimation; Extracting a region of interest to be searched using the estimated road information; And detecting an object existing on the road by searching the extracted ROI.

As described above, according to the present invention, when an error occurs in the current image while estimating the road information for detecting an object on the road, the posterior probability is calculated using the road information estimated from the previous image and the road information estimated from the current image The object detection speed and the detection performance are improved as the search area is reduced by searching for a specific object such as a pedestrian or a vehicle in the ROI accurately extracted from the road information estimated to be high.

In addition, the present invention can accurately extract a region of interest even when a stereo matching error occurs when processing left and right images photographed using a stereo vision camera or when there is a large loss of road information due to a front obstacle, It is possible to accurately detect a specific object by determining the presence or absence of the object using the classifier learned in advance.

Further, the present invention can be applied to the fields of an unmanned vehicle, a robot, and the like that can detect an object such as a front pedestrian or a vehicle, and estimate a road area without an obstacle by autonomous travel.

FIG. 1 is a diagram for explaining an object detection process using a conventional single camera.
FIG. 2 is a diagram showing a stereo matching error due to shielding in an object detection process using a conventional stereo camera.
3 is a diagram illustrating an example of road information loss caused by a forward obstacle in the object detection process using a conventional stereo camera.
4 is a block diagram showing an apparatus for detecting objects on a road according to an embodiment of the present invention.
5 is a flowchart illustrating an operational flow of a method for detecting an object on a road according to an embodiment of the present invention.
6 is a flowchart showing a detailed operation flow of a road information estimation step of a method for detecting an object on a road according to an embodiment of the present invention.
7 is a diagram for explaining a process of estimating stereo-based road information in a method for detecting an object on a road according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An apparatus and method for detecting an object on a road according to an embodiment of the present invention will be described with reference to the accompanying drawings. In this process, the thicknesses of the lines and the sizes of the components shown in the drawings may be exaggerated for clarity and convenience of explanation.

Further, the terms described below are defined in consideration of the functions of the present invention, which may vary depending on the intention or custom of the user, the operator. Therefore, definitions of these terms should be made based on the contents throughout this specification.

First, an apparatus for detecting an object on a road according to an embodiment of the present invention will be described with reference to FIG.

4 is a block diagram showing an apparatus for detecting objects on a road according to an embodiment of the present invention.

4, an apparatus 100 for detecting an object on a road according to an exemplary embodiment of the present invention includes a depth image generator 110, a road information estimator 120, a ROI extractor 130, And a detection unit 140.

The depth image generation unit 110 generates a depth image of a road from the left and right images of the road taken from a stereo camera (not shown).

In detail, the road gradient is extracted using the left and right images of the road taken from the stereo camera, and a depth image is generated using the stereo matching algorithm.

The road information estimating unit 120 estimates the road information using the depth image of the road generated by the depth image generating unit 110. If the road information estimating unit 120 determines that an error has occurred during the road information estimation, Thereby estimating road information.

More specifically, the depth image generated by the depth image generation unit 110 is accumulated on the y-axis to generate a v-disparity map. At this time, the road information is estimated using the Hough transformation algorithm or the RANSAC algorithm for the portion corresponding to the slope in the generated v-disparity map.

At this time, when it is determined that an error has occurred during the road information estimation, the road information estimating unit 120 estimates the road information by reflecting the road information estimated by the depth image of the road generated before the depth image of the generated road, The road information is updated and estimated by assigning weights according to the degree of similarity.

In this case, a case where the road information can not be estimated, such as an error in the stereo matching due to the partial shielding or a road information loss due to the presence of a large number of obstacle objects ahead, can be defined as an error occurrence determination element.

The ROI extracting unit 130 extracts a region of interest (ROI) to be searched using the road information estimated by the ROI estimating unit 120.

That is, the present invention has an effect of extracting a region of interest to be searched and improving object detection speed and detection performance as the search region is reduced.

At this time, the ROI extracting unit 130 may extract ROIs estimated by the ROI estimation unit 120 and ROIs using a stixel technique.

Here, the stick cell technique refers to a method of representing an image in the same group as a stick of a certain width instead of a pixel on the assumption that obstacle objects standing upright on the road have the same parallax information, May be extracted.

The object detecting unit 140 detects an object existing on the road by searching the ROI extracted by the ROI extracting unit 130.

In detail, the presence or absence of an object existing on the road is searched by searching the ROI, and if it is determined that the object exists, the type of the object corresponding to the object determined using the pre- .

At this time, the presence or absence of an object may be determined using a model learned by machine learning (for example, a classifier such as SVM (support vector machine), boosting, neural network, etc.).

For example, it is possible to determine whether an object exists by using a model learned by machine learning, and to detect the kind of an object such as a person and an automobile when the object exists.

Hereinafter, a method for detecting an object on a road according to an embodiment of the present invention will be described with reference to FIGs. 5 and 6. FIG.

FIG. 5 is a flowchart showing an operational flow of a method for detecting an object on a road according to an embodiment of the present invention, and a specific operation of the present invention will be described with reference to the flowchart.

According to the method for detecting an object on the road according to the embodiment of the present invention, the depth image generator 110 of the object detecting apparatus 100 generates a depth image of the road from the left and right images of the road taken from the stereo camera (S510).

6 is a diagram for explaining a process of estimating stereo-based road information in a method for detecting an object on a road according to an embodiment of the present invention.

More specifically, in step S510, the slope (a) of the road is extracted using the left and right images of the road taken from the stereo camera, and a depth image of the road is generated using the stereo matching algorithm.

Next, the road information estimating unit 120 estimates road information using the depth image of the road generated in step S510 (S520).

6 is a flowchart showing a detailed operation flow of a road information estimation step of a method for detecting an object on a road according to an embodiment of the present invention.

More specifically, as shown in FIG. 6, the depth image generated in step S510 is accumulated in the y-axis to generate a v-disparity map (S521). At this time, the road information is estimated using the Hough transformation algorithm or the RANSAC algorithm for the portion corresponding to the slope in the generated v-disparity map (S522).

If it is determined that an error has occurred during the road information estimation, the road information estimation unit 120 reflects the road information estimated by the depth image of the road generated before the depth image of the road generated in step S510, And estimates the road information by assigning weights according to the similarity to the observations.

Here, the case where the road information can not be estimated, such as the occurrence of an error in the stereo matching due to the partial shielding or the loss of the road information due to the presence of a large number of obstacle objects ahead, can be defined as an error occurrence determination element.

에 대하여, 통계적으로 빈도 수가 높은 N 개를 선택한다(S523). N 개의 선택된 변수

와 이전 상태 변수

에 의하여 사후 확률 즉, 갱신되는 도로 정보

을 아래의 수학식 1에서와 같이 계산하여, 현재의 상태 변수

를 결정한다(S524). In addition, the road information estimating unit 120 estimates the road information by using two observation variables

N, which are statistically higher in frequency, are selected (S523). N selected variables

And previous state variables

The posterior probability, that is, the road information to be updated

Is calculated as shown in the following equation (1), and the current state variable

(S524).

는 정규화 상수를 나타내고,

는 N개의 변수

와 현재 상태 변수

에 대한 사전 관측치인

와의 거리에 따른 가중치를 나타낸다. here,

Represents a normalization constant,

N variables

And current state variables

Pre-observations of

And a weight corresponding to the distance from the center.

That is, the current road information is updated by assigning a larger weight to the observation variables that are close to the previous observation value of the previous state variable (S525).

는 허프 변환에서 voting되는 빈도수

에 의하여 아래의 수학식 2와 같이 상대적 확률에 의해 계산된다. At this time,

Is the frequency of voting in the Hough transform

Is calculated by the relative probability as shown in Equation (2) below.

That is, it is determined that the slope a having a high frequency is highly reliable with respect to the slope a, and the road information is updated and estimated using the slope.

At this time, the updated road information can be modeled with a Gaussian probability, on the assumption that the road gradient does not change abruptly.

Then, the ROI extractor 130 extracts a region of interest (ROI) to be searched using the road information estimated in operation S520 in operation S530.

That is, the present invention has an effect of extracting a region of interest to be searched and improving object detection speed and detection performance as the search region is reduced.

At this time, the ROI extracting unit 130 may extract ROIs estimated by the ROI estimation unit 120 and ROIs using a stixel technique.

Here, the stick cell technique refers to a method of representing an image in the same group as a stick of a certain width instead of a pixel on the assumption that obstacle objects standing up on the road have the same parallax information. May be extracted.

Finally, the object detection unit 140 searches the ROI extracted by the ROI extraction unit 130 to detect an object existing on the road (S540).

In detail, the presence or absence of an object existing on the road is searched by searching the ROI, and if it is determined that the object exists, the type of the object corresponding to the object determined using the pre- .

At this time, the presence or absence of an object may be determined using a model learned by machine learning (for example, a classifier such as SVM (support vector machine), boosting, neural network, etc.).

For example, it is possible to determine whether an object exists by using a model learned by machine learning, and to detect the kind of an object such as a person and an automobile when the object exists.

As described above, an apparatus and method for detecting an object on a road according to an embodiment of the present invention can detect an object on the road when an error occurs in the current image while estimating road information for detecting an object on the road, By searching for a specific object such as a pedestrian or a vehicle in a region of interest accurately extracted from the road information estimated to have the highest posterior probability using the road information and the road information estimated from the current image, Speed and detection performance can be improved.

In addition, it is possible to accurately extract a region of interest even when a stereo matching error occurs when processing left and right images photographed using a stereo vision camera, or when road information is lost due to a front obstacle, The presence or absence of the object is determined by using the learned classifier, so that the specific object can be accurately detected.

In addition, the present invention can be applied to the fields of an unmanned vehicle, a robot, and the like that can detect an object such as a forward pedestrian or a vehicle and estimate a road area without an obstacle.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the scope of the invention as defined by the appended claims. will be. Accordingly, the true scope of the present invention should be determined by the following claims.

100: object detecting apparatus 110: depth image generating unit
120: Road information estimation unit 130: ROI extracting unit
140: Object detection unit 150:

Claims (10)

A depth image generator for generating a depth image of the road from a left and right image of a road taken from a stereo camera;
Estimating the road information by using the depth image of the generated road, and estimating the road information by reflecting previously estimated road information when it is determined that an error occurred during the road information estimation;
An interest region extractor for extracting a region of interest to be searched using the estimated road information; And
And an object detection unit searching the extracted ROI to detect an object existing on the road. The method according to claim 1,
The road information estimating unit estimates,
A v-disparity map is generated by accumulating in the y-axis using the generated depth image, and a portion corresponding to the gradient in the generated v-disparity map is transformed using a Hough transformation algorithm, The object detection apparatus comprising: 3. The method of claim 2,
The road information estimating unit estimates,
And estimating the road information by reflecting the road information estimated by the depth image of the road generated before the depth image of the generated road when the error is estimated to have occurred during the road information estimation. And updates the road information to estimate the road information. The method according to claim 1,
Wherein the ROI extractor comprises:
And extracts the ROI from the estimated ROI using a stixel technique that expresses the estimated road information and an image in a group of a predetermined width. The method according to claim 1,
Wherein the object detection unit comprises:
The method comprising the steps of: detecting an existence of an object existing on the road by searching the ROI, determining whether the object exists, detecting a type of the object corresponding to the object using the pre- The object detection apparatus comprising: A method for detecting an object on a road using an object detecting apparatus,
Generating a depth image of the road from left and right images of a road taken from a stereo camera;
Estimating the road information by using the depth image of the road; estimating the road information by reflecting previously estimated road information when an error occurs during the road information estimation;
Extracting a region of interest to be searched using the estimated road information; And
And detecting an object existing on the road by searching the extracted ROI. The method according to claim 6,
The step of estimating the road information includes:
A v-disparity map is generated by accumulating in the y-axis using the generated depth image, and a portion corresponding to the gradient in the generated v-disparity map is transformed using a Hough transformation algorithm, Of the object. 8. The method of claim 7,
The step of estimating the road information includes:
And estimating the road information by reflecting the road information estimated by the depth image of the road generated before the depth image of the generated road when the error is estimated to have occurred during the road information estimation. And estimates the road information by updating the road information. The method according to claim 6,
Wherein the extracting of the ROI comprises:
And extracting the ROI from the estimated ROI using a stixel technique that expresses the estimated road information and an image in a group of a predetermined width. The method according to claim 6,
Wherein detecting the object comprises:
The method comprising the steps of: detecting an existence of an object existing on the road by searching the ROI, determining whether the object exists, detecting a type of the object corresponding to the object using the pre- Object detection method.

Images