KR101724735B1 - An road analysis and object detection system using stereovision camera - Google Patents

Abstract

Disclosure of the Invention The present invention divides u-displacement and v-displacement information of an image acquired through a stereo camera of a vehicle into a plurality of virtual planes and divides the ground area and the object area based on the type of the image intersected on each divided plane The present invention relates to a ground estimation and object detection system using a stereo camera capable of more accurately recognizing a ground and an object from image information, and more particularly, to a system and method for generating a u-displacement image and a v-displacement image from a stereo image acquired from the stereo camera A displacement image generating means for generating displacement image; A ground area discriminating means for discriminating an area of the ground from the output image of the displacement image generating unit and determining the ground area to generate a function of the ground area; And detection means for separating the ground and object regions using the function of the ground region and the u-displacement image and the v-displacement image, and detecting the ground, the lane and the object.
According to the present invention having the above-described configuration, it is possible to distinguish the ground and the object from each other more accurately even when the vehicle is in an unstable state such as a shake during running, a roll or a tilt, Thereby enhancing the safety of the driver and increasing the convenience of the driver.

Description

BACKGROUND OF THE INVENTION Field of the Invention [0001] The present invention relates to an object detection and object detection system using a stereo camera,

The present invention relates to a ground surface estimation and object detection system using a stereo camera. More particularly, the present invention relates to a disparity image of an image acquired through a stereo camera mounted on a vehicle, And a technology for extracting an object on the ground by using it to more accurately recognize a ground and an object.

In order to judge the present driving situation such as the obstacle ahead of the vehicle or the departure of the lane in the running vehicle, and to secure the safe driving of the vehicle, it is necessary to estimate the ground through the car camera system and estimate the object displayed in front of the vehicle The process is absolutely necessary. In the case of a frontal collision warning system, the position of the object must be known to warn of the risk of running. In addition, in the case of the vehicle departure warning system, it is possible to improve the accuracy of lane detection if the ground area is found.

On the other hand, as the related art, the following contents are disclosed. In patent application No. 10-2009-0009337 (titled "Method and apparatus for extracting a ground from a stereo camera image and a recording medium on which a program embodying such a method is recorded" filed on Feb. 5, 2009) And a discriminator were used to extract the ground area. Based on this, the ground surface of the object was estimated on a one - dimensional plane, and the ground was detected in the remaining area. However, this is a model in which the ground surface is modeled in a one-dimensional plane shape, which can cope with the yaw of the vehicle, but can not cope with the roll or tilt of the vehicle. In the paper, Z.Hu and K.Uchimura (UV-Disparity: An efficient algorithm for stereovision based scene analysis), we extracted the ground, vehicle, and object using line extraction method on u-displacement and v- There is a problem in that it can not cope with roll and tilt of the vehicle although it can cope with rocking. In another paper (Hwang Jae-pil, Choi Hyuk-Du, Park Sung-Keun, Kim Eun-Tae, "Color Stereo Road Surface Extraction System for Intelligent Vehicle System"), we proposed a method of extracting the ground area through a stochastic approach. However, It was not an easy task, and there is no algorithm for detecting objects.

Disclosure of the Invention The present invention divides u-displacement and v-displacement information of an image acquired through a stereo camera of a vehicle into a plurality of virtual planes and divides the ground area and the object area based on the type of the image intersected on each divided plane The object of the present invention is to provide a ground estimation and object detection system using a stereo camera capable of more accurately recognizing the ground and objects from the image information.

According to an aspect of the present invention, there is provided a ground surface estimation and object detection system using a stereo camera, comprising: displacement image generation means for generating a u-displacement image and a v- displacement image from a stereo image acquired from the stereo camera; A ground area discriminating means for discriminating an area of the ground from the output image of the image generating means and determining a ground area to generate a function of the ground; and a function of the ground and using the u-displacement image and the v- And detecting means for detecting the ground, the lane, and the object by separating the ground and the object region.

In the present invention, the displacement image generating means includes a stacked u-displacement generator for generating a stacked u-displacement image from the u-displacement image and applying the stacked u-displacement image to the ground area discrimination means as an output image of the displacement image generation means, Displacement v-displacement image from the displacement image and applying the generated v-displacement image to the ground image discrimination means as an output image of the displacement image generation means.

The ground area discrimination means may include a ground area discriminator for discriminating a ground area from an output image of the displacement image generation unit and a ground area function generator for generating a ground area function based on the result of the ground area discriminator .

In the present invention, the ground area discriminator can select a ground candidate region from an image obtained by performing clustering on u-v displacement using the u-displacement image and the v-displacement image.

In the present invention, the detection means comprises a ground and object zone separator for separating the ground and object zones using the stereo image, the function of the ground, and the u-displacement image and the v-displacement image, The object region separator obtains a plurality of planes on the ground and a vertical axis from the stereo image along the height and obtains a portion of the u-displacement image and the v-displacement image cut by the plurality of planes, The object is set as an area of interest of the object and the area excluding the cut part is assumed as the ground.

According to the present invention having the above-described configuration, it has an effect in terms of a countermeasure against vehicle rocking, an increase in safety, and an increase in convenience for the driver.

First, in the aspect of vehicle shaking, the conventional method estimates the ground surface as a one-dimensional function. Therefore, it is impossible to accurately estimate the ground surface when the area of the ground surface is distorted due to roll or tilt of the vehicle. However, the method can cope with the unstable situation by estimating the ground surface with the plane function, so that accurate ground surface estimation and object detection are possible.

In addition, since the separation performance between the ground and the object is improved by using the ground area discriminator, the ground surface area function generator, and the ground surface and the object area separator in the present invention in terms of increasing the safety of the driver, It is possible to provide accurate information and, when the occurrence of a dangerous situation is anticipated, provides an accurate information that can generate an alarm, thereby providing safety of the driver.

Lastly, in the present invention, since the separation performance between the ground and the object is improved by using the ground area discriminator, the ground area function generator, and the ground and object area separator, the present invention provides the driver with accurate information about the surrounding situation So that it is possible to provide information that can be helpful to the driver during driving, thereby enhancing the driver's driving convenience.

1 is a block diagram of a ground surface estimation and object detection system using a stereo camera according to an embodiment of the present invention.
FIGS. 2 to 15 are views illustrating an exemplary image for explaining a ground surface estimation method and an object detection method using a stereo camera according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

1 is a block diagram of a ground estimation and object detection system using a stereo camera according to an embodiment of the present invention.

1, the ground surface estimation and object detection system according to an embodiment of the present invention includes a stereo camera 11, a displacement image generator 12, a stacked u-displacement generator 13, a stacked v- A ground surface area function generator 16, a ground and object zone separator 17, a ground and lane detector 18, and an object detector 19, as shown in FIG.

1, the stereo camera 11 is installed, for example, on a rear surface of a rearview mirror of a vehicle, and captures a front and a road surface of the vehicle and acquires a stereo image therefrom.

The displacement image generator 12 generates a u-displacement image and a v-displacement image using the stereo image acquired by the stereo camera 11.

The stacked u-displacement generator 13 generates a stacked u-displacement image from the u-displacement image generated by the displacement image generator 12, and the stacked v-displacement generator 14 generates a stacked u- And generates a stacked v-displacement image from the generated v-displacement image.

The ground area discriminator 15 discriminates between the stereo image obtained in the stereo camera 11, the stacked u-displacement image generated in the stacked u-displacement generator 13 and the stacked u- displacement image generated in the stacked v- and performs a function of discriminating the ground area from the v-displacement image.

The ground area function generator 16 performs the function of generating a function of the ground area based on the result of the ground area discriminator 15 and the ground and object area separator 17 generates the ground area function by the displacement image generator 12 And separates the stereo image obtained by the stereo camera 11 into an object area and a ground area using the land area function generated by the land area function generator 16. [

On the other hand, the ground and lane detector 18 performs the function of extracting the ground information and the lane information from the ground area image information inputted from the ground and the object zone separator 17, and the object detector 19 detects the ground and the object zone And detects and identifies an object in the object region image information input from the separator 17.

Next, the operation of the apparatus having the above-described configuration will be described with reference to FIGS. 2 to 15. FIG.

The left image in FIGS. 2 and 3 represents a virtual image, and the right image represents a screen when actually applied. An image acquired by the stereo camera 11 is shown in Fig. The disparity occurs in the image captured by the left / right camera due to the parallax according to the imaging position and angle of the camera. The displacement image generator 12 obtains a displacement for each pixel, and a disparity map corresponding to the obtained image is shown in FIG. In a hypothetical image, a bright region is a region with small displacement. In an actual experimental image, a region close to green indicates a region having a large displacement and a blue region indicates a region having a small displacement. The number of pixels having the same u-displacement value from the image information of FIG. 3 is measured and the image shown in the u-displacement plane is as shown in FIG. 4. The number of pixels having the same v- Is shown in Fig.

On the other hand, an image as shown in Fig. 4 is generated by a stacked u-displacement generator 13, and an image as shown in Fig. 5 is generated by a stacked V-displacement generator 14. Fig.

Then, the ground area discriminator 15 filters and passes the u-displacement value of the image shown in Fig. 4 and the v-displacement value of the image shown in Fig. 5 as feature values of the uv point for the displacement image , And clustering the result with respect to the uv-displacement, the area is divided. 6 is selected as the ground candidate region, the left side of FIG. 6 shows the result of the virtual image, and the left side of FIG. The right side shows the actual applied result.

The surface area function generator 16 estimates a two-dimensional plane equation of the ground through a regression algorithm using the u-v-displacement value of the ground area as an input value. The resulting floor candidate area is shown in FIG.

The ground area and the object area separator 17 separate the ground and the object using the following method.

As shown in Fig. 8, assuming a plurality of virtual planes (planes 1 to 4) with respect to the front view of the vehicle and cutting the image captured by the camera in this virtual plane onto the ground plane on the vertical axis, As shown in FIG. 9 on the image. 10 to 13 are obtained by dividing the plane image (plane, plane 1, plane 2, plane 3, plane 4) shown in Fig. 9 by the displacement image. Fig. 10 is an image region cut by plane 1, Fig. 11 is an image region cut by plane 2, Fig. 12 is an image region cut by plane 3, and Fig. When the image information of the object (for example, an object) is cut off according to the respective heights, it is set as an area of interest of the object. Then, the area excluding the cut-off part assumes the ground. The object detector 19 detects and judges dangerous objects on the ground, and the ground and lane detector 18 detects lanes on the ground.

That is, according to the embodiment, a plurality of virtual planes are assumed with respect to the front view of the vehicle, and the ground area and the object area can be precisely separated based on the u-displacement image and the v-displacement image in this plane do.

Fig. 14 is an image showing an area where an object is present, and Fig. 15 is a result of an actual implementation. In FIG. 15, reference numeral 1501 denotes an original image, reference numeral 1502 denotes a ground area detected using the present method, and reference numeral 1503 denotes a ground detection result using the estimation method according to the present invention.

As described above, according to the present invention, a ground plane is inferred through a post-processing step on a displacement image of an image acquired through a stereo camera mounted on a vehicle, and an object on the ground is extracted using the thus- And provides the driver with information on the surrounding environment, thereby enhancing the safety of the driver and increasing the convenience of the driver.

11: Stereo camera 12: Displacement image generator
13: stacked u-displacement generator 14: stacked v-displacement generator
15: ground area discriminator 16: ground area function generator
17: ground and object zone separator 18: ground and lane detector
19: object detector

Claims (5)

In a ground surface estimation and object detection system using a stereo camera,
Displacement image generating means for generating an u-displacement image and a v-displacement image from the stereo image acquired from the stereo camera;
A ground area discriminating means for discriminating an area of the ground from the output image of the displacement image generating unit and determining the ground area to generate a function of the ground area;
And detection means for separating the ground and object regions using the function of the ground region and the u-displacement image and the v-displacement image, and detecting the ground, the lane and the object,
The detection means
Wherein a plurality of planes on the ground and a vertical axis are obtained from the stereo image along a height, and the ground area and the object area are separated based on the u-displacement image and the v-displacement image in the plane. Useful Ground Estimation and Object Detection System. The image processing apparatus according to claim 1, wherein the displacement image generating means
A stacked u-displacement generator for generating a stacked u-displacement image from the u-displacement image and applying the generated stacked u-displacement image to the ground area discrimination means; And
And a stacked v-displacement generator for generating a stacked v-displacement image from the v-displacement image and applying the stacked v-displacement image to the ground area discrimination means. [2] The apparatus according to claim 1,
A ground area discriminator for discriminating a ground area from an output image of the displacement image generating unit;
And a ground area function generator for generating a function of a ground area based on a result of the ground area discriminator. 4. The apparatus of claim 3, wherein the ground-
Wherein the clustering is performed for the uv displacement using the u-displacement image and the v-displacement image, and a ground candidate region is selected from the acquired image. system. [2] The apparatus according to claim 1,
And a ground and object zone separator for separating the ground and object zones using the function of the ground zone and the u-displacement image and the v-displacement image,
The ground and object zone separator comprises:
A plurality of planes on the ground and a vertical axis from the stereo image are obtained along a height and a portion where the u-displacement image and the v-displacement image are cut are obtained by the plurality of planes, Wherein when the predetermined object is cut, the object is set as an area of interest of the object, and the area excluding the cut-off part is separated into the ground.

Images